A big data platform for large scale event processing

To date, big data applications have focused on the store-and-process paradigm. In this paper we describe an initiative to deal with big data applications for continuous streams of events. In many emerging applications, the volume of data being streamed is so large that the traditional ‘store-then-process’ paradigm is either not suitable or too inefficient. Moreover, soft-real time requirements might severely limit the engineering solutions. Many scenarios fit this description. In network security for cloud data centres, for instance, very high volumes of IP packets and events from sensors at firewalls, network switches and routers and servers need to be analyzed and should detect attacks in minimal time, in order to limit the effect of the malicious activity over the IT infrastructure. Similarly, in the fraud department of a credit card company, payment requests should be processed online and need to be processed as quickly as possible in order to provide meaningful results in real-time. An ideal system would detect fraud during the authorization process that lasts hundreds of milliseconds and deny the payment authorization, minimizing the damage to the user and the credit card company.

Data Fusion based on Game Theory for Speaker Diarization

A novel algorithm based on bimatrix game theory has been developed to improve the accuracy and reliability of a speaker diarization system. This algorithm fuses the output data of two open-source speaker diarization programs, LIUM and SHoUT, taking advantage of the best properties of each one. The performance of this new system has been tested by means of audio streams from several movies. From preliminary results on fragments of five movies, improvements of 63% in false alarms and missed speech mistakes have been achieved with respect to LIUM and SHoUT systems working alone. Moreover, we also improve in a 20% the number of recognized speakers, getting close to the real number of speakers in the audio stream

TrendMiner: Large-scale Cross-lingual Trend Mining Summarization of Real-time Media Streams

El reciente crecimiento masivo de medios on-line y el incremento de los contenidos generados por los usuarios (por ejemplo, weblogs, Twitter, Facebook) plantea retos en el acceso e interpretación de datos multilingües de manera eficiente, rápida y asequible. El objetivo del proyecto TredMiner es desarrollar métodos innovadores, portables, de código abierto y que funcionen en tiempo real para generación de resúmenes y minería cross-lingüe de medios sociales a gran escala. Los resultados se están validando en tres casos de uso: soporte a la decisión en el dominio financiero (con analistas, empresarios, reguladores y economistas), monitorización y análisis político (con periodistas, economistas y políticos) y monitorización de medios sociales sobre salud con el fin de detectar información sobre efectos adversos a medicamentos.

Sediment echosounding data files from the Amundsen Sea

The flow of ice streams, which account for most discharge from large ice sheets, is controlled by processes operating at their bed. Data from modern ice stream beds are difficult to obtain, but where ice advanced onto continental shelves during glacial periods extensive areas of the former bed can be imaged using modern swath sonar tools. We present new multibeam swath bathymetry data analyzed alongside sparse pre-existing data from the Amundsen Sea Embayment. The compilation is the most extensive, continuous area of multibeam data coverage yet obtained on the inner continental shelf of Antarctica. The data reveal streamlined subglacial bedforms that define a zone of paleo-ice stream convergence but, in contrast to previous models, do not show a simple down-flow progression of bedform types along paleo-ice stream troughs. We interpret high spatial variability of bedforms as indicating a complex mechanical and hydrodynamic regime at the former ice stream beds, consistent with observations from some modern ice streams. We conclude that care must be taken when using bedforms to infer paleo-ice stream velocities.

An intensive survey of the Vermilion River Basin (Wabash River drainage) 1997 : data collection methods and summary /

"The Illinois Environmental Protection Agency monitors surface waters (i.e. lakes and streams) through a variety of programs. The most extensive is the Ambient Water Quality Monitoring Network (AWQMN) which consists of 203 stream stations statewide sampled on a 6 week cycle since October 1977." -- p. 1.

Data collection methods and summaries for the assessment of stream water quality in the Vermilion River Basin (Wabash River), 2001 /

Cover title.

Surface water pesticide monitoring of Illinois streams : summary of results 1985-1998 /

This report provides a review and summarization of stream pesticide data collected by the Illinois EPA Ambient Water Quality Monitoring Network between October 1985 and December 1998. A list of sampling stations is provided in Appendix A along with data from the seven most detected pesticides which are provided in Appendix B.

Water chemistry controls on minor elemental concentrations and strontium isotopic compositions of freshwater biogenic carbonate in northeastern Washington streams

Tracking the movement of migratory freshwater fish is essential to those invested in rebuilding declining fish populations. Using strontium isotopic signatures to match calcified fish tissues to streams where fish spawn is a useful method of tracking migratory fish where physical tracking methods such as radio, acoustic, or external tags, have proven unsuccessful. In this study, we develop tools to practice this method of tracking fish in Lake Roosevelt and its upstream tributaries in Washington State by analyzing the elemental concentrations and 87Sr/86Sr ratios of water samples, and mussel shell samples. This study evaluates whether mussel shells act as an appropriate proxy for water chemistry by comparing the 87Sr/86Sr isotope ratios of water samples to the 87Sr/86Sr isotope ratios of mussel shells sampled from the same, or nearby, locations. We compare concentrations of Ba, Ca, Cd, Cu, Fe, Mg, Pb, Sr, and U in the water and mussel shell samples to determine the feasibility of using mussel shells as a proxy for water chemistry. If it is determined that the concentrations of these elements in mussel shells reflect that of the surrounding water composition, the elemental composition of mussel shells can be compared to that of calcified tissues in fish, such as otoliths, to infer the location of the natal stream. We report analyses of water and mussel shell samples collected from Lake Roosevelt, Sanpoil River, Spokane River, Colville River, Kettle River, Pend Oreille River, Kootenay River, and Columbia River in Washington State. Each of these rivers is a tributary to Lake Roosevelt, and each flows through different geologic units. We hypothesize that the differences in the rock units of each stream’s watershed are reflected in the elemental concentrations and strontium isotopic ratios of water in each stream and in the lake. We also hypothesize that the composition of the mussel shells will match the composition of the water samples, therefore allowing us to use the mussel shells as a proxy for local water chemistry. Additionally, we hypothesize that the composition of the mussel shells will vary by location, and that we will be able to then infer where a fish is from by matching the composition of the fish in question to the mussels we have analyzed. We found that 87Sr/86Sr values for water and mussel hinge samples collected from tributaries east of Lake Roosevelt are significantly higher than the 87Sr/86Sr values for samples collected from tributaries west of Lake Roosevelt with averages of 0.7235 and 0.7089, respectively. The average 87Sr/86Sr ratios for water and mussel hinge samples collected within Lake Roosevelt is 0.7158, which is between the averages for samples collected east and west of the lake. Generally, older rocks are exposed on the east side of the lake, and younger rocks on the west side of the lake, so our 87Sr/86Sr values support the hypothesis that geologic units are a primary control on water chemistry, and that tributary compositions mix to form an average weighed by flow in Lake Roosevelt. The 87Sr/86Sr values for water and mussel shell samples collected from the same locations have a strong, positive linear correlation, suggesting that mussel shell 87Sr/86Sr ratios reflect the 87Sr/86Sr ratios of the ambient water. With these data, we can distinguish between different streams and the lake, but cannot distinguish between samples from within the same stream or within Lake Roosevelt. The Sr:Ca and Fe:Ca ratios of water samples show positive correlations with mussel shell compositions, with R2 values of 0.82 and 0.52, respectively. Ratios of Mg, Ba, Cu, Cd, Pb, and U to Ca showed little or no positive correlation between water and mussel shell samples. The elemental concentration data collected for this study do not demonstrate whether a correlation between elemental ratios in water samples and elemental ratios in mussel shell samples collected from the same location exists. Positive Sr:Ca and Fe:Ca correlations for water versus mussel shell samples indicate that perhaps for some elements, the composition of mussel shells are representative of the composition of ambient water. Using elemental concentration ratios to complement 87Sr/86Sr isotopic data may enhance our ability to identify correlations between water and mussel shell samples, and ultimately between mussel shell and otolith samples. The hinge part of a mussel shell may be used as a proxy for local water composition because the mussel shell composition reflects that of the local ambient water. The hinge of the mussel has the same composition as the whole mussel shell. We measured variation of 87Sr/86Sr ratios in the water among different streams and Lake Roosevelt. The 87Sr/86Sr values for samples collected in tributaries east of Lake Roosevelt, which erode older rocks, are higher for mussel shell and water samples than the average 87Sr/86Sr values for mussel shell and water samples collected in tributaries west of Lake Roosevelt, which flow through younger rocks.

Matching commercial clips from TV streams using a unique, robust and compact signature

One of critical challenges in automatic recognition of TV commercials is to generate a unique, robust and compact signature. Uniqueness indicates the ability to identify the similarity among the commercial video clips which may have slight content variation. Robustness means the ability to match commercial video clips containing the same content but probably with different digitalization/encoding, some noise data, and/or transmission and recording distortion. Efficiency is about the capability of effectively matching commercial video sequences with a low computation cost and storage overhead. In this paper, we present a binary signature based method, which meets all the three criteria above, by combining the techniques of ordinal and color measurements. Experimental results on a real large commercial video database show that our novel approach delivers a significantly better performance comparing to the existing methods.

Surprises and Insights from Long-Term Aquatic Data Sets and Experiments

long-term research on freshwater ecosystems provides insights that can be difficult to obtain from other approaches. Widespread monitoring of ecologically relevant water-quality parameters spanning decades can facilitate important tests of ecological principles. Unique long-term data sets and analytical tools are increasingly available, allowing for powerful and synthetic analyses across sites. long-term measurements or experiments in aquatic systems can catch rare events, changes in highly variable systems, time-lagged responses, cumulative effects of stressors, and biotic responses that encompass multiple generations. Data are available from formal networks, local to international agencies, private organizations, various institutions, and paleontological and historic records; brief literature surveys suggest much existing data are not synthesized. Ecological sciences will benefit from careful maintenance and analyses of existing long-term programs, and subsequent insights can aid in the design of effective future long-term experimental and observational efforts. long-term research on freshwaters is particularly important because of their value to humanity.

Ensemble Stream Model for Data-Cleaning in Sensor Networks

Ensemble Stream Modeling and Data-cleaning are sensor information processing systems have different training and testing methods by which their goals are cross-validated. This research examines a mechanism, which seeks to extract novel patterns by generating ensembles from data. The main goal of label-less stream processing is to process the sensed events to eliminate the noises that are uncorrelated, and choose the most likely model without over fitting thus obtaining higher model confidence. Higher quality streams can be realized by combining many short streams into an ensemble which has the desired quality. The framework for the investigation is an existing data mining tool. First, to accommodate feature extraction such as a bush or natural forest-fire event we make an assumption of the burnt area (BA*), sensed ground truth as our target variable obtained from logs. Even though this is an obvious model choice the results are disappointing. The reasons for this are two: One, the histogram of fire activity is highly skewed. Two, the measured sensor parameters are highly correlated. Since using non descriptive features does not yield good results, we resort to temporal features. By doing so we carefully eliminate the averaging effects; the resulting histogram is more satisfactory and conceptual knowledge is learned from sensor streams. Second is the process of feature induction by cross-validating attributes with single or multi-target variables to minimize training error. We use F-measure score, which combines precision and accuracy to determine the false alarm rate of fire events. The multi-target data-cleaning trees use information purity of the target leaf-nodes to learn higher order features. A sensitive variance measure such as ƒ-test is performed during each node's split to select the best attribute. Ensemble stream model approach proved to improve when using complicated features with a simpler tree classifier. The ensemble framework for data-cleaning and the enhancements to quantify quality of fitness (30% spatial, 10% temporal, and 90% mobility reduction) of sensor led to the formation of streams for sensor-enabled applications. Which further motivates the novelty of stream quality labeling and its importance in solving vast amounts of real-time mobile streams generated today.