IPTV data reduction strategy to measure real users’ behaviours

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Development of data reduction pipelines for GTC instruments at the UCM

The UCM Instrumentation Group (GUAIX) is developing currently Data Reduction Pipelines (DRP) for four instruments of the GTC: EMIR, FRIDA, MEGARA and MIRADAS. The purpose of the DRPs is to provide astronomers scientific quality data, removing instrumental biases, calibrating the images in physical units and providing a estimation of the associated uncertainties.

Data reduction pipeline for emir, the near-IR multi-object spectrograph for GTC

EMIR (Balcells et al. 2000) is a near-infrared wide-field camera and multi-object spectrograph being built for the GTC. The Data Reduction Pipeline (DRP) will be optimized for handling and reducing near-infrared data acquired with EMIR.

Mobile parametric measurement device. Volume III: data reduction procedures. Users manual.

National Highway Traffic Safety Administration, Washington, D.C.

Automation of an ion chromatograph for precipitation analysis with computerized data reduction /

U.S. Geological Survey, Water Resources Division"--P. [2] of cover.

Data Reduction in Intrusion Alert Correlation

Network intrusion detection sensors are usually built around low level models of network traffic. This means that their output is of a similarly low level and as a consequence, is difficult to analyze. Intrusion alert correlation is the task of automating some of this analysis by grouping related alerts together. Attack graphs provide an intuitive model for such analysis. Unfortunately alert flooding attacks can still cause a loss of service on sensors, and when performing attack graph correlation, there can be a large number of extraneous alerts included in the output graph. This obscures the fine structure of genuine attacks and makes them more difficult for human operators to discern. This paper explores modified correlation algorithms which attempt to minimize the impact of this attack.

DEVELOPMENT OF AN OFF-LINE RAINFLOW COUNTING ALGORITHM WITH TEMPORAL PARAMETERS AND DATA REDUCTION

Rainflow counting methods convert a complex load time history into a set of load reversals for use in fatigue damage modeling. Rainflow counting methods were originally developed to assess fatigue damage associated with mechanical cycling where creep of the material under load was not considered to be a significant contributor to failure. However, creep is a significant factor in some cyclic loading cases such as solder interconnects under temperature cycling. In this case, fatigue life models require the dwell time to account for stress relaxation and creep. This study develops a new version of the multi-parameter rainflow counting algorithm that provides a range-based dwell time estimation for use with time-dependent fatigue damage models. To show the applicability, the method is used to calculate the life of solder joints under a complex thermal cycling regime and is verified by experimental testing. An additional algorithm is developed in this study to provide data reduction in the results of the rainflow counting. This algorithm uses a damage model and a statistical test to determine which of the resultant cycles are statistically insignificant to a given confidence level. This makes the resulting data file to be smaller, and for a simplified load history to be reconstructed.

Electropalatographic assessment of tongue-to-palate contacts exhibited in dysarthria following traumatic brain injury: Spatial characteristics

Consonant imprecision has been reported to be a common feature of the dysarthric speech disturbances exhibited by individuals who have sustained a traumatic brain injury (TBI). Inaccurate tongue placements against the hard palate during consonant articulation may be one factor underlying the imprecision. To investigate this hypothesis, electropalatography (EPG) was used to assess the spatial characteristics of the tongue-to-palate contacts exhibited by three males (aged 23-29 years) with dysarthria following severe TBI. Five nonneurologically impaired adults served as control subjects. Twelve single-syllable words of CV or CVC construction (where initial C = /t, d, S, z, k, g/, V=/i, a/) were read aloud three times by each subject while wearing an EPG palate. Spatial characteristics were analyzed in terms of the location, pattern, and amount of tongue-to-palate contact at the frame of maximum contact during production of each consonant. The results revealed that for the majority of consonants, the patterns and locations of contacts exhibited by the TBI subjects were consistent with the contacts generated by the group of control subjects. One notable exception was one subject's production of the alveolar fricatives in which complete closure across the palate was demonstrated, rather than the characteristic groove configuration. Major discrepancies were also noted in relation to the amount of tongue-to-palate contact exhibited, with two TBI subjects consistently demonstrating increased contacts compared to the control subjects. The implications of these findings for the development of treatment programs for dysarthric speech disorders subsequent to TBI are highlighted.

Histology-driven data mining of lipid signatures from multiple imaging mass spectrometry analyses: application to human colorectal cancer liver metastasis biopsies.

Imaging mass spectrometry (IMS) represents an innovative tool in the cancer research pipeline, which is increasingly being used in clinical and pharmaceutical applications. The unique properties of the technique, especially the amount of data generated, make the handling of data from multiple IMS acquisitions challenging. This work presents a histology-driven IMS approach aiming to identify discriminant lipid signatures from the simultaneous mining of IMS data sets from multiple samples. The feasibility of the developed workflow is evaluated on a set of three human colorectal cancer liver metastasis (CRCLM) tissue sections. Lipid IMS on tissue sections was performed using MALDI-TOF/TOF MS in both negative and positive ionization modes after 1,5-diaminonaphthalene matrix deposition by sublimation. The combination of both positive and negative acquisition results was performed during data mining to simplify the process and interrogate a larger lipidome into a single analysis. To reduce the complexity of the IMS data sets, a sub data set was generated by randomly selecting a fixed number of spectra from a histologically defined region of interest, resulting in a 10-fold data reduction. Principal component analysis confirmed that the molecular selectivity of the regions of interest is maintained after data reduction. Partial least-squares and heat map analyses demonstrated a selective signature of the CRCLM, revealing lipids that are significantly up- and down-regulated in the tumor region. This comprehensive approach is thus of interest for defining disease signatures directly from IMS data sets by the use of combinatory data mining, opening novel routes of investigation for addressing the demands of the clinical setting.

Evaluating the Relationship between the Driver and Roadway to Address Rural Intersection Safety using the SHRP 2 Naturalistic Driving Study Data, RB05-013, 2016

Rural intersections account for 30% of crashes in rural areas and 6% of all fatal crashes, representing a significant but poorly understood safety problem. Transportation agencies have traditionally implemented countermeasures to address rural intersection crashes but frequently do not understand the dynamic interaction between the driver and roadway and the driver factors leading to these types of crashes. The Second Strategic Highway Research Program (SHRP 2) conducted a large-scale naturalistic driving study (NDS) using instrumented vehicles. The study has provided a significant amount of on-road driving data for a range of drivers. The present study utilizes the SHRP 2 NDS data as well as SHRP 2 Roadway Information Database (RID) data to observe driver behavior at rural intersections first hand using video, vehicle kinematics, and roadway data to determine how roadway, driver, environmental, and vehicle factors interact to affect driver safety at rural intersections. A model of driver braking behavior was developed using a dataset of vehicle activity traces for several rural stop-controlled intersections. The model was developed using the point at which a driver reacts to the upcoming intersection by initiating braking as its dependent variable, with the driver’s age, type and direction of turning movement, and countermeasure presence as independent variables. Countermeasures such as on-pavement signing and overhead flashing beacons were found to increase the braking point distance, a finding that provides insight into the countermeasures’ effect on safety at rural intersections. The results of this model can lead to better roadway design, more informed selection of traffic control and countermeasures, and targeted information that can inform policy decisions. Additionally, a model of gap acceptance was attempted but was ultimately not developed due to the small size of the dataset. However, a protocol for data reduction for a gap acceptance model was determined. This protocol can be utilized in future studies to develop a gap acceptance model that would provide additional insight into the roadway, vehicle, environmental, and driver factors that play a role in whether a driver accepts or rejects a gap.

Assimilation of Geosat Altimeter Data for the Agulhas Current Using the Ensemble Kalman Filter with a Quasigeostrophic Model

The ring-shedding process in the Agulhas Current is studied using the ensemble Kalman filter to assimilate geosat altimeter data into a two-layer quasigeostrophic ocean model. The properties of the ensemble Kalman filter are further explored with focus on the analysis scheme and the use of gridded data. The Geosat data consist of 10 fields of gridded sea-surface height anomalies separated 10 days apart that are added to a climatic mean field. This corresponds to a huge number of data values, and a data reduction scheme must be applied to increase the efficiency of the analysis procedure. Further, it is illustrated how one can resolve the rank problem occurring when a too large dataset or a small ensemble is used.

Two-Phase Mapping for Projecting Massive Data Sets

Most multidimensional projection techniques rely on distance (dissimilarity) information between data instances to embed high-dimensional data into a visual space. When data are endowed with Cartesian coordinates, an extra computational effort is necessary to compute the needed distances, making multidimensional projection prohibitive in applications dealing with interactivity and massive data. The novel multidimensional projection technique proposed in this work, called Part-Linear Multidimensional Projection (PLMP), has been tailored to handle multivariate data represented in Cartesian high-dimensional spaces, requiring only distance information between pairs of representative samples. This characteristic renders PLMP faster than previous methods when processing large data sets while still being competitive in terms of precision. Moreover, knowing the range of variation for data instances in the high-dimensional space, we can make PLMP a truly streaming data projection technique, a trait absent in previous methods.

Conceptual model for adaptable and extensible visual data exploration

Interactive visual representations complement traditional statistical and machine learning techniques for data analysis, allowing users to play a more active role in a knowledge discovery process and making the whole process more understandable. Though visual representations are applicable to several stages of the knowledge discovery process, a common use of visualization is in the initial stages to explore and organize a sometimes unknown and complex data set. In this context, the integrated and coordinated - that is, user actions should be capable of affecting multiple visualizations when desired - use of multiple graphical representations allows data to be observed from several perspectives and offers richer information than isolated representations. In this paper we propose an underlying model for an extensible and adaptable environment that allows independently developed visualization components to be gradually integrated into a user configured knowledge discovery application. Because a major requirement when using multiple visual techniques is the ability to link amongst them, so that user actions executed on a representation propagate to others if desired, the model also allows runtime configuration of coordinated user actions over different visual representations. We illustrate how this environment is being used to assist data exploration and organization in a climate classification problem.