Anomaly detection from time-changing environmental sensor data streams

This thesis stems from the project with real-time environmental monitoring company EMSAT Corporation. They were looking for methods to automatically ag spikes and other anomalies in their environmental sensor data streams. The problem presents several challenges: near real-time anomaly detection, absence of labeled data and time-changing data streams. Here, we address this problem using both a statistical parametric approach as well as a non-parametric approach like Kernel Density Estimation (KDE). The main contribution of this thesis is extending the KDE to work more effectively for evolving data streams, particularly in presence of concept drift. To address that, we have developed a framework for integrating Adaptive Windowing (ADWIN) change detection algorithm with KDE. We have tested this approach on several real world data sets and received positive feedback from our industry collaborator. Some results appearing in this thesis have been presented at ECML PKDD 2015 Doctoral Consortium.

Using Image Processing Methods to Improve the Detection of Buried Explosive Threats in GPR Data

Current state of the art techniques for landmine detection in ground penetrating radar (GPR) utilize statistical methods to identify characteristics of a landmine response. This research makes use of 2-D slices of data in which subsurface landmine responses have hyperbolic shapes. Various methods from the field of visual image processing are adapted to the 2-D GPR data, producing superior landmine detection results. This research goes on to develop a physics-based GPR augmentation method motivated by current advances in visual object detection. This GPR specific augmentation is used to mitigate issues caused by insufficient training sets. This work shows that augmentation improves detection performance under training conditions that are normally very difficult. Finally, this work introduces the use of convolutional neural networks as a method to learn feature extraction parameters. These learned convolutional features outperform hand-designed features in GPR detection tasks. This work presents a number of methods, both borrowed from and motivated by the substantial work in visual image processing. The methods developed and presented in this work show an improvement in overall detection performance and introduce a method to improve the robustness of statistical classification.

A field experiment on a steel Gerber-truss bridge for damage detection utilizing vehicle-induced vibrations

A field experiment was conducted on a real continuous steel Gerber-truss bridge with artificial damage applied. This article summarizes the results of the experiment for bridge damage detection utilizing traffic-induced vibrations. It investigates the sensitivities of a number of quantities to bridge damage including the identified modal parameters and their statistical patterns, Nair’s damage indicator and its statistical pattern and different sets of measurement points. The modal parameters are identified by autoregressive time-series models. The decision on bridge health condition is made and the sensitivity of variables is evaluated with the aid of the Mahalanobis–Taguchi system, a multivariate pattern recognition tool. Several observations are made as follows. For the modal parameters, although bridge damage detection can be achieved by performing Mahalanobis–Taguchi system on certain modal parameters of certain sets of measurement points, difficulties were faced in subjective selection of meaningful bridge modes and low sensitivity of the statistical pattern of the modal parameters to damage. For Nair’s damage indicator, bridge damage detection could be achieved by performing Mahalanobis–Taguchi system on Nair’s damage indicators of most sets of measurement points. As a damage indicator, Nair’s damage indicator was superior to the modal parameters. Three main advantages were observed: it does not require any subjective decision in calculating Nair’s damage indicator, thus potential human errors can be prevented and an automatic detection task can be achieved; its statistical pattern has high sensitivity to damage and, finally, it is flexible regarding the choice of sets of measurement points.

Convex Optimization Algorithms and Statistical Bounds for Learning Structured Models

Thesis (Ph.D.)--University of Washington, 2016-08

Phenobarbital reduces EEG amplitude and propagation of neonatal seizures but does not alter performance of automated seizure detection

Objective: Phenobarbital increases electroclinical uncoupling and our preliminary observations suggest it may also affect electrographic seizure morphology. This may alter the performance of a novel seizure detection algorithm (SDA) developed by our group. The objectives of this study were to compare the morphology of seizures before and after phenobarbital administration in neonates and to determine the effect of any changes on automated seizure detection rates. Methods: The EEGs of 18 term neonates with seizures both pre- and post-phenobarbital (524 seizures) administration were studied. Ten features of seizures were manually quantified and summary measures for each neonate were statistically compared between pre- and post-phenobarbital seizures. SDA seizure detection rates were also compared. Results: Post-phenobarbital seizures showed significantly lower amplitude (p < 0.001) and involved fewer EEG channels at the peak of seizure (p < 0.05). No other features or SDA detection rates showed a statistical difference. Conclusion: These findings show that phenobarbital reduces both the amplitude and propagation of seizures which may help to explain electroclinical uncoupling of seizures. The seizure detection rate of the algorithm was unaffected by these changes. Significance: The results suggest that users should not need to adjust the SDA sensitivity threshold after phenobarbital administration.

Anomaly Detection in Noisy Images

Finding rare events in multidimensional data is an important detection problem that has applications in many fields, such as risk estimation in insurance industry, finance, flood prediction, medical diagnosis, quality assurance, security, or safety in transportation. The occurrence of such anomalies is so infrequent that there is usually not enough training data to learn an accurate statistical model of the anomaly class. In some cases, such events may have never been observed, so the only information that is available is a set of normal samples and an assumed pairwise similarity function. Such metric may only be known up to a certain number of unspecified parameters, which would either need to be learned from training data, or fixed by a domain expert. Sometimes, the anomalous condition may be formulated algebraically, such as a measure exceeding a predefined threshold, but nuisance variables may complicate the estimation of such a measure. Change detection methods used in time series analysis are not easily extendable to the multidimensional case, where discontinuities are not localized to a single point. On the other hand, in higher dimensions, data exhibits more complex interdependencies, and there is redundancy that could be exploited to adaptively model the normal data. In the first part of this dissertation, we review the theoretical framework for anomaly detection in images and previous anomaly detection work done in the context of crack detection and detection of anomalous components in railway tracks. In the second part, we propose new anomaly detection algorithms. The fact that curvilinear discontinuities in images are sparse with respect to the frame of shearlets, allows us to pose this anomaly detection problem as basis pursuit optimization. Therefore, we pose the problem of detecting curvilinear anomalies in noisy textured images as a blind source separation problem under sparsity constraints, and propose an iterative shrinkage algorithm to solve it. Taking advantage of the parallel nature of this algorithm, we describe how this method can be accelerated using graphical processing units (GPU). Then, we propose a new method for finding defective components on railway tracks using cameras mounted on a train. We describe how to extract features and use a combination of classifiers to solve this problem. Then, we scale anomaly detection to bigger datasets with complex interdependencies. We show that the anomaly detection problem naturally fits in the multitask learning framework. The first task consists of learning a compact representation of the good samples, while the second task consists of learning the anomaly detector. Using deep convolutional neural networks, we show that it is possible to train a deep model with a limited number of anomalous examples. In sequential detection problems, the presence of time-variant nuisance parameters affect the detection performance. In the last part of this dissertation, we present a method for adaptively estimating the threshold of sequential detectors using Extreme Value Theory on a Bayesian framework. Finally, conclusions on the results obtained are provided, followed by a discussion of possible future work.

Detection of mesoscale thermal fronts from 4km data using smoothing techniques: Gradient-based fronts classification and basin scale application

In order to optimize frontal detection in sea surface temperature fields at 4 km resolution, a combined statistical and expert-based approach is applied to test different spatial smoothing of the data prior to the detection process. Fronts are usually detected at 1 km resolution using the histogram-based, single image edge detection (SIED) algorithm developed by Cayula and Cornillon in 1992, with a standard preliminary smoothing using a median filter and a 3 × 3 pixel kernel. Here, detections are performed in three study regions (off Morocco, the Mozambique Channel, and north-western Australia) and across the Indian Ocean basin using the combination of multiple windows (CMW) method developed by Nieto, Demarcq and McClatchie in 2012 which improves on the original Cayula and Cornillon algorithm. Detections at 4 km and 1 km of resolution are compared. Fronts are divided in two intensity classes (“weak” and “strong”) according to their thermal gradient. A preliminary smoothing is applied prior to the detection using different convolutions: three type of filters (median, average and Gaussian) combined with four kernel sizes (3 × 3, 5 × 5, 7 × 7, and 9 × 9 pixels) and three detection window sizes (16 × 16, 24 × 24 and 32 × 32 pixels) to test the effect of these smoothing combinations on reducing the background noise of the data and therefore on improving the frontal detection. The performance of the combinations on 4 km data are evaluated using two criteria: detection efficiency and front length. We find that the optimal combination of preliminary smoothing parameters in enhancing detection efficiency and preserving front length includes a median filter, a 16 × 16 pixel window size, and a 5 × 5 pixel kernel for strong fronts and a 7 × 7 pixel kernel for weak fronts. Results show an improvement in detection performance (from largest to smallest window size) of 71% for strong fronts and 120% for weak fronts. Despite the small window used (16 × 16 pixels), the length of the fronts has been preserved relative to that found with 1 km data. This optimal preliminary smoothing and the CMW detection algorithm on 4 km sea surface temperature data are then used to describe the spatial distribution of the monthly frequencies of occurrence for both strong and weak fronts across the Indian Ocean basin. In general strong fronts are observed in coastal areas whereas weak fronts, with some seasonal exceptions, are mainly located in the open ocean. This study shows that adequate noise reduction done by a preliminary smoothing of the data considerably improves the frontal detection efficiency as well as the global quality of the results. Consequently, the use of 4 km data enables frontal detections similar to 1 km data (using a standard median 3 × 3 convolution) in terms of detectability, length and location. This method, using 4 km data is easily applicable to large regions or at the global scale with far less constraints of data manipulation and processing time relative to 1 km data.

Hand detection on images based on deformable part models and additional features

Hand detection on images has important applications on person activities recognition. This thesis focuses on PASCAL Visual Object Classes (VOC) system for hand detection. VOC has become a popular system for object detection, based on twenty common objects, and has been released with a successful deformable parts model in VOC2007. A hand detection on an image is made when the system gets a bounding box which overlaps with at least 50% of any ground truth bounding box for a hand on the image. The initial average precision of this detector is around 0.215 compared with a state-of-art of 0.104; however, color and frequency features for detected bounding boxes contain important information for re-scoring, and the average precision can be improved to 0.218 with these features. Results show that these features help on getting higher precision for low recall, even though the average precision is similar.

Statistical Analysis of the heavy neutral Atoms measured by IBEX

We investigate the directional distribution of heavy neutral atoms in the heliosphere by using heavy neutral maps generated with the IBEX-Lo instrument over three years from 2009 to 2011. The interstellar neutral (ISN) O&Ne gas flow was found in the first-year heavy neutral map at 601 keV and its flow direction and temperature were studied. However, due to the low counting statistics, researchers have not treated the full sky maps in detail. The main goal of this study is to evaluate the statistical significance of each pixel in the heavy neutral maps to get a better understanding of the directional distribution of heavy neutral atoms in the heliosphere. Here, we examine three statistical analysis methods: the signal-to-noise filter, the confidence limit method, and the cluster analysis method. These methods allow us to exclude background from areas where the heavy neutral signal is statistically significant. These methods also allow the consistent detection of heavy neutral atom structures. The main emission feature expands toward lower longitude and higher latitude from the observational peak of the ISN O&Ne gas flow. We call this emission the extended tail. It may be an imprint of the secondary oxygen atoms generated by charge exchange between ISN hydrogen atoms and oxygen ions in the outer heliosheath.

Fault detection with Conditional Gaussian Network

International audience

Fast Detection and Chemical Characterization of Gunshot Residues by CMV-GC-MS and LIBS

Gunshot residue (GSR) is the term used to describe the particles originating from different parts of the firearm and ammunition during the discharge. A fast and practical field tool to detect the presence of GSR can assist law enforcement in the accurate identification of subjects. A novel field sampling device is presented for the first time for the fast detection and quantitation of volatile organic compounds (VOCs). The capillary microextraction of volatiles (CMV) is a headspace sampling technique that provides fast results (< 2 min. sampling time) and is reported as a versatile and high-efficiency sampling tool. The CMV device can be coupled to a Gas Chromatography-Mass Spectrometry (GC-MS) instrument by installation of a thermal separation probe in the injection port of the GC. An analytical method using the CMV device was developed for the detection of 17 compounds commonly found in polluted environments. The acceptability of the CMV as a field sampling method for the detection of VOCs is demonstrated by following the criteria established by the Environmental Protection Agency (EPA) compendium method TO-17. The CMV device was used, for the first time, for the detection of VOCs on swabs from the hands of shooters, and non-shooters and spent cartridges from different types of ammunition (i.e., pistol, rifle, and shotgun). The proposed method consists in the headspace extraction of VOCs in smokeless powders present in the propellant of ammunition. The sensitivity of this method was demonstrated with method detection limits (MDLs) 4-26 ng for diphenylamine (DPA), nitroglycerine (NG), 2,4-dinitrotoluene (2,4-DNT), and ethyl centralite (EC). In addition, a fast method was developed for the detection of the inorganic components (i.e., Ba, Pb, and Sb) characteristic of GSR presence by Laser Induced Breakdown Spectroscopy (LIBS). Advantages of LIBS include fast analysis (~ 12 seconds per sample) and good sensitivity, with expected MDLs in the range of 0.1-20 ng for target elements. Statistical analysis of the results using both techniques was performed to determine any correlation between the variables analyzed. This work demonstrates that the information collected from the analysis of organic components has the potential to improve the detection of GSR.

Factors affecting southern water vole (Arvicola sapidus) detection and occupancy probabilities in Mediterranean farmland

Failure to detect a species at sites where it is present (i.e. imperfect detection) is known to occur frequently, but this is often disregarded in monitoring programs and metapopulation studies. Here we modelled for the first time the probability of patch occupancy by a threatened small mammal, the southern water vole (Arvicola sapidus, while accounting for the probability of detection given occupancy. Based on replicated presence sign surveys conducted in autumn (November–December 2013) and winter (February–March 2014) in a farmland landscape, we used occupancy detection modelling to test the effects of vegetation, sampling effort, observer experience, and rainfall on detection probability. We then assessed whether occupancy was related to patch size, isolation, vegetation, or presence of water, after correcting for imperfect detection. The mean detection probabilities of water vole signs in autumn (0.71) and winter (0.81) indicated that false absences may be generated in about 20–30% of occupied patches surveyed by a single observer on a single occasion. There was no statistical support for the effects of covariates on detectability. After controlling for imperfect detection, the mean probabilities of occupancy in autumn (0.31) and winter (0.29) were positively related to patch size and presence of water, and negatively so, albeit weakly, to patch isolation. Overall, our study underlined the importance of accounting for imperfect detection in sign surveys of small mammals such as water voles, pointing out the need to use occupancy detection modelling together with replicate surveys for accurately estimating occupancy and the factors affecting it.

Characterization of DNA sequence properties through network and statistical approaches

In this thesis we will see that the DNA sequence is constantly shaped by the interactions with its environment at multiple levels, showing footprints of DNA methylation, of its 3D organization and, in the case of bacteria, of the interaction with the host organisms. In the first chapter, we will see that analyzing the distribution of distances between consecutive dinucleotides of the same type along the sequence, we can detect epigenetic and structural footprints. In particular, we will see that CG distance distribution allows to distinguish among organisms of different biological complexity, depending on how much CG sites are involved in DNA methylation. Moreover, we will see that CG and TA can be described by the same fitting function, suggesting a relationship between the two. We will also provide an interpretation of the observed trend, simulating a positioning process guided by the presence and absence of memory. In the end, we will focus on TA distance distribution, characterizing deviations from the trend predicted by the best fitting function, and identifying specific patterns that might be related to peculiar mechanical properties of the DNA and also to epigenetic and structural processes. In the second chapter, we will see how we can map the 3D structure of the DNA onto its sequence. In particular, we devised a network-based algorithm that produces a genome assembly starting from its 3D configuration, using as inputs Hi-C contact maps. Specifically, we will see how we can identify the different chromosomes and reconstruct their sequences by exploiting the spectral properties of the Laplacian operator of a network. In the third chapter, we will see a novel method for source clustering and source attribution, based on a network approach, that allows to identify host-bacteria interaction starting from the detection of Single-Nucleotide Polymorphisms along the sequence of bacterial genomes.

Deep Learning Architectures for time of arrival detection in Acoustic Emissions Monitoring

Acoustic Emission (AE) monitoring can be used to detect the presence of damage as well as determine its location in Structural Health Monitoring (SHM) applications. Information on the time difference of the signal generated by the damage event arriving at different sensors is essential in performing localization. This makes the time of arrival (ToA) an important piece of information to retrieve from the AE signal. Generally, this is determined using statistical methods such as the Akaike Information Criterion (AIC) which is particularly prone to errors in the presence of noise. And given that the structures of interest are surrounded with harsh environments, a way to accurately estimate the arrival time in such noisy scenarios is of particular interest. In this work, two new methods are presented to estimate the arrival times of AE signals which are based on Machine Learning. Inspired by great results in the field, two models are presented which are Deep Learning models - a subset of machine learning. They are based on Convolutional Neural Network (CNN) and Capsule Neural Network (CapsNet). The primary advantage of such models is that they do not require the user to pre-define selected features but only require raw data to be given and the models establish non-linear relationships between the inputs and outputs. The performance of the models is evaluated using AE signals generated by a custom ray-tracing algorithm by propagating them on an aluminium plate and compared to AIC. It was found that the relative error in estimation on the test set was < 5% for the models compared to around 45% of AIC. The testing process was further continued by preparing an experimental setup and acquiring real AE signals to test on. Similar performances were observed where the two models not only outperform AIC by more than a magnitude in their average errors but also they were shown to be a lot more robust as compared to AIC which fails in the presence of noise.

Non-invasive detection of acute cell-mediated graft rejection in paediatric heart transplant recipients: the role of cardiovascular magnetic resonance

Background and Aim: Acute cardiac rejection is currently diagnosed by endomyocardial biopsy (EMB), but multiparametric cardiac magnetic resonance (CMR) may be a non-invasive alternative by its capacity for myocardial structure and function characterization. Our primary aim was to determine the utility of multiparametric CMR in identifying acute graft rejection in paediatric heart transplant recipients. The second aim was to compare textural features of parametric maps in cases of rejection versus those without rejection. Methods: Fifteen patients were prospectively enrolled for contrast-enhanced CMR followed by EMB and right heart catheterization. Images were acquired on a 1,5 Tesla scanner including T1 mapping (modified Look-Locker inversion recovery sequence – MOLLI) and T2 mapping (modified GraSE sequence). The extracellular volume (ECV) was calculated using pre- and post-gadolinium T1 times of blood and myocardium and the patient’s hematocrit. Markers of graft dysfunction including hemodynamic measurements from echocardiography, catheterization and CMR were collated. Patients were divided into two groups based on degree of rejection at EMB: no rejection with no change in treatment (Group A) and acute rejection requiring new therapy (Group B). Statistical analysis included student’t t test and Pearson correlation. Results: Acute rejection was diagnosed in five patients. Mean T1 values were significantly associated with acute rejection. A monotonic, increasing trend was noted in both mean and peak T1 values, with increasing degree of rejection. ECV was significantly higher in Group B. There was no difference in T2 signal between two groups. Conclusion: Multiparametric CMR serves as a noninvasive screening tool during surveillance encounters and may be used to identify those patients that may be at higher risk of rejection and therefore require further evaluation. Future and multicenter studies are necessary to confirm these results and explore whether multiparametric CMR can decrease the number of surveillance EMBs in paediatric heart transplant recipients.