Software tools and efficient algorithms for the feature detection, feature tracking, event localization, and visualization of large sets of atmospheric data

Data sets describing the state of the earth's atmosphere are of great importance in the atmospheric sciences. Over the last decades, the quality and sheer amount of the available data increased significantly, resulting in a rising demand for new tools capable of handling and analysing these large, multidimensional sets of atmospheric data. The interdisciplinary work presented in this thesis covers the development and the application of practical software tools and efficient algorithms from the field of computer science, aiming at the goal of enabling atmospheric scientists to analyse and to gain new insights from these large data sets. For this purpose, our tools combine novel techniques with well-established methods from different areas such as scientific visualization and data segmentation. In this thesis, three practical tools are presented. Two of these tools are software systems (Insight and IWAL) for different types of processing and interactive visualization of data, the third tool is an efficient algorithm for data segmentation implemented as part of Insight.Insight is a toolkit for the interactive, three-dimensional visualization and processing of large sets of atmospheric data, originally developed as a testing environment for the novel segmentation algorithm. It provides a dynamic system for combining at runtime data from different sources, a variety of different data processing algorithms, and several visualization techniques. Its modular architecture and flexible scripting support led to additional applications of the software, from which two examples are presented: the usage of Insight as a WMS (web map service) server, and the automatic production of a sequence of images for the visualization of cyclone simulations. The core application of Insight is the provision of the novel segmentation algorithm for the efficient detection and tracking of 3D features in large sets of atmospheric data, as well as for the precise localization of the occurring genesis, lysis, merging and splitting events. Data segmentation usually leads to a significant reduction of the size of the considered data. This enables a practical visualization of the data, statistical analyses of the features and their events, and the manual or automatic detection of interesting situations for subsequent detailed investigation. The concepts of the novel algorithm, its technical realization, and several extensions for avoiding under- and over-segmentation are discussed. As example applications, this thesis covers the setup and the results of the segmentation of upper-tropospheric jet streams and cyclones as full 3D objects. Finally, IWAL is presented, which is a web application for providing an easy interactive access to meteorological data visualizations, primarily aimed at students. As a web application, the needs to retrieve all input data sets and to install and handle complex visualization tools on a local machine are avoided. The main challenge in the provision of customizable visualizations to large numbers of simultaneous users was to find an acceptable trade-off between the available visualization options and the performance of the application. Besides the implementational details, benchmarks and the results of a user survey are presented.

IMILAST: A community effort to intercompare extratropical cyclone detection and tracking algorithms: assessing method-related uncertainties

The variability of results from different automated methods of detection and tracking of extratropical cyclones is assessed in order to identify uncertainties related to the choice of method. Fifteen international teams applied their own algorithms to the same dataset - the period 1989-2009 of interim European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis (ERAInterim) data. This experiment is part of the community project Intercomparison of Mid Latitude Storm Diagnostics (IMILAST; see www.proclim.ch/imilast/index.html). The spread of results for cyclone frequency, intensity, life cycle, and track location is presented to illustrate the impact of using different methods. Globally, methods agree well for geographical distribution in large oceanic regions, interannual variability of cyclone numbers, geographical patterns of strong trends, and distribution shape for many life cycle characteristics. In contrast, the largest disparities exist for the total numbers of cyclones, the detection of weak cyclones, and distribution in some densely populated regions. Consistency between methods is better for strong cyclones than for shallow ones. Two case studies of relatively large, intense cyclones reveal that the identification of the most intense part of the life cycle of these events is robust between methods, but considerable differences exist during the development and the dissolution phases.

Improvement for detection of microcalcifications through clustering algorithms and artificial neural networks

A new method for detecting microcalcifications in regions of interest (ROIs) extracted from digitized mammograms is proposed. The top-hat transform is a technique based on mathematical morphology operations and, in this paper, is used to perform contrast enhancement of the mi-crocalcifications. To improve microcalcification detection, a novel image sub-segmentation approach based on the possibilistic fuzzy c-means algorithm is used. From the original ROIs, window-based features, such as the mean and standard deviation, were extracted; these features were used as an input vector in a classifier. The classifier is based on an artificial neural network to identify patterns belonging to microcalcifications and healthy tissue. Our results show that the proposed method is a good alternative for automatically detecting microcalcifications, because this stage is an important part of early breast cancer detection

Evaluation of a Change Detection Methodology by Means of Binary Thresholding Algorithms and Informational Fusion Processes

Landcover is subject to continuous changes on a wide variety of temporal and spatial scales. Those changes produce significant effects in human and natural activities. Maintaining an updated spatial database with the occurred changes allows a better monitoring of the Earth?s resources and management of the environment. Change detection (CD) techniques using images from different sensors, such as satellite imagery, aerial photographs, etc., have proven to be suitable and secure data sources from which updated information can be extracted efficiently, so that changes can also be inventoried and monitored. In this paper, a multisource CD methodology for multiresolution datasets is applied. First, different change indices are processed, then different thresholding algorithms for change/no_change are applied to these indices in order to better estimate the statistical parameters of these categories, finally the indices are integrated into a change detection multisource fusion process, which allows generating a single CD result from several combination of indices. This methodology has been applied to datasets with different spectral and spatial resolution properties. Then, the obtained results are evaluated by means of a quality control analysis, as well as with complementary graphical representations. The suggested methodology has also been proved efficiently for identifying the change detection index with the higher contribution.

Development and improvement of protection-system algorithms for ground-fault detection and location in synchronous machines

Uno de los defectos más frecuentes en los generadores síncronos son los defectos a tierra tanto en el devanado estatórico, como de excitación. Se produce un defecto cuando el aislamiento eléctrico entre las partes activas de cualquiera de estos devanados y tierra se reduce considerablemente o desaparece. La detección de los defectos a tierra en ambos devanados es un tema ampliamente estudiado a nivel industrial. Tras la detección y confirmación de la existencia del defecto, dicha falta debe ser localizada a lo largo del devanado para su reparación, para lo que habitualmente el rotor debe ser extraído del estator. Esta operación resulta especialmente compleja y cara. Además, el hecho de limitar la corriente de defecto en ambos devanados provoca que el defecto no sea localizable visualmente, pues apenas existe daño en el generador. Por ello, se deben aplicar técnicas muy laboriosas para localizar exactamente el defecto y poder así reparar el devanado. De cara a reducir el tiempo de reparación, y con ello el tiempo en que el generador esta fuera de servicio, cualquier información por parte del relé de protección acerca de la localización del defecto resultaría de gran utilidad. El principal objetivo de esta tesis doctoral ha sido el desarrollo de nuevos algoritmos que permitan la estimación de la localización de los defectos a tierra tanto en el devanado rotórico como estatórico de máquinas síncronas. Respecto al devanado de excitación, se ha presentado un nuevo método de localización de defectos a tierra para generadores con excitación estática. Este método permite incluso distinguir si el defecto se ha producido en el devanado de excitación, o en cualquiera de los componentes del sistema de excitación, esto es, transformador de excitación, conductores de alimentación del rectificador controlado, etc. En caso de defecto a tierra en del devanado rotórico, este método proporciona una estimación de su localización. Sin embargo, para poder obtener la localización del defecto, se precisa conocer el valor de resistencia de defecto. Por ello, en este trabajo se presenta además un nuevo método para la estimación de este parámetro de forma precisa. Finalmente, se presenta un nuevo método de detección de defectos a tierra, basado en el criterio direccional, que complementa el método de localización, permitiendo tener en cuenta la influencia de las capacidades a tierra del sistema. Estas capacidades resultan determinantes a la hora de localizar el defecto de forma adecuada. En relación con el devanado estatórico, en esta tesis doctoral se presenta un nuevo algoritmo de localización de defectos a tierra para generadores que dispongan de la protección de faltas a tierra basada en la inyección de baja frecuencia. Se ha propuesto un método general, que tiene en cuenta todos los parámetros del sistema, así como una versión simplificada del método para generadores con capacidades a tierra muy reducida, que podría resultar de fácil implementación en relés de protección comercial. Los algoritmos y métodos presentados se han validado mediante ensayos experimentales en un generador de laboratorio de 5 kVA, así como en un generador comercial de 106 MVA con resultados satisfactorios y prometedores. ABSTRACT One of the most common faults in synchronous generators is the ground fault in both the stator winding and the excitation winding. In case of fault, the insulation level between the active part of any of these windings and ground lowers considerably, or even disappears. The detection of ground faults in both windings is a very researched topic. The fault current is typically limited intentionally to a reduced level. This allows to detect easily the ground faults, and therefore to avoid damage in the generator. After the detection and confirmation of the existence of a ground fault, it should be located along the winding in order to repair of the machine. Then, the rotor has to be extracted, which is a very complex and expensive operation. Moreover, the fact of limiting the fault current makes that the insulation failure is not visually detectable, because there is no visible damage in the generator. Therefore, some laborious techniques have to apply to locate accurately the fault. In order to reduce the repair time, and therefore the time that the generator is out of service, any information about the approximate location of the fault would be very useful. The main objective of this doctoral thesis has been the development of new algorithms and methods to estimate the location of ground faults in the stator and in the rotor winding of synchronous generators. Regarding the excitation winding, a new location method of ground faults in excitation winding of synchronous machines with static excitation has been presented. This method allows even to detect if the fault is at the excitation winding, or in any other component of the excitation system: controlled rectifier, excitation transformer, etc. In case of ground fault in the rotor winding, this method provides an estimation of the fault location. However, in order to calculate the location, the value of fault resistance is necessary. Therefore, a new fault-resistance estimation algorithm is presented in this text. Finally, a new fault detection algorithm based on directional criterion is described to complement the fault location method. This algorithm takes into account the influence of the capacitance-to-ground of the system, which has a remarkable impact in the accuracy of the fault location. Regarding the stator winding, a new fault-location algorithm has been presented for stator winding of synchronous generators. This algorithm is applicable to generators with ground-fault protection based in low-frequency injection. A general algorithm, which takes every parameter of the system into account, has been presented. Moreover, a simplified version of the algorithm has been proposed for generators with especially low value of capacitance to ground. This simplified algorithm might be easily implementable in protective relays. The proposed methods and algorithms have been tested in a 5 kVA laboratory generator, as well as in a 106 MVA synchronous generator with satisfactory and promising results.

Research on vehicle-based driver status/performance monitoring; development, validation, and refinement of algorithms for detection of driver drowsiness.

National Highway Traffic Safety Administration, Washington, D.C.

Data Mining for Network Intrusion Detection : A comparison of data mining algorithms and an analysis of relevant features for detecting cyber-attacks

Data mining can be defined as the extraction of implicit, previously un-known, and potentially useful information from data. Numerous re-searchers have been developing security technology and exploring new methods to detect cyber-attacks with the DARPA 1998 dataset for Intrusion Detection and the modified versions of this dataset KDDCup99 and NSL-KDD, but until now no one have examined the performance of the Top 10 data mining algorithms selected by experts in data mining. The compared classification learning algorithms in this thesis are: C4.5, CART, k-NN and Naïve Bayes. The performance of these algorithms are compared with accuracy, error rate and average cost on modified versions of NSL-KDD train and test dataset where the instances are classified into normal and four cyber-attack categories: DoS, Probing, R2L and U2R. Additionally the most important features to detect cyber-attacks in all categories and in each category are evaluated with Weka’s Attribute Evaluator and ranked according to Information Gain. The results show that the classification algorithm with best performance on the dataset is the k-NN algorithm. The most important features to detect cyber-attacks are basic features such as the number of seconds of a network connection, the protocol used for the connection, the network service used, normal or error status of the connection and the number of data bytes sent. The most important features to detect DoS, Probing and R2L attacks are basic features and the least important features are content features. Unlike U2R attacks, where the content features are the most important features to detect attacks.

Pre-processing for noise detection in gene expression classification data

Due to the imprecise nature of biological experiments, biological data is often characterized by the presence of redundant and noisy data. This may be due to errors that occurred during data collection, such as contaminations in laboratorial samples. It is the case of gene expression data, where the equipments and tools currently used frequently produce noisy biological data. Machine Learning algorithms have been successfully used in gene expression data analysis. Although many Machine Learning algorithms can deal with noise, detecting and removing noisy instances from the training data set can help the induction of the target hypothesis. This paper evaluates the use of distance-based pre-processing techniques for noise detection in gene expression data classification problems. This evaluation analyzes the effectiveness of the techniques investigated in removing noisy data, measured by the accuracy obtained by different Machine Learning classifiers over the pre-processed data.

Local search multiuser detection

In this work, a wide analysis of local search multiuser detection (LS-MUD) for direct sequence/code division multiple access (DS/CDMA) systems under multipath channels is carried out considering the performance-complexity trade-off. It is verified the robustness of the LS-MUD to variations in loading, E(b)/N(0), near-far effect, number of fingers of the Rake receiver and errors in the channel coefficients estimates. A compared analysis of the bit error rate (BER) and complexity trade-off is accomplished among LS, genetic algorithm (GA) and particle swarm optimization (PSO). Based on the deterministic behavior of the LS algorithm, it is also proposed simplifications over the cost function calculation, obtaining more efficient algorithms (simplified and combined LS-MUD versions) and creating new perspectives for the MUD implementation. The computational complexity is expressed in terms of the number of operations in order to converge. Our conclusion pointed out that the simplified LS (s-LS) method is always more efficient, independent of the system conditions, achieving a better performance with a lower complexity than the others heuristics detectors. Associated to this, the deterministic strategy and absence of input parameters made the s-LS algorithm the most appropriate for the MUD problem. (C) 2008 Elsevier GmbH. All rights reserved.

Jointly multi-user detection and channel estimation with genetic algorithm

This work aims at proposing the use of the evolutionary computation methodology in order to jointly solve the multiuser channel estimation (MuChE) and detection problems at its maximum-likelihood, both related to the direct sequence code division multiple access (DS/CDMA). The effectiveness of the proposed heuristic approach is proven by comparing performance and complexity merit figures with that obtained by traditional methods found in literature. Simulation results considering genetic algorithm (GA) applied to multipath, DS/CDMA and MuChE and multi-user detection (MuD) show that the proposed genetic algorithm multi-user channel estimation (GAMuChE) yields a normalized mean square error estimation (nMSE) inferior to 11%, under slowly varying multipath fading channels, large range of Doppler frequencies and medium system load, it exhibits lower complexity when compared to both maximum likelihood multi-user channel estimation (MLMuChE) and gradient descent method (GrdDsc). A near-optimum multi-user detector (MuD) based on the genetic algorithm (GAMuD), also proposed in this work, provides a significant reduction in the computational complexity when compared to the optimum multi-user detector (OMuD). In addition, the complexity of the GAMuChE and GAMuD algorithms were (jointly) analyzed in terms of number of operations necessary to reach the convergence, and compared to other jointly MuChE and MuD strategies. The joint GAMuChE-GAMuD scheme can be regarded as a promising alternative for implementing third-generation (3G) and fourth-generation (4G) wireless systems in the near future. Copyright (C) 2010 John Wiley & Sons, Ltd.

SKAN: Skin Scanner - System for Skin Cancer Detection Using Adaptive Techniques

SKAN: Skin Scanner - System for Skin Cancer Detection Using Adaptive Techniques - combines computer engineering concepts with areas like dermatology and oncology. Its objective is to discern images of skin cancer, specifically melanoma, from others that show only common spots or other types of skin diseases, using image recognition. This work makes use of the ABCDE visual rule, which is often used by dermatologists for melanoma identification, to define which characteristics are analyzed by the software. It then applies various algorithms and techniques, including an ellipse-fitting algorithm, to extract and measure these characteristics and decide whether the spot is a melanoma or not. The achieved results are presented with special focus on the adaptive decision-making and its effect on the diagnosis. Finally, other applications of the software and its algorithms are presented.

Robust algorithms for solving stochastic partial differential equations

A robust semi-implicit central partial difference algorithm for the numerical solution of coupled stochastic parabolic partial differential equations (PDEs) is described. This can be used for calculating correlation functions of systems of interacting stochastic fields. Such field equations can arise in the description of Hamiltonian and open systems in the physics of nonlinear processes, and may include multiplicative noise sources. The algorithm can be used for studying the properties of nonlinear quantum or classical field theories. The general approach is outlined and applied to a specific example, namely the quantum statistical fluctuations of ultra-short optical pulses in chi((2)) parametric waveguides. This example uses a non-diagonal coherent state representation, and correctly predicts the sub-shot noise level spectral fluctuations observed in homodyne detection measurements. It is expected that the methods used wilt be applicable for higher-order correlation functions and other physical problems as well. A stochastic differencing technique for reducing sampling errors is also introduced. This involves solving nonlinear stochastic parabolic PDEs in combination with a reference process, which uses the Wigner representation in the example presented here. A computer implementation on MIMD parallel architectures is discussed. (C) 1997 Academic Press.

Jamming detection in GNSS signals using the sample covariance matrix.

We propose a blind method to detect interference in GNSS signals whereby the algorithms do not require knowledge of the interference or channel noise features. A sample covariance matrix is constructed from the received signal and its eigenvalues are computed. The generalized likelihood ratio test (GLRT) and the condition number test (CNT) are developed and compared in the detection of sinusoidal and chirp jamming signals. A computationally-efficient decision threshold was proposed for the CNT.

Automatic sleep apnea detection and sleep classification using the ECG and the SpO2 signals

Dissertation for a Masters Degree in Computer and Electronic Engineering