Data-driven clustering : new methods and applications

Abstract : This work is concerned with the development and application of novel unsupervised learning methods, having in mind two target applications: the analysis of forensic case data and the classification of remote sensing images. First, a method based on a symbolic optimization of the inter-sample distance measure is proposed to improve the flexibility of spectral clustering algorithms, and applied to the problem of forensic case data. This distance is optimized using a loss function related to the preservation of neighborhood structure between the input space and the space of principal components, and solutions are found using genetic programming. Results are compared to a variety of state-of--the-art clustering algorithms. Subsequently, a new large-scale clustering method based on a joint optimization of feature extraction and classification is proposed and applied to various databases, including two hyperspectral remote sensing images. The algorithm makes uses of a functional model (e.g., a neural network) for clustering which is trained by stochastic gradient descent. Results indicate that such a technique can easily scale to huge databases, can avoid the so-called out-of-sample problem, and can compete with or even outperform existing clustering algorithms on both artificial data and real remote sensing images. This is verified on small databases as well as very large problems. Résumé : Ce travail de recherche porte sur le développement et l'application de méthodes d'apprentissage dites non supervisées. Les applications visées par ces méthodes sont l'analyse de données forensiques et la classification d'images hyperspectrales en télédétection. Dans un premier temps, une méthodologie de classification non supervisée fondée sur l'optimisation symbolique d'une mesure de distance inter-échantillons est proposée. Cette mesure est obtenue en optimisant une fonction de coût reliée à la préservation de la structure de voisinage d'un point entre l'espace des variables initiales et l'espace des composantes principales. Cette méthode est appliquée à l'analyse de données forensiques et comparée à un éventail de méthodes déjà existantes. En second lieu, une méthode fondée sur une optimisation conjointe des tâches de sélection de variables et de classification est implémentée dans un réseau de neurones et appliquée à diverses bases de données, dont deux images hyperspectrales. Le réseau de neurones est entraîné à l'aide d'un algorithme de gradient stochastique, ce qui rend cette technique applicable à des images de très haute résolution. Les résultats de l'application de cette dernière montrent que l'utilisation d'une telle technique permet de classifier de très grandes bases de données sans difficulté et donne des résultats avantageusement comparables aux méthodes existantes.

Data Envelopment Analysis (DEA) : A pedagogical guide for decision makers in the public sector

This guide introduces Data Envelopment Analysis (DEA), a performance measurement technique, in such a way as to be appropriate to decision makers with little or no background in economics and operational research. The use of mathematics is kept to a minimum. This guide therefore adopts a strong practical approach in order to allow decision makers to conduct their own efficiency analysis and to easily interpret results. DEA helps decision makers for the following reasons: - By calculating an efficiency score, it indicates if a firm is efficient or has capacity for improvement. - By setting target values for input and output, it calculates how much input must be decreased or output increased in order to become efficient. - By identifying the nature of returns to scale, it indicates if a firm has to decrease or increase its scale (or size) in order to minimize the average cost. - By identifying a set of benchmarks, it specifies which other firms' processes need to be analysed in order to improve its own practices.

Data Envelopment Analysis (DEA) : un guide pédagogique à l'intention des décideurs dans le secteur public

Ce guide présente la méthode Data Envelopment Analysis (DEA), une méthode d'évaluation de la performance . Il est destiné aux responsables d'organisations publiques qui ne sont pas familiers avec les notions d'optimisation mathématique, autrement dit de recherche opérationnelle. L'utilisation des mathématiques est par conséquent réduite au minimum. Ce guide est fortement orienté vers la pratique. Il permet aux décideurs de réaliser leurs propres analyses d'efficience et d'interpréter facilement les résultats obtenus. La méthode DEA est un outil d'analyse et d'aide à la décision dans les domaines suivants : - en calculant un score d'efficience, elle indique si une organisation dispose d'une marge d'amélioration ; - en fixant des valeurs-cibles, elle indique de combien les inputs doivent être réduits et les outputs augmentés pour qu'une organisation devienne efficiente ; - en identifiant le type de rendements d'échelle, elle indique si une organisation doit augmenter ou au contraire réduire sa taille pour minimiser son coût moyen de production ; - en identifiant les pairs de référence, elle désigne quelles organisations disposent des best practice à analyser.

Nonresponse to clozapine and ultrarapid CYP1A2 activity: clinical data and analysis of CYP1A2 gene.

Clozapine (CLO), an atypical antipsychotic, depends mainly on cytochrome P450 1A2 (CYP1A2) for its metabolic clearance. Four patients treated with CLO, who were smokers, were nonresponders and had low plasma levels while receiving usual doses. Their plasma levels to dose ratios of CLO (median; range, 0.34; 0.22 to 0.40 ng x day/mL x mg) were significantly lower than ratios calculated from another study with 29 patients (0.75; 0.22 to 2.83 ng x day/mL x mg; P < 0.01). These patients were confirmed as being CYP1A2 ultrarapid metabolizers by the caffeine phenotyping test (median systemic caffeine plasma clearance; range, 3.85; 3.33 to 4.17 mL/min/kg) when compared with previous studies (0.3 to 3.33 mL/min/kg). The sequencing of the entire CYP1A2 gene from genomic DNA of these patients suggests that the -164C > A mutation (CYP1A2*1F) in intron 1, which confers a high inducibility of CYP1A2 in smokers, is the most likely explanation for their ultrarapid CYP1A2 activity. A marked (2 patients) or a moderate (2 patients) improvement of the clinical state of the patients occurred after the increase of CLO blood levels above the therapeutic threshold by the increase of CLO doses to very high values (ie, up to 1400 mg/d) or by the introduction of fluvoxamine, a potent CYP1A2 inhibitor, at low dosage (50 to 100 mg/d). Due to the high frequency of smokers among patients with schizophrenia and to the high frequency of the -164C > A polymorphism, CYP1A2 genotyping could have important clinical implications for the treatment of patients with CLO.

Schizophrenia and complications of pregnancy and labor: an individual patient data meta-analysis.

Several epidemiological studies have reported an association between complications of pregnancy and delivery and schizophrenia, but none have had sufficient power to examine specific complications that, individually, are of low prevalence. We, therefore, performed an individual patient meta-analysis using the raw data from case control studies that used the Lewis-Murray scale. Data were obtained from 12 studies on 700 schizophrenia subjects and 835 controls. There were significant associations between schizophrenia and premature rupture of membranes, gestational age shorter than 37 weeks, and use of resuscitation or incubator. There were associations of borderline significance between schizophrenia and birthweight lower than 2,500 g and forceps delivery. There was no significant interaction between these complications and sex. We conclude that some abnormalities of pregnancy and delivery may be associated with development of schizophrenia. The pathophysiology may involve hypoxia and so future studies should focus on the accurate measurement of this exposure.

Data-driven topo-climatic mapping with machine learning methods

Automatic environmental monitoring networks enforced by wireless communication technologies provide large and ever increasing volumes of data nowadays. The use of this information in natural hazard research is an important issue. Particularly useful for risk assessment and decision making are the spatial maps of hazard-related parameters produced from point observations and available auxiliary information. The purpose of this article is to present and explore the appropriate tools to process large amounts of available data and produce predictions at fine spatial scales. These are the algorithms of machine learning, which are aimed at non-parametric robust modelling of non-linear dependencies from empirical data. The computational efficiency of the data-driven methods allows producing the prediction maps in real time which makes them superior to physical models for the operational use in risk assessment and mitigation. Particularly, this situation encounters in spatial prediction of climatic variables (topo-climatic mapping). In complex topographies of the mountainous regions, the meteorological processes are highly influenced by the relief. The article shows how these relations, possibly regionalized and non-linear, can be modelled from data using the information from digital elevation models. The particular illustration of the developed methodology concerns the mapping of temperatures (including the situations of Föhn and temperature inversion) given the measurements taken from the Swiss meteorological monitoring network. The range of the methods used in the study includes data-driven feature selection, support vector algorithms and artificial neural networks.

Data Envelopment Analysis and non-discretionary inputs : How to select the most suitable model using multi-criteria decision analysis

Within Data Envelopment Analysis, several alternative models allow for an environmental adjustment. The majority of them deliver divergent results. Decision makers face the difficult task of selecting the most suitable model. This study is performed to overcome this difficulty. By doing so, it fills a research gap. First, a two-step web-based survey is conducted. It aims (1) to identify the selection criteria, (2) to prioritize and weight the selection criteria with respect to the goal of selecting the most suitable model and (3) to collect the preferences about which model is preferable to fulfil each selection criterion. Second, Analytic Hierarchy Process is used to quantify the preferences expressed in the survey. Results show that the understandability, the applicability and the acceptability of the alternative models are valid selection criteria. The selection of the most suitable model depends on the preferences of the decision makers with regards to these criteria.

Multiple Kernel Learning of Environmental Data. Case study: Analysis and Mapping of Wind Fields

The paper presents the Multiple Kernel Learning (MKL) approach as a modelling and data exploratory tool and applies it to the problem of wind speed mapping. Support Vector Regression (SVR) is used to predict spatial variations of the mean wind speed from terrain features (slopes, terrain curvature, directional derivatives) generated at different spatial scales. Multiple Kernel Learning is applied to learn kernels for individual features and thematic feature subsets, both in the context of feature selection and optimal parameters determination. An empirical study on real-life data confirms the usefulness of MKL as a tool that enhances the interpretability of data-driven models.

Advanced geostatistical and machine-learning models for spatial data analysis of radioactively contaminated regions

Radioactive soil-contamination mapping and risk assessment is a vital issue for decision makers. Traditional approaches for mapping the spatial concentration of radionuclides employ various regression-based models, which usually provide a single-value prediction realization accompanied (in some cases) by estimation error. Such approaches do not provide the capability for rigorous uncertainty quantification or probabilistic mapping. Machine learning is a recent and fast-developing approach based on learning patterns and information from data. Artificial neural networks for prediction mapping have been especially powerful in combination with spatial statistics. A data-driven approach provides the opportunity to integrate additional relevant information about spatial phenomena into a prediction model for more accurate spatial estimates and associated uncertainty. Machine-learning algorithms can also be used for a wider spectrum of problems than before: classification, probability density estimation, and so forth. Stochastic simulations are used to model spatial variability and uncertainty. Unlike regression models, they provide multiple realizations of a particular spatial pattern that allow uncertainty and risk quantification. This paper reviews the most recent methods of spatial data analysis, prediction, and risk mapping, based on machine learning and stochastic simulations in comparison with more traditional regression models. The radioactive fallout from the Chernobyl Nuclear Power Plant accident is used to illustrate the application of the models for prediction and classification problems. This fallout is a unique case study that provides the challenging task of analyzing huge amounts of data ('hard' direct measurements, as well as supplementary information and expert estimates) and solving particular decision-oriented problems.

Advanced mapping of environmental data: Geostatistics, Machine Learning and Bayesian Maximum Entropy

This book combines geostatistics and global mapping systems to present an up-to-the-minute study of environmental data. Featuring numerous case studies, the reference covers model dependent (geostatistics) and data driven (machine learning algorithms) analysis techniques such as risk mapping, conditional stochastic simulations, descriptions of spatial uncertainty and variability, artificial neural networks (ANN) for spatial data, Bayesian maximum entropy (BME), and more.

Modular analysis of gene expression data with R.

SUMMARY: Large sets of data, such as expression profiles from many samples, require analytic tools to reduce their complexity. The Iterative Signature Algorithm (ISA) is a biclustering algorithm. It was designed to decompose a large set of data into so-called 'modules'. In the context of gene expression data, these modules consist of subsets of genes that exhibit a coherent expression profile only over a subset of microarray experiments. Genes and arrays may be attributed to multiple modules and the level of required coherence can be varied resulting in different 'resolutions' of the modular mapping. In this short note, we introduce two BioConductor software packages written in GNU R: The isa2 package includes an optimized implementation of the ISA and the eisa package provides a convenient interface to run the ISA, visualize its output and put the biclusters into biological context. Potential users of these packages are all R and BioConductor users dealing with tabular (e.g. gene expression) data. AVAILABILITY: http://www.unil.ch/cbg/ISA CONTACT: sven.bergmann@unil.ch

Individual patient data systematic review and meta-analysis of optic nerve sheath diameter ultrasonography for detecting raised intracranial pressure: protocol of the ONSD research group.

BACKGROUND: The purpose of the optic nerve sheath diameter (ONSD) research group project is to establish an individual patient-level database from high quality studies of ONSD ultrasonography for the detection of raised intracranial pressure (ICP), and to perform a systematic review and an individual patient data meta-analysis (IPDMA), which will provide a cutoff value to help physicians making decisions and encourage further research. Previous meta-analyses were able to assess the diagnostic accuracy of ONSD ultrasonography in detecting raised ICP but failed to determine a precise cutoff value. Thus, the ONSD research group was founded to synthesize data from several recent studies on the subject and to provide evidence on the diagnostic accuracy of ONSD ultrasonography in detecting raised ICP. METHODS: This IPDMA will be conducted in different phases. First, we will systematically search for eligible studies. To be eligible, studies must have compared ONSD ultrasonography to invasive intracranial devices, the current reference standard for diagnosing raised ICP. Subsequently, we will assess the quality of studies included based on the QUADAS-2 tool, and then collect and validate individual patient data. The objectives of the primary analyses will be to assess the diagnostic accuracy of ONSD ultrasonography and to determine a precise cutoff value for detecting raised ICP. Secondly, we will construct a logistic regression model to assess whether patient and study characteristics influence diagnostic accuracy. DISCUSSION: We believe that this IPD MA will provide the most reliable basis for the assessment of diagnostic accuracy of ONSD ultrasonography for detecting raised ICP and to provide a cutoff value. We also hope that the creation of the ONSD research group will encourage further study. TRIAL REGISTRATION: PROSPERO registration number: CRD42012003072.

Predictive analysis and mapping of indoor radon concentrations in Switzerland

It is estimated that around 230 people die each year due to radon (222Rn) exposure in Switzerland. 222Rn occurs mainly in closed environments like buildings and originates primarily from the subjacent ground. Therefore it depends strongly on geology and shows substantial regional variations. Correct identification of these regional variations would lead to substantial reduction of 222Rn exposure of the population based on appropriate construction of new and mitigation of already existing buildings. Prediction of indoor 222Rn concentrations (IRC) and identification of 222Rn prone areas is however difficult since IRC depend on a variety of different variables like building characteristics, meteorology, geology and anthropogenic factors. The present work aims at the development of predictive models and the understanding of IRC in Switzerland, taking into account a maximum of information in order to minimize the prediction uncertainty. The predictive maps will be used as a decision-support tool for 222Rn risk management. The construction of these models is based on different data-driven statistical methods, in combination with geographical information systems (GIS). In a first phase we performed univariate analysis of IRC for different variables, namely the detector type, building category, foundation, year of construction, the average outdoor temperature during measurement, altitude and lithology. All variables showed significant associations to IRC. Buildings constructed after 1900 showed significantly lower IRC compared to earlier constructions. We observed a further drop of IRC after 1970. In addition to that, we found an association of IRC with altitude. With regard to lithology, we observed the lowest IRC in sedimentary rocks (excluding carbonates) and sediments and the highest IRC in the Jura carbonates and igneous rock. The IRC data was systematically analyzed for potential bias due to spatially unbalanced sampling of measurements. In order to facilitate the modeling and the interpretation of the influence of geology on IRC, we developed an algorithm based on k-medoids clustering which permits to define coherent geological classes in terms of IRC. We performed a soil gas 222Rn concentration (SRC) measurement campaign in order to determine the predictive power of SRC with respect to IRC. We found that the use of SRC is limited for IRC prediction. The second part of the project was dedicated to predictive mapping of IRC using models which take into account the multidimensionality of the process of 222Rn entry into buildings. We used kernel regression and ensemble regression tree for this purpose. We could explain up to 33% of the variance of the log transformed IRC all over Switzerland. This is a good performance compared to former attempts of IRC modeling in Switzerland. As predictor variables we considered geographical coordinates, altitude, outdoor temperature, building type, foundation, year of construction and detector type. Ensemble regression trees like random forests allow to determine the role of each IRC predictor in a multidimensional setting. We found spatial information like geology, altitude and coordinates to have stronger influences on IRC than building related variables like foundation type, building type and year of construction. Based on kernel estimation we developed an approach to determine the local probability of IRC to exceed 300 Bq/m3. In addition to that we developed a confidence index in order to provide an estimate of uncertainty of the map. All methods allow an easy creation of tailor-made maps for different building characteristics. Our work is an essential step towards a 222Rn risk assessment which accounts at the same time for different architectural situations as well as geological and geographical conditions. For the communication of 222Rn hazard to the population we recommend to make use of the probability map based on kernel estimation. The communication of 222Rn hazard could for example be implemented via a web interface where the users specify the characteristics and coordinates of their home in order to obtain the probability to be above a given IRC with a corresponding index of confidence. Taking into account the health effects of 222Rn, our results have the potential to substantially improve the estimation of the effective dose from 222Rn delivered to the Swiss population.

Environmental data mining and modeling based on machine learning algorithms and geostatistics

The paper presents some contemporary approaches to spatial environmental data analysis. The main topics are concentrated on the decision-oriented problems of environmental spatial data mining and modeling: valorization and representativity of data with the help of exploratory data analysis, spatial predictions, probabilistic and risk mapping, development and application of conditional stochastic simulation models. The innovative part of the paper presents integrated/hybrid model-machine learning (ML) residuals sequential simulations-MLRSS. The models are based on multilayer perceptron and support vector regression ML algorithms used for modeling long-range spatial trends and sequential simulations of the residuals. NIL algorithms deliver non-linear solution for the spatial non-stationary problems, which are difficult for geostatistical approach. Geostatistical tools (variography) are used to characterize performance of ML algorithms, by analyzing quality and quantity of the spatially structured information extracted from data with ML algorithms. Sequential simulations provide efficient assessment of uncertainty and spatial variability. Case study from the Chernobyl fallouts illustrates the performance of the proposed model. It is shown that probability mapping, provided by the combination of ML data driven and geostatistical model based approaches, can be efficiently used in decision-making process. (C) 2003 Elsevier Ltd. All rights reserved.