Data-driven clustering : new methods and applications

Abstract : This work is concerned with the development and application of novel unsupervised learning methods, having in mind two target applications: the analysis of forensic case data and the classification of remote sensing images. First, a method based on a symbolic optimization of the inter-sample distance measure is proposed to improve the flexibility of spectral clustering algorithms, and applied to the problem of forensic case data. This distance is optimized using a loss function related to the preservation of neighborhood structure between the input space and the space of principal components, and solutions are found using genetic programming. Results are compared to a variety of state-of--the-art clustering algorithms. Subsequently, a new large-scale clustering method based on a joint optimization of feature extraction and classification is proposed and applied to various databases, including two hyperspectral remote sensing images. The algorithm makes uses of a functional model (e.g., a neural network) for clustering which is trained by stochastic gradient descent. Results indicate that such a technique can easily scale to huge databases, can avoid the so-called out-of-sample problem, and can compete with or even outperform existing clustering algorithms on both artificial data and real remote sensing images. This is verified on small databases as well as very large problems. Résumé : Ce travail de recherche porte sur le développement et l'application de méthodes d'apprentissage dites non supervisées. Les applications visées par ces méthodes sont l'analyse de données forensiques et la classification d'images hyperspectrales en télédétection. Dans un premier temps, une méthodologie de classification non supervisée fondée sur l'optimisation symbolique d'une mesure de distance inter-échantillons est proposée. Cette mesure est obtenue en optimisant une fonction de coût reliée à la préservation de la structure de voisinage d'un point entre l'espace des variables initiales et l'espace des composantes principales. Cette méthode est appliquée à l'analyse de données forensiques et comparée à un éventail de méthodes déjà existantes. En second lieu, une méthode fondée sur une optimisation conjointe des tâches de sélection de variables et de classification est implémentée dans un réseau de neurones et appliquée à diverses bases de données, dont deux images hyperspectrales. Le réseau de neurones est entraîné à l'aide d'un algorithme de gradient stochastique, ce qui rend cette technique applicable à des images de très haute résolution. Les résultats de l'application de cette dernière montrent que l'utilisation d'une telle technique permet de classifier de très grandes bases de données sans difficulté et donne des résultats avantageusement comparables aux méthodes existantes.

Task 4: Testing Iowa Portland Cement Concrete Mixtures for the AASHTO Mechanistic-Empirical Pavement Design Procedure, May 2008

The present research project was designed to identify the typical Iowa material input values that are required by the Mechanistic-Empirical Pavement Design Guide (MEPDG) for the Level 3 concrete pavement design. It was also designed to investigate the existing equations that might be used to predict Iowa pavement concrete for the Level 2 pavement design. In this project, over 20,000 data were collected from the Iowa Department of Transportation (DOT) and other sources. These data, most of which were concrete compressive strength, slump, air content, and unit weight data, were synthesized and their statistical parameters (such as the mean values and standard variations) were analyzed. Based on the analyses, the typical input values of Iowa pavement concrete, such as 28-day compressive strength (f’c), splitting tensile strength (fsp), elastic modulus (Ec), and modulus of rupture (MOR), were evaluated. The study indicates that the 28-day MOR of Iowa concrete is 646 + 51 psi, very close to the MEPDG default value (650 psi). The 28-day Ec of Iowa concrete (based only on two available data of the Iowa Curling and Warping project) is 4.82 + 0.28x106 psi, which is quite different from the MEPDG default value (3.93 x106 psi); therefore, the researchers recommend re-evaluating after more Iowa test data become available. The drying shrinkage (εc) of a typical Iowa concrete (C-3WR-C20 mix) was tested at Concrete Technology Laboratory (CTL). The test results show that the ultimate shrinkage of the concrete is about 454 microstrain and the time for the concrete to reach 50% of ultimate shrinkage is at 32 days; both of these values are very close to the MEPDG default values. The comparison of the Iowa test data and the MEPDG default values, as well as the recommendations on the input values to be used in MEPDG for Iowa PCC pavement design, are summarized in Table 20 of this report. The available equations for predicting the above-mentioned concrete properties were also assembled. The validity of these equations for Iowa concrete materials was examined. Multiple-parameters nonlinear regression analyses, along with the artificial neural network (ANN) method, were employed to investigate the relationships among Iowa concrete material properties and to modify the existing equations so as to be suitable for Iowa concrete materials. However, due to lack of necessary data sets, the relationships between Iowa concrete properties were established based on the limited data from CP Tech Center’s projects and ISU classes only. The researchers suggest that the resulting relationships be used by Iowa pavement design engineers as references only. The present study furthermore indicates that appropriately documenting concrete properties, including flexural strength, elastic modulus, and information on concrete mix design, is essential for updating the typical Iowa material input values and providing rational prediction equations for concrete pavement design in the future.

Resting-State Functional Connectivity Emerges from Structurally and Dynamically Shaped Slow Linear Fluctuations.

Brain fluctuations at rest are not random but are structured in spatial patterns of correlated activity across different brain areas. The question of how resting-state functional connectivity (FC) emerges from the brain's anatomical connections has motivated several experimental and computational studies to understand structure-function relationships. However, the mechanistic origin of resting state is obscured by large-scale models' complexity, and a close structure-function relation is still an open problem. Thus, a realistic but simple enough description of relevant brain dynamics is needed. Here, we derived a dynamic mean field model that consistently summarizes the realistic dynamics of a detailed spiking and conductance-based synaptic large-scale network, in which connectivity is constrained by diffusion imaging data from human subjects. The dynamic mean field approximates the ensemble dynamics, whose temporal evolution is dominated by the longest time scale of the system. With this reduction, we demonstrated that FC emerges as structured linear fluctuations around a stable low firing activity state close to destabilization. Moreover, the model can be further and crucially simplified into a set of motion equations for statistical moments, providing a direct analytical link between anatomical structure, neural network dynamics, and FC. Our study suggests that FC arises from noise propagation and dynamical slowing down of fluctuations in an anatomically constrained dynamical system. Altogether, the reduction from spiking models to statistical moments presented here provides a new framework to explicitly understand the building up of FC through neuronal dynamics underpinned by anatomical connections and to drive hypotheses in task-evoked studies and for clinical applications.

Optimization of snowmaking in high mountains ski resort

Ski resorts are deploying more and more systems of artificial snow. These tools are necessary to ensure an important economic activity for the high alpine valleys. However, artificial snow raises important environmental issues that can be reduced by an optimization of its production. This paper presents a software prototype based on artificial intelligence to help ski resorts better manage their snowpack. It combines on one hand a General Neural Network for the analysis of the snow cover and the spatial prediction, with on the other hand a multiagent simulation of skiers for the analysis of the spatial impact of ski practice. The prototype has been tested on the ski resort of Verbier (Switzerland).

Multiple Kernel Learning of Environmental Data. Case study: Analysis and Mapping of Wind Fields

The paper presents the Multiple Kernel Learning (MKL) approach as a modelling and data exploratory tool and applies it to the problem of wind speed mapping. Support Vector Regression (SVR) is used to predict spatial variations of the mean wind speed from terrain features (slopes, terrain curvature, directional derivatives) generated at different spatial scales. Multiple Kernel Learning is applied to learn kernels for individual features and thematic feature subsets, both in the context of feature selection and optimal parameters determination. An empirical study on real-life data confirms the usefulness of MKL as a tool that enhances the interpretability of data-driven models.

Vulnerabilität und psychische Erkrankung

Vulnerability and psychic illness Based on a sample of 1701 college and university students from four different sites in Switzerland, the U.S., and Argentina, this study investigated the interrelationships between insufficient coping skills under chronic stress and impaired general health. We sought to develop standardised means for "early" identification of students at risk of mental health problems, as these students may benefit from "early" interventions before psychiatric symptoms develop and reach clinically relevant thresholds. All students completed two self-report questionnaires: the Coping Strategies Inventory "COPE" and the Zurich Health Questionnaire "ZHQ", with the latter assessing "regular exercises", "consumption behavior", "impaired physical health", "psychosomatic disturbances", and "impaired mental health". This data was subjected to structure analyses based on neural network approaches, using the different study sites' data subsets as independent "learning" and "test" samples. We found two highly stable COPE scales that quantified basic coping behaviour in terms of "activity-passivity" and "defeatism-resilience". The excellent reproducibility across study sites suggested that the new scales characterise socioculturally independent personality traits. Correlation analyses for external validation revealed a close relationship between high scores on the defeatism scale and impaired physical and mental health, hence underlining the scales' clinical relevance. Our results suggested in particular: (1.) the proposed method to be a powerful screening tool for early detection and prevention of psychiatric disorders; (2.) physical activity like regular exercises to play a critical role not only in preventing health problems but also in contributing to early intervention programs.

Roles of transient guidepost neurons in corpus callosum formation and guidance

RESUMENeurones transitoires jouant un rôle de cibles intermédiaires dans le guidage des axones du corps calleuxLe guidage axonal est une étape clé permettant aux neurones d'établir des connexions synaptiques et de s'intégrer dans un réseau neural fonctionnel de manière spécifique. Des cellules-cibles intermédiaires appelées « guidepost » aident les axones à parcourir de longues distances dans le cerveau en leur fournissant des informations directionnelles tout au long de leur trajet. Il a été démontré que des sous-populations de cellules gliales au niveau de la ligne médiane guident les axones du corps calleux (CC) d'un hémisphère vers l'autre. Bien qu'il fût observé que le CC en développement contenait aussi des neurones, leur rôle était resté jusqu'alors inconnu.La publication de nos résultats a montré que pendant le développement embryonnaire, le CC contient des glies mais aussi un nombre considérable de neurones glutamatergiques et GABAergiques, nécessaires à la formation du corps calleux (Niquille et al., PLoS Biology, 2009). Dans ce travail, j'ai utilisé des techniques de morphologie et d'imagerie confocale 3D pour définir le cadre neuro-anatomique de notre modèle. De plus, à l'aide de transplantations sur tranches in vitro, de co-explants, d'expression de siRNA dans des cultures de neurones primaires et d'analyse in vivo sur des souris knock-out, nous avons démontré que les neurones du CC guident les axones callosaux en partie grâce à l'action attractive du facteur de guidage Sema3C sur son récepteur Npn- 1.Récemment, nous avons étudié l'origine, les aspects dynamiques de ces processus, ainsi que les mécanismes moléculaires impliqués dans la mise en place de ce faisceau axonal (Niquille et al., soumis). Tout d'abord, nous avons précisé l'origine et l'identité des neurones guidepost GABAergiques du CC par une étude approfondie de traçage génétique in vivo. J'ai identifié, dans le CC, deux populations distinctes de neurones GABAergiques venant des éminences ganglionnaires médiane (MGE) et caudale (CGE). J'ai ensuite étudié plus en détail les interactions dynamiques entre neurones et axones du corps calleux par microscopie confocale en temps réel. Puis nous avons défini le rôle de chaque sous-population neuronale dans le guidage des axones callosaux et de manière intéressante les neurones GABAergic dérivés de la MGE comme ceux de la CGE se sont révélés avoir une action attractive pour les axones callosaux dans des expériences de transplantation. Enfin, nous avons clarifié la base moléculaire de ces mécanismes de guidage par FACS sorting associé à un large criblage génétique de molécules d'intérêt par une technique très sensible de RT-PCR et ensuite ces résultats ont été validés par hybridation in situ.Nous avons également étudié si les neurones guidepost du CC étaient impliqués dans son agénésie (absence de CC), présente dans nombreux syndromes congénitaux chez 1 humain. Le gène homéotique Aristaless (Arx) contrôle la migration des neurones GABAergiques et sa mutation conduit à de nombreuses pathologies humaines, notamment la lissencéphalie liée à IX avec organes génitaux anormaux (XLAG) et agénésie du CC. Fait intéressant, nous avons constaté qu'ARX est exprimé dans toutes les populations GABAergiques guidepost du CC et que les embryons mutant pour Arx présentent une perte drastique de ces neurones accompagnée de défauts de navigation des axones (Niquille et al., en préparation). En outre, nous avons découvert que les souris déficientes pour le facteur de transcription ciliogenic RFX3 souffrent d'une agénésie du CC associé avec des défauts de mise en place de la ligne médiane et une désorganisation secondaire des neurones glutamatergiques guidepost (Benadiba et al., submitted). Ceci suggère fortement l'implication potentielle des deux types de neurones guidepost dans l'agénésie du CC chez l'humain.Ainsi, mon travail de thèse révèle de nouvelles fonctions pour ces neurones transitoires dans le guidage axonal et apporte de nouvelles perspectives sur les rôles respectifs des cellules neuronales et gliales dans ce processus.ABSTRACTRole of transient guidepost neurons in corpus callosum development and guidanceAxonal guidance is a key step that allows neurons to build specific synaptic connections and to specifically integrate in a functional neural network. Intermediate targets or guidepost cells act as critical elements that help to guide axons through long distance in the brain and provide information all along their travel. Subpopulations of midline glial cells have been shown to guide corpus callosum (CC) axons to the contralateral cerebral hemisphere. While neuronal cells are also present in the developing corpus callosum, their role still remains elusive.Our published results unravelled that, during embryonic development, the CC is populated in addition to astroglia by numerous glutamatergic and GABAergic guidepost neurons that are essential for the correct midline crossing of callosal axons (Niquille et al., PLoS Biology, 2009). In this work, I have combined morphological and 3D confocal imaging techniques to define the neuro- anatomical frame of our system. Moreover, with the use of in vitro transplantations in slices, co- explant experiments, siRNA manipulations on primary neuronal culture and in vivo analysis of knock-out mice we have been able to demonstrate that CC neurons direct callosal axon outgrowth, in part through the attractive action of Sema3C on its Npn-1 receptor.Recently, we have studied the origin, the dynamic aspects of these processes as well as the molecular mechanisms involved in the establishment of this axonal tract (Niquille et al., submitted). First, we have clarified the origin and the identity of the CC GABAergic guidepost neurons using extensive in vivo cell fate-mapping experiments. We identified two distinct GABAergic neuronal subpopulations, originating from the medial (MGE) and caudal (CGE) ganglionic eminences. I then studied in more details the dynamic interactions between CC neurons and callosal axons by confocal time-lapse video microscopy and I have also further characterized the role of each guidepost neuronal subpopulation in callosal guidance. Interestingly, MGE- and CGE-derived GABAergic neurons are both attractive for callosal axons in transplantation experiments. Finally, we have dissected the molecular basis of these guidance mechanisms by using FACS sorting combined with an extensive genetic screen for molecules of interest by a sensitive RT-PCR technique, as well as, in situ hybridization.I have also investigated whether CC guidepost neurons are involved in agenesis of the CC which occurs in numerous human congenital syndromes. Aristaless-related homeobox gene (Arx) regulates GABAergic neuron migration and its mutation leads to numerous human pathologies including X-linked lissencephaly with abnormal genitalia (XLAG) and severe CC agenesis. Interestingly, I found that ARX is expressed in all the guidepost GABAergic neuronal populations of the CC and that Arx-/- embryos exhibit a drastic loss of CC GABAergic interneurons accompanied by callosal axon navigation defects (Niquille et al, in preparation). In addition, we discovered that mice deficient for the ciliogenic transcription factor RFX3 suffer from CC agenesis associated with early midline patterning defects and a secondary disorganisation of guidepost glutamatergic neurons (Benadiba et al., submitted). This strongly points out the potential implication of both types of guidepost neurons in human CC agenesis.Taken together, my thesis work reveals novel functions for transient neurons in axonal guidance and brings new perspectives on the respective roles of neuronal and glial cells in these processes.

Prediction of cross-recognition of peptide-HLA A2 by Melan-A-specific cytotoxic T lymphocytes using three-dimensional quantitative structure-activity relationships.

The cross-recognition of peptides by cytotoxic T lymphocytes is a key element in immunology and in particular in peptide based immunotherapy. Here we develop three-dimensional (3D) quantitative structure-activity relationships (QSARs) to predict cross-recognition by Melan-A-specific cytotoxic T lymphocytes of peptides bound to HLA A*0201 (hereafter referred to as HLA A2). First, we predict the structure of a set of self- and pathogen-derived peptides bound to HLA A2 using a previously developed ab initio structure prediction approach [Fagerberg et al., J. Mol. Biol., 521-46 (2006)]. Second, shape and electrostatic energy calculations are performed on a 3D grid to produce similarity matrices which are combined with a genetic neural network method [So et al., J. Med. Chem., 4347-59 (1997)] to generate 3D-QSAR models. The models are extensively validated using several different approaches. During the model generation, the leave-one-out cross-validated correlation coefficient (q (2)) is used as the fitness criterion and all obtained models are evaluated based on their q (2) values. Moreover, the best model obtained for a partitioned data set is evaluated by its correlation coefficient (r = 0.92 for the external test set). The physical relevance of all models is tested using a functional dependence analysis and the robustness of the models obtained for the entire data set is confirmed using y-randomization. Finally, the validated models are tested for their utility in the setting of rational peptide design: their ability to discriminate between peptides that only contain side chain substitutions in a single secondary anchor position is evaluated. In addition, the predicted cross-recognition of the mono-substituted peptides is confirmed experimentally in chromium-release assays. These results underline the utility of 3D-QSARs in peptide mimetic design and suggest that the properties of the unbound epitope are sufficient to capture most of the information to determine the cross-recognition.

Parametric Approach to Blind Deconvolution of Nonlinear Channels

A parametric procedure for the blind inversion of nonlinear channels is proposed, based on a recent method of blind source separation in nonlinear mixtures. Experiments show that the proposed algorithms perform efficiently, even in the presence of hard distortion. The method, based on the minimization of the output mutual information, needs the knowledge of log-derivative of input distribution (the so-called score function). Each algorithm consists of three adaptive blocks: one devoted to adaptive estimation of the score function, and two other blocks estimating the inverses of the linear and nonlinear parts of the channel, (quasi-)optimally adapted using the estimated score functions. This paper is mainly concerned by the nonlinear part, for which we propose two parametric models, the first based on a polynomial model and the second on a neural network, while [14, 15] proposed non-parametric approaches.

Automatic Recognition of Leaves by Shape Detection Pre-Processing with Ica

In this work we present a simulation of a recognition process with perimeter characterization of a simple plant leaves as a unique discriminating parameter. Data coding allowing for independence of leaves size and orientation may penalize performance recognition for some varieties. Border description sequences are then used to characterize the leaves. Independent Component Analysis (ICA) is then applied in order to study which is the best number of components to be considered for the classification task, implemented by means of an Artificial Neural Network (ANN). Obtained results with ICA as a pre-processing tool are satisfactory, and compared with some references our system improves the recognition success up to 80.8% depending on the number of considered independent components.

Empirical mode decomposition-based face recognition system

In this work we explore the multivariate empirical mode decomposition combined with a Neural Network classifier as technique for face recognition tasks. Images are simultaneously decomposed by means of EMD and then the distance between the modes of the image and the modes of the representative image of each class is calculated using three different distance measures. Then, a neural network is trained using 10- fold cross validation in order to derive a classifier. Preliminary results (over 98 % of classification rate) are satisfactory and will justify a deep investigation on how to apply mEMD for face recognition.

Correlations between negative-induced cerebral dynamics and idiopathic intellectual disability: an EEG study

Intellectual disability has long been associated with deficits in socio-emotional processing. However, studies investigating brain dynamics of maladaptive socio-emotional skills associated with intellectual disability are scarce. Here, we compared differences in brain activity between low intelligence quotient (I.Q.<75, N=13) and normal controls (N=15) while evaluating their subjective emotions. Positive (P) and negative (N) valenced pictures were presented one at a time to participants of both groups, at a rate of ¾. The task required that each participant evaluate their subjective emotion and press a predefined push-button when done, alternatively P and N. Electroencephalographic (EEG) signals were continuously recorded, and the 1000ms time window following each picture was analyzed offline for power in frequency domain. Alpha low (8-10Hz) and upper (10-13Hz) frequency bands were then compared for both groups and for both P and N emotions in 12 distributed scalp electrodes. The qualitative evaluation of emotions was similar between both groups, with constant longer reaction times for the low IQ participants. The EEG signal comparison shows marked power decrease in upper alpha frequency range for N emotions in low intelligence group. Otherwise no significant difference was noticed between low and normal IQ. Main findings of the present study are (1) results do not support the hypothesis that impairment in developmental intelligence roots in maladaptive emotional processing; (2) the strong alpha power suppression during negative-induced emotions suggests the involvement of an extended neural network and more effortful inhibition processes than positive ones. We call for further studies with a larger sample.

Statistical methods for river runoff prediction

Methods used to analyze one type of nonstationary stochastic processes?the periodically correlated process?are considered. Two methods of one-step-forward prediction of periodically correlated time series are examined. One-step-forward predictions made in accordance with an autoregression model and a model of an artificial neural network with one latent neuron layer and with an adaptation mechanism of network parameters in a moving time window were compared in terms of efficiency. The comparison showed that, in the case of prediction for one time step for time series of mean monthly water discharge, the simpler autoregression model is more efficient.

Prediction of soil orders with high spatial resolution: response of different classifiers to sampling density

The objective of this work was to evaluate sampling density on the prediction accuracy of soil orders, with high spatial resolution, in a viticultural zone of Serra Gaúcha, Southern Brazil. A digital elevation model (DEM), a cartographic base, a conventional soil map, and the Idrisi software were used. Seven predictor variables were calculated and read along with soil classes in randomly distributed points, with sampling densities of 0.5, 1, 1.5, 2, and 4 points per hectare. Data were used to train a decision tree (Gini) and three artificial neural networks: adaptive resonance theory, fuzzy ARTMap; self‑organizing map, SOM; and multi‑layer perceptron, MLP. Estimated maps were compared with the conventional soil map to calculate omission and commission errors, overall accuracy, and quantity and allocation disagreement. The decision tree was less sensitive to sampling density and had the highest accuracy and consistence. The SOM was the less sensitive and most consistent network. The MLP had a critical minimum and showed high inconsistency, whereas fuzzy ARTMap was more sensitive and less accurate. Results indicate that sampling densities used in conventional soil surveys can serve as a reference to predict soil orders in Serra Gaúcha.

Automatic Decision-Oriented Mapping of Pollution Data

The paper deals with the development and application of the methodology for automatic mapping of pollution/contamination data. General Regression Neural Network (GRNN) is considered in detail and is proposed as an efficient tool to solve this problem. The automatic tuning of isotropic and an anisotropic GRNN model using cross-validation procedure is presented. Results are compared with k-nearest-neighbours interpolation algorithm using independent validation data set. Quality of mapping is controlled by the analysis of raw data and the residuals using variography. Maps of probabilities of exceeding a given decision level and ?thick? isoline visualization of the uncertainties are presented as examples of decision-oriented mapping. Real case study is based on mapping of radioactively contaminated territories.