Machine learning for geospatial data : algorithms, software tools and case studies

Résumé Cette thèse est consacrée à l'analyse, la modélisation et la visualisation de données environnementales à référence spatiale à l'aide d'algorithmes d'apprentissage automatique (Machine Learning). L'apprentissage automatique peut être considéré au sens large comme une sous-catégorie de l'intelligence artificielle qui concerne particulièrement le développement de techniques et d'algorithmes permettant à une machine d'apprendre à partir de données. Dans cette thèse, les algorithmes d'apprentissage automatique sont adaptés pour être appliqués à des données environnementales et à la prédiction spatiale. Pourquoi l'apprentissage automatique ? Parce que la majorité des algorithmes d'apprentissage automatiques sont universels, adaptatifs, non-linéaires, robustes et efficaces pour la modélisation. Ils peuvent résoudre des problèmes de classification, de régression et de modélisation de densité de probabilités dans des espaces à haute dimension, composés de variables informatives spatialisées (« géo-features ») en plus des coordonnées géographiques. De plus, ils sont idéaux pour être implémentés en tant qu'outils d'aide à la décision pour des questions environnementales allant de la reconnaissance de pattern à la modélisation et la prédiction en passant par la cartographie automatique. Leur efficacité est comparable au modèles géostatistiques dans l'espace des coordonnées géographiques, mais ils sont indispensables pour des données à hautes dimensions incluant des géo-features. Les algorithmes d'apprentissage automatique les plus importants et les plus populaires sont présentés théoriquement et implémentés sous forme de logiciels pour les sciences environnementales. Les principaux algorithmes décrits sont le Perceptron multicouches (MultiLayer Perceptron, MLP) - l'algorithme le plus connu dans l'intelligence artificielle, le réseau de neurones de régression généralisée (General Regression Neural Networks, GRNN), le réseau de neurones probabiliste (Probabilistic Neural Networks, PNN), les cartes auto-organisées (SelfOrganized Maps, SOM), les modèles à mixture Gaussiennes (Gaussian Mixture Models, GMM), les réseaux à fonctions de base radiales (Radial Basis Functions Networks, RBF) et les réseaux à mixture de densité (Mixture Density Networks, MDN). Cette gamme d'algorithmes permet de couvrir des tâches variées telle que la classification, la régression ou l'estimation de densité de probabilité. L'analyse exploratoire des données (Exploratory Data Analysis, EDA) est le premier pas de toute analyse de données. Dans cette thèse les concepts d'analyse exploratoire de données spatiales (Exploratory Spatial Data Analysis, ESDA) sont traités selon l'approche traditionnelle de la géostatistique avec la variographie expérimentale et selon les principes de l'apprentissage automatique. La variographie expérimentale, qui étudie les relations entre pairs de points, est un outil de base pour l'analyse géostatistique de corrélations spatiales anisotropiques qui permet de détecter la présence de patterns spatiaux descriptible par une statistique. L'approche de l'apprentissage automatique pour l'ESDA est présentée à travers l'application de la méthode des k plus proches voisins qui est très simple et possède d'excellentes qualités d'interprétation et de visualisation. Une part importante de la thèse traite de sujets d'actualité comme la cartographie automatique de données spatiales. Le réseau de neurones de régression généralisée est proposé pour résoudre cette tâche efficacement. Les performances du GRNN sont démontrées par des données de Comparaison d'Interpolation Spatiale (SIC) de 2004 pour lesquelles le GRNN bat significativement toutes les autres méthodes, particulièrement lors de situations d'urgence. La thèse est composée de quatre chapitres : théorie, applications, outils logiciels et des exemples guidés. Une partie importante du travail consiste en une collection de logiciels : Machine Learning Office. Cette collection de logiciels a été développée durant les 15 dernières années et a été utilisée pour l'enseignement de nombreux cours, dont des workshops internationaux en Chine, France, Italie, Irlande et Suisse ainsi que dans des projets de recherche fondamentaux et appliqués. Les cas d'études considérés couvrent un vaste spectre de problèmes géoenvironnementaux réels à basse et haute dimensionnalité, tels que la pollution de l'air, du sol et de l'eau par des produits radioactifs et des métaux lourds, la classification de types de sols et d'unités hydrogéologiques, la cartographie des incertitudes pour l'aide à la décision et l'estimation de risques naturels (glissements de terrain, avalanches). Des outils complémentaires pour l'analyse exploratoire des données et la visualisation ont également été développés en prenant soin de créer une interface conviviale et facile à l'utilisation. Machine Learning for geospatial data: algorithms, software tools and case studies Abstract The thesis is devoted to the analysis, modeling and visualisation of spatial environmental data using machine learning algorithms. In a broad sense machine learning can be considered as a subfield of artificial intelligence. It mainly concerns with the development of techniques and algorithms that allow computers to learn from data. In this thesis machine learning algorithms are adapted to learn from spatial environmental data and to make spatial predictions. Why machine learning? In few words most of machine learning algorithms are universal, adaptive, nonlinear, robust and efficient modeling tools. They can find solutions for the classification, regression, and probability density modeling problems in high-dimensional geo-feature spaces, composed of geographical space and additional relevant spatially referenced features. They are well-suited to be implemented as predictive engines in decision support systems, for the purposes of environmental data mining including pattern recognition, modeling and predictions as well as automatic data mapping. They have competitive efficiency to the geostatistical models in low dimensional geographical spaces but are indispensable in high-dimensional geo-feature spaces. The most important and popular machine learning algorithms and models interesting for geo- and environmental sciences are presented in details: from theoretical description of the concepts to the software implementation. The main algorithms and models considered are the following: multi-layer perceptron (a workhorse of machine learning), general regression neural networks, probabilistic neural networks, self-organising (Kohonen) maps, Gaussian mixture models, radial basis functions networks, mixture density networks. This set of models covers machine learning tasks such as classification, regression, and density estimation. Exploratory data analysis (EDA) is initial and very important part of data analysis. In this thesis the concepts of exploratory spatial data analysis (ESDA) is considered using both traditional geostatistical approach such as_experimental variography and machine learning. Experimental variography is a basic tool for geostatistical analysis of anisotropic spatial correlations which helps to understand the presence of spatial patterns, at least described by two-point statistics. A machine learning approach for ESDA is presented by applying the k-nearest neighbors (k-NN) method which is simple and has very good interpretation and visualization properties. Important part of the thesis deals with a hot topic of nowadays, namely, an automatic mapping of geospatial data. General regression neural networks (GRNN) is proposed as efficient model to solve this task. Performance of the GRNN model is demonstrated on Spatial Interpolation Comparison (SIC) 2004 data where GRNN model significantly outperformed all other approaches, especially in case of emergency conditions. The thesis consists of four chapters and has the following structure: theory, applications, software tools, and how-to-do-it examples. An important part of the work is a collection of software tools - Machine Learning Office. Machine Learning Office tools were developed during last 15 years and was used both for many teaching courses, including international workshops in China, France, Italy, Ireland, Switzerland and for realizing fundamental and applied research projects. Case studies considered cover wide spectrum of the real-life low and high-dimensional geo- and environmental problems, such as air, soil and water pollution by radionuclides and heavy metals, soil types and hydro-geological units classification, decision-oriented mapping with uncertainties, natural hazards (landslides, avalanches) assessments and susceptibility mapping. Complementary tools useful for the exploratory data analysis and visualisation were developed as well. The software is user friendly and easy to use.

Nondeclarative learning in children with Specific Language Impairment: Predicting regularities in the visuomotor, phonological, and cognitive domains.

Ullman (2004) suggested that Specific Language Impairment (SLI) results from a general procedural learning deficit. In order to test this hypothesis, we investigated children with SLI via procedural learning tasks exploring the verbal, motor, and cognitive domains. Results showed that compared with a Control Group, the children with SLI (a) were unable to learn a phonotactic learning task, (b) were able but less efficiently to learn a motor learning task and (c) succeeded in a cognitive learning task. Regarding the motor learning task (Serial Reaction Time Task), reaction times were longer and learning slower than in controls. The learning effect was not significant in children with an associated Developmental Coordination Disorder (DCD), and future studies should consider comorbid motor impairment in order to clarify whether impairments are related to the motor rather than the language disorder. Our results indicate that a phonotactic learning but not a cognitive procedural deficit underlies SLI, thus challenging Ullmans' general procedural deficit hypothesis, like a few other recent studies.

On learning dynamics underlying the evolution of learning rules.

In order to understand the development of non-genetically encoded actions during an animal's lifespan, it is necessary to analyze the dynamics and evolution of learning rules producing behavior. Owing to the intrinsic stochastic and frequency-dependent nature of learning dynamics, these rules are often studied in evolutionary biology via agent-based computer simulations. In this paper, we show that stochastic approximation theory can help to qualitatively understand learning dynamics and formulate analytical models for the evolution of learning rules. We consider a population of individuals repeatedly interacting during their lifespan, and where the stage game faced by the individuals fluctuates according to an environmental stochastic process. Individuals adjust their behavioral actions according to learning rules belonging to the class of experience-weighted attraction learning mechanisms, which includes standard reinforcement and Bayesian learning as special cases. We use stochastic approximation theory in order to derive differential equations governing action play probabilities, which turn out to have qualitative features of mutator-selection equations. We then perform agent-based simulations to find the conditions where the deterministic approximation is closest to the original stochastic learning process for standard 2-action 2-player fluctuating games, where interaction between learning rules and preference reversal may occur. Finally, we analyze a simplified model for the evolution of learning in a producer-scrounger game, which shows that the exploration rate can interact in a non-intuitive way with other features of co-evolving learning rules. Overall, our analyses illustrate the usefulness of applying stochastic approximation theory in the study of animal learning.

Virulent bacterial infection improves aversive learning performance in Drosophila melanogaster.

Virulent infections are expected to impair learning ability, either as a direct consequence of stressed physiological state or as an adaptive response that minimizes diversion of energy from immune defense. This prediction has been well supported for mammals and bees. Here, we report an opposite result in Drosophila melanogaster. Using an odor-mechanical shock conditioning paradigm, we found that intestinal infection with bacterial pathogens Pseudomonas entomophila or Erwinia c. carotovora improved flies' learning performance after a 1h retention interval. Infection with P. entomophila (but not E. c. carotovora) also improved learning performance after 5 min retention. No effect on learning performance was detected for intestinal infections with an avirulent GacA mutant of P. entomophila or for virulent systemic (hemocoel) infection with E. c. carotovora. Assays of unconditioned responses to odorants and shock do not support a major role for changes in general responsiveness to stimuli in explaining the changes in learning performance, although differences in their specific salience for learning cannot be excluded. Our results demonstrate that the effects of pathogens on learning performance in insects are less predictable than suggested by previous studies, and support the notion that immune stress can sometimes boost cognitive abilities.

Auditory stimulation does not induce implicit memory during anaesthesia

Background and aim of the study: Formation of implicit memory during general anaesthesia is still debated. Perceptual learning is the ability to learn to perceive. In this study, an auditory perceptual learning paradigm, using frequency discrimination, was performed to investigate the implicit memory. It was hypothesized that auditory stimulation would successfully induce perceptual learning. Thus, initial thresholds of the frequency discrimination postoperative task should be lower for the stimulated group (group S) compared to the control group (group C). Material and method: Eighty-seven patients ASA I-III undergoing visceral and orthopaedic surgery during general anaesthesia lasting more than 60 minutes were recruited. The anaesthesia procedure was standardized (BISR monitoring included). Group S received auditory stimulation (2000 pure tones applied for 45 minutes) during the surgery. Twenty-four hours after the operation, both groups performed ten blocks of the frequency discrimination task. Mean of the thresholds for the first three blocks (T1) were compared between groups. Results: Mean age and BIS value of group S and group C are respectively 40 } 11 vs 42 } 11 years (p = 0,49) and 42 } 6 vs 41 } 8 (p = 0.87). T1 is respectively 31 } 33 vs 28 } 34 (p = 0.72) in group S and C. Conclusion: In our study, no implicit memory during general anaesthesia was demonstrated. This may be explained by a modulation of the auditory evoked potentials caused by the anaesthesia, or by an insufficient longer time of repetitive stimulation to induce perceptual learning.

Septal lesions impair the acquisition of a cued place navigation task: attentional or memory deficit?

These experiments were designed to analyze how medial septal lesions reducing the cholinergic innervation in the hippocampus might affect place learning. Rats with quisqualic lesions of the medial septal area (MS) were trained in a water maze and on a homing table where the escape position was located at a spatially fixed position and further indicated by a salient cue suspended above it. The lesioned rats were significantly impaired in reaching the cued escape platform during training. In addition rats, did not show any discrimination of the training sector during a probe trial in which no platform or cue was present. This impairment remained significant during further training in the absence of the cue. When the cued escape platform was located at an unpredictable spatial location, the MS-lesioned rats showed no deficit and spent more time under the cue than control rats during the probe trial. On the homing board, with a salient object in close proximity to the escape hole, the MS rats showed no deficit in escape latencies, although a significant reduction in spatial memory was observed. However, this was overcome by additional training in the absence of the cue. Under these conditions, rats with septal lesions were prone to develop a pure guidance strategy, whereas normal rats combined a guidance strategy with a memory of the escape position relative to more distant landmarks. The presence of a salient cue appeared to decrease attention to environmental landmarks, thus reducing spatial memory. These data confirm the general hypothesis that MS lesions reduce the capacity to rely on a representation of the relation between several landmarks with different salience.

Etalonnage de trois épreuves de recognition continue : Effets de l'âge, du sexe et du niveau socio-professionnel sur les mesures de discrimination et de biais de réponse. [Standardization of three continuous recognition test forms - Effects of age, sex and socio-economic level on measurements of discrimination and response bias.]

Nous présontons l'étalonnage d'un test mnésique de recognition dans un échantillon de 180 adultes francophones de la Suisse Romande. Le test comprend trois formes utilisant un matériel verbal (mots) ou non verbal (visages ou paysages). Une attention particulière est accordée à l'âge dans la présentation des résultats. Celui-ci affecte plus précocement et plus intensément la performance aux formes non verbales qu'à la forme verbale du test. Il induit également une importante augmentation du nombre de fausses reconnaissances pour les formes non verbales.

Spine trabecular bone score subsequent to bone mineral density improves fracture discrimination in women.

Bone mineral density (BMD) measured by dual-energy X-ray absorptiometry (DXA) is used to diagnose osteoporosis and assess fracture risk. However, DXA cannot evaluate trabecular microarchitecture. This study used a novel software program (TBS iNsight; Med-Imaps, Geneva, Switzerland) to estimate bone texture (trabecular bone score [TBS]) from standard spine DXA images. We hypothesized that TBS assessment would differentiate women with low trauma fracture from those without. In this study, TBS was performed blinded to fracture status on existing research DXA lumbar spine (LS) images from 429 women. Mean participant age was 71.3 yr, and 158 had prior fractures. The correlation between LS BMD and TBS was low (r = 0.28), suggesting these parameters reflect different bone properties. Age- and body mass index-adjusted odds ratios (ORs) ranged from 1.36 to 1.63 for LS or hip BMD in discriminating women with low trauma nonvertebral and vertebral fractures. TBS demonstrated ORs from 2.46 to 2.49 for these respective fractures; these remained significant after lowest BMD T-score adjustment (OR = 2.38 and 2.44). Seventy-three percent of all fractures occurred in women without osteoporosis (BMD T-score > -2.5); 72% of these women had a TBS score below the median, thereby appropriately classified them as being at increased risk. In conclusion, TBS assessment enhances DXA by evaluating trabecular pattern and identifying individuals with vertebral or low trauma fracture. TBS identifies 66-70% of women with fracture who were not classified with osteoporosis by BMD alone.

Learning in the working place: the educational potential of a multihead microscope in pathology postgraduate training.

Training future pathologists is an important mission of many hospital anatomic pathology departments. Apprenticeship-a process in which learning and teaching tightly intertwine with daily work, is one of the main educational methods in use in postgraduate medical training. However, patient care, including pathological diagnosis, often comes first, diagnostic priorities prevailing over educational ones. Recognition of the unique educational opportunities is a prerequisite for enhancing the postgraduate learning experience. The aim of this paper is to draw attention of senior pathologists with a role as supervisor in postgraduate training on the potential educational value of a multihead microscope, a common setting in pathology departments. After reporting on an informal observation of senior and junior pathologists' meetings around the multihead microscope in our department, we review the literature on current theories of learning to provide support to the high potential educational value of these meetings for postgraduate training in pathology. We also draw from the literature on learner-centered teaching some recommendations to better support learning in this particular context. Finally, we propose clues for further studies and effective instruction during meetings around a multihead microscope.

Metabolic activation pattern of distinct hippocampal subregions during spatial learning and memory retrieval.

Activation dynamics of hippocampal subregions during spatial learning and their interplay with neocortical regions is an important dimension in the understanding of hippocampal function. Using the (14C)-2-deoxyglucose autoradiographic method, we have characterized the metabolic changes occurring in hippocampal subregions in mice while learning an eight-arm radial maze task. Autoradiogram densitometry revealed a heterogeneous and evolving pattern of enhanced metabolic activity throughout the hippocampus during the training period and on recall. In the early stages of training, activity was enhanced in the CA1 area from the intermediate portion to the posterior end as well as in the CA3 area within the intermediate portion of the hippocampus. At later stages, CA1 and CA3 activations spread over the entire longitudinal axis, while dentate gyrus (DG) activation occurred from the anterior to the intermediate zone. Activation of the retrosplenial cortex but not the amygdala was also observed during the learning process. On recall, only DG activation was observed in the same anterior part of the hippocampus. These results suggest the existence of a functional segmentation of the hippocampus, each subregion being dynamically but also differentially recruited along the acquisition, consolidation, and retrieval process in parallel with some neocortical sites.

Ibotenate Lesions of Hippocampus and/or Subiculum: Dissociating Components of Allocentric Spatial Learning

This study examined the effects of ibotenic acid-induced lesions of the hippocampus, subiculum and hippocampus +/- subiculum upon the capacity of rats to learn and perform a series of allocentric spatial learning tasks in an open-field water maze. The lesions were made by infusing small volumes of the neurotoxin at a total of 26 (hippocampus) or 20 (subiculum) sites intended to achieve complete target cell loss but minimal extratarget damage. The regional extent and axon-sparing nature of these lesions was evaluated using both cresyl violet and Fink - Heimer stained sections. The behavioural findings indicated that both the hippocampus and subiculum lesions caused impairment to the initial postoperative acquisition of place navigation but did not prevent eventual learning to levels of performance almost as effective as those of controls. However, overtraining of the hippocampus + subiculum lesioned rats did not result in significant place learning. Qualitative observations of the paths taken to find a hidden escape platform indicated that different strategies were deployed by hippocampal and subiculum lesioned groups. Subsequent training on a delayed matching to place task revealed a deficit in all lesioned groups across a range of sample choice intervals, but the subiculum lesioned group was less impaired than the group with the hippocampal lesion. Finally, unoperated control rats given both the initial training and overtraining were later given either a hippocampal lesion or sham surgery. The hippocampal lesioned rats were impaired during a subsequent retention/relearning phase. Together, these findings suggest that total hippocampal cell loss may cause a dual deficit: a slower rate of place learning and a separate navigational impairment. The prospect of unravelling dissociable components of allocentric spatial learning is discussed.

Introducing blended learning into a Master Program in Nursing Sciences: a first exploration of a holistic evaluation framework

This contribution presents the first stage of a project to assist the transition of a traditional to a blended program in higher nursing education. We shall describe the goals and context of this project, present the evaluation framework, discuss some early results and then discuss the usefulness of the first version of the evaluation framework.

Environmental data mining and modeling based on machine learning algorithms and geostatistics

The paper presents some contemporary approaches to spatial environmental data analysis. The main topics are concentrated on the decision-oriented problems of environmental spatial data mining and modeling: valorization and representativity of data with the help of exploratory data analysis, spatial predictions, probabilistic and risk mapping, development and application of conditional stochastic simulation models. The innovative part of the paper presents integrated/hybrid model-machine learning (ML) residuals sequential simulations-MLRSS. The models are based on multilayer perceptron and support vector regression ML algorithms used for modeling long-range spatial trends and sequential simulations of the residuals. NIL algorithms deliver non-linear solution for the spatial non-stationary problems, which are difficult for geostatistical approach. Geostatistical tools (variography) are used to characterize performance of ML algorithms, by analyzing quality and quantity of the spatially structured information extracted from data with ML algorithms. Sequential simulations provide efficient assessment of uncertainty and spatial variability. Case study from the Chernobyl fallouts illustrates the performance of the proposed model. It is shown that probability mapping, provided by the combination of ML data driven and geostatistical model based approaches, can be efficiently used in decision-making process. (C) 2003 Elsevier Ltd. All rights reserved.

Fc block treatment, dead cells exclusion, and cell aggregates discrimination concur to prevent phenotypical artifacts in the analysis of subpopulations of tumor-infiltrating CD11b(+) myelomonocytic cells.

It is well established that cancer cells can recruit CD11b(+) myeloid cells to promote tumor angiogenesis and tumor growth. Increasing interest has emerged on the identification of subpopulations of tumor-infiltrating CD11b(+) myeloid cells using flow cytometry techniques. In the literature, however, discrepancies exist on the phenotype of these cells (Coffelt et al., Am J Pathol 2010;176:1564-1576). Since flow cytometry analysis requires particular precautions for accurate sample preparation and trustable data acquisition, analysis, and interpretation, some discrepancies might be due to technical reasons rather than biological grounds. We used the syngenic orthotopic 4T1 mammary tumor model in immunocompetent BALB/c mice to analyze and compare the phenotype of CD11b(+) myeloid cells isolated from peripheral blood and from tumors, using six-color flow cytometry. We report here that the nonspecific antibody binding through Fc receptors, the presence of dead cells and cell doublets in tumor-derived samples concur to generate artifacts in the phenotype of tumor-infiltrating CD11b(+) subpopulations. We show that the heterogeneity of tumor-infiltrating CD11b(+) subpopulations analyzed without particular precautions was greatly reduced upon Fc block treatment, dead cells, and cell doublets exclusion. Phenotyping of tumor-infiltrating CD11b(+) cells was particularly sensitive to these parameters compared to circulating CD11b(+) cells. Taken together, our results identify Fc block treatment, dead cells, and cell doublets exclusion as simple but crucial steps for the proper analysis of tumor-infiltrating CD11b(+) cell populations.