Machine learning for geospatial data : algorithms, software tools and case studies

Résumé Cette thèse est consacrée à l'analyse, la modélisation et la visualisation de données environnementales à référence spatiale à l'aide d'algorithmes d'apprentissage automatique (Machine Learning). L'apprentissage automatique peut être considéré au sens large comme une sous-catégorie de l'intelligence artificielle qui concerne particulièrement le développement de techniques et d'algorithmes permettant à une machine d'apprendre à partir de données. Dans cette thèse, les algorithmes d'apprentissage automatique sont adaptés pour être appliqués à des données environnementales et à la prédiction spatiale. Pourquoi l'apprentissage automatique ? Parce que la majorité des algorithmes d'apprentissage automatiques sont universels, adaptatifs, non-linéaires, robustes et efficaces pour la modélisation. Ils peuvent résoudre des problèmes de classification, de régression et de modélisation de densité de probabilités dans des espaces à haute dimension, composés de variables informatives spatialisées (« géo-features ») en plus des coordonnées géographiques. De plus, ils sont idéaux pour être implémentés en tant qu'outils d'aide à la décision pour des questions environnementales allant de la reconnaissance de pattern à la modélisation et la prédiction en passant par la cartographie automatique. Leur efficacité est comparable au modèles géostatistiques dans l'espace des coordonnées géographiques, mais ils sont indispensables pour des données à hautes dimensions incluant des géo-features. Les algorithmes d'apprentissage automatique les plus importants et les plus populaires sont présentés théoriquement et implémentés sous forme de logiciels pour les sciences environnementales. Les principaux algorithmes décrits sont le Perceptron multicouches (MultiLayer Perceptron, MLP) - l'algorithme le plus connu dans l'intelligence artificielle, le réseau de neurones de régression généralisée (General Regression Neural Networks, GRNN), le réseau de neurones probabiliste (Probabilistic Neural Networks, PNN), les cartes auto-organisées (SelfOrganized Maps, SOM), les modèles à mixture Gaussiennes (Gaussian Mixture Models, GMM), les réseaux à fonctions de base radiales (Radial Basis Functions Networks, RBF) et les réseaux à mixture de densité (Mixture Density Networks, MDN). Cette gamme d'algorithmes permet de couvrir des tâches variées telle que la classification, la régression ou l'estimation de densité de probabilité. L'analyse exploratoire des données (Exploratory Data Analysis, EDA) est le premier pas de toute analyse de données. Dans cette thèse les concepts d'analyse exploratoire de données spatiales (Exploratory Spatial Data Analysis, ESDA) sont traités selon l'approche traditionnelle de la géostatistique avec la variographie expérimentale et selon les principes de l'apprentissage automatique. La variographie expérimentale, qui étudie les relations entre pairs de points, est un outil de base pour l'analyse géostatistique de corrélations spatiales anisotropiques qui permet de détecter la présence de patterns spatiaux descriptible par une statistique. L'approche de l'apprentissage automatique pour l'ESDA est présentée à travers l'application de la méthode des k plus proches voisins qui est très simple et possède d'excellentes qualités d'interprétation et de visualisation. Une part importante de la thèse traite de sujets d'actualité comme la cartographie automatique de données spatiales. Le réseau de neurones de régression généralisée est proposé pour résoudre cette tâche efficacement. Les performances du GRNN sont démontrées par des données de Comparaison d'Interpolation Spatiale (SIC) de 2004 pour lesquelles le GRNN bat significativement toutes les autres méthodes, particulièrement lors de situations d'urgence. La thèse est composée de quatre chapitres : théorie, applications, outils logiciels et des exemples guidés. Une partie importante du travail consiste en une collection de logiciels : Machine Learning Office. Cette collection de logiciels a été développée durant les 15 dernières années et a été utilisée pour l'enseignement de nombreux cours, dont des workshops internationaux en Chine, France, Italie, Irlande et Suisse ainsi que dans des projets de recherche fondamentaux et appliqués. Les cas d'études considérés couvrent un vaste spectre de problèmes géoenvironnementaux réels à basse et haute dimensionnalité, tels que la pollution de l'air, du sol et de l'eau par des produits radioactifs et des métaux lourds, la classification de types de sols et d'unités hydrogéologiques, la cartographie des incertitudes pour l'aide à la décision et l'estimation de risques naturels (glissements de terrain, avalanches). Des outils complémentaires pour l'analyse exploratoire des données et la visualisation ont également été développés en prenant soin de créer une interface conviviale et facile à l'utilisation. Machine Learning for geospatial data: algorithms, software tools and case studies Abstract The thesis is devoted to the analysis, modeling and visualisation of spatial environmental data using machine learning algorithms. In a broad sense machine learning can be considered as a subfield of artificial intelligence. It mainly concerns with the development of techniques and algorithms that allow computers to learn from data. In this thesis machine learning algorithms are adapted to learn from spatial environmental data and to make spatial predictions. Why machine learning? In few words most of machine learning algorithms are universal, adaptive, nonlinear, robust and efficient modeling tools. They can find solutions for the classification, regression, and probability density modeling problems in high-dimensional geo-feature spaces, composed of geographical space and additional relevant spatially referenced features. They are well-suited to be implemented as predictive engines in decision support systems, for the purposes of environmental data mining including pattern recognition, modeling and predictions as well as automatic data mapping. They have competitive efficiency to the geostatistical models in low dimensional geographical spaces but are indispensable in high-dimensional geo-feature spaces. The most important and popular machine learning algorithms and models interesting for geo- and environmental sciences are presented in details: from theoretical description of the concepts to the software implementation. The main algorithms and models considered are the following: multi-layer perceptron (a workhorse of machine learning), general regression neural networks, probabilistic neural networks, self-organising (Kohonen) maps, Gaussian mixture models, radial basis functions networks, mixture density networks. This set of models covers machine learning tasks such as classification, regression, and density estimation. Exploratory data analysis (EDA) is initial and very important part of data analysis. In this thesis the concepts of exploratory spatial data analysis (ESDA) is considered using both traditional geostatistical approach such as_experimental variography and machine learning. Experimental variography is a basic tool for geostatistical analysis of anisotropic spatial correlations which helps to understand the presence of spatial patterns, at least described by two-point statistics. A machine learning approach for ESDA is presented by applying the k-nearest neighbors (k-NN) method which is simple and has very good interpretation and visualization properties. Important part of the thesis deals with a hot topic of nowadays, namely, an automatic mapping of geospatial data. General regression neural networks (GRNN) is proposed as efficient model to solve this task. Performance of the GRNN model is demonstrated on Spatial Interpolation Comparison (SIC) 2004 data where GRNN model significantly outperformed all other approaches, especially in case of emergency conditions. The thesis consists of four chapters and has the following structure: theory, applications, software tools, and how-to-do-it examples. An important part of the work is a collection of software tools - Machine Learning Office. Machine Learning Office tools were developed during last 15 years and was used both for many teaching courses, including international workshops in China, France, Italy, Ireland, Switzerland and for realizing fundamental and applied research projects. Case studies considered cover wide spectrum of the real-life low and high-dimensional geo- and environmental problems, such as air, soil and water pollution by radionuclides and heavy metals, soil types and hydro-geological units classification, decision-oriented mapping with uncertainties, natural hazards (landslides, avalanches) assessments and susceptibility mapping. Complementary tools useful for the exploratory data analysis and visualisation were developed as well. The software is user friendly and easy to use.

Habitat fragmentation, ecology and sexual selection in forest bird species in Monteverde, Costa Rica

Abstract Forest fragmentation is often associated with local extinction and changes in species abundance patterns. The main topic of this thesis is the effect of forest fragmentation on birds in Monteverde, Costa Rica. This thesis also studies aspects of sexual selection and ecology of Long-tailed Manakins, Chiroxiphia Linearis. Chapter 1 investigates bird species assemblages in two degrees of forest fragmentation. It is shown that the distribution, abundance and diversity of forest bird species are strongly influenced by the amount of forest in the landscape matrix. Presence of cattle within the forest influences the presence of some bird species. The prevalence and intensity of ticks and blood parasites on birds in relation to fragmentation is described in Chapter 2. Overall tick prevalence is 3%. Understory birds are significantly more infested with ticks than species at intermediate heights. Tick prevalence on birds does not differ significantly between two degrees of forest fragmentation and individual tick loads tend to be higher in High- than in Low-fragmentation sites. Infestations by the blood parasites Haemoproteus sp. was low except in white-eared ground sparrow, Melozone leucotis, that is 28% and is significantly higher in High- than in Low-fragmentation sites. In chapter 3 results on the ecology and habitat movements of the Bare-necked Umbrellabird, Cephalopterus glabricollis, are presented. The abundance of umbrellabirds at high elevations during the breeding season coincides with the highest peak of fruit abundance. Birds leave the protected area during the non-breeding season moving to unprotected forest fragments. In chapter 4 ontogenetic changes in feather morphology through sexual maturity in Long-tailed Manakins are described. In adult males, rectrices length is positively correlated to testis volume. Changes in male morphology during ontogeny in the long-tailed manakin appear to be associated with their specific-display behaviours. Significant interpopulation differences in the morphology of Long-tailed Manakins are shown in chapter 5. These differences are more accentuated in morphological traits related to flight displays. A field experiment demonstrates that long rectrices impose flying costs for males and females. A reduction in flying ability was found to be strongest in males from a population presenting the highest degree of sexual dimorphism. Résumé La fragmentation des forêts est souvent associée avec des modifications dans l'abondance des espèces et des extinctions locales. Le thème principale de cette thèse est l'étude de l'effet de la fragmentation des forêts sur les oiseaux de Monteverde, Costa Rica. Elle décrit par ailleurs certains aspects de la sélection sexuelle et l'écologie du manakin à longue queue, Chiroxiphia linearis. Dans le Chapitre 1 je montre que la distribution, l'abondance et la diversité des assemblages d'oiseaux vivant dans la forêt sont fortement influencées pas le degré de fragmentation de celle ci. Par ailleurs, la présence ou l'absence de bétail dans les forêts influence la présence de certaines espèces d'oiseaux. Dans le chapitre 2 j'ai étudié la prévalence et l'intensité d'infestation par des tiques ainsi que la présence de parasites sanguins chez les oiseaux en relation avec la fragmentation des forêts. La prévalence globale de tiques est de 3 %, les oiseaux vivant au niveau du sol étaient plus souvent infectés par des tiques que les espèces se déplaçant à un niveau plus élevé. La prévalence de tiques sur les oiseaux n'était pas significativement différente entre les paysages avec différentes fragmentations. Les parasites sanguins du genre Haemoproteus sp. étaient présents à très basse fréquence à l'exception chez Melozone leucotis ou la prevalence était de 28% et significativement plus élevée chez les oiseaux vivant dans les forêts à forte fragmentation. Dans le Chapitre 3 je présente des résultats sur l'écologie et les mouvements entre habitats chez le "Bare-necked umbrellabird", Cephalopterus glabricollis. Cette espèce endémique du Costa Rica niche à haute altitude durant la période d'abondance des fruits et réalise une migration altitudinale vers des zones basses durant la saison de non reproduction. Dans le chapitre 4 je présente les changements ontogénétiques dans la morphologie du plumage des manakins à longue queue. Chez les mâles, les changements de morphologie semblent être associés avec leurs comportements de parade spécifiques. Dans le chapitre 5 je présente des différences morphologiques significative entre deux populations chez le manakin à longue queue et je montre que la capacité de vols chez les mâles est plus fortement influencée dans la population avec le degré de dimorphisme sexuel le plus prononcé.

Towards Machine Consciousness: Grounding Abstract Models as pi-Processes

We present a two-level model of concurrent communicating systems (CCS) to serve as a basis formachine consciousness. A language implementing threads within logic programming is ¯rstintroduced. This high-level framework allows for the de¯nition of abstract processes that can beexecuted on a virtual machine. We then look for a possible grounding of these processes into thebrain. Towards this end, we map abstract de¯nitions (including logical expressions representingcompiled knowledge) into a variant of the pi-calculus. We illustrate this approach through aseries of examples extending from a purely reactive behavior to patterns of consciousness.

Environmental data mining and modeling based on machine learning algorithms and geostatistics

The paper presents some contemporary approaches to spatial environmental data analysis. The main topics are concentrated on the decision-oriented problems of environmental spatial data mining and modeling: valorization and representativity of data with the help of exploratory data analysis, spatial predictions, probabilistic and risk mapping, development and application of conditional stochastic simulation models. The innovative part of the paper presents integrated/hybrid model-machine learning (ML) residuals sequential simulations-MLRSS. The models are based on multilayer perceptron and support vector regression ML algorithms used for modeling long-range spatial trends and sequential simulations of the residuals. NIL algorithms deliver non-linear solution for the spatial non-stationary problems, which are difficult for geostatistical approach. Geostatistical tools (variography) are used to characterize performance of ML algorithms, by analyzing quality and quantity of the spatially structured information extracted from data with ML algorithms. Sequential simulations provide efficient assessment of uncertainty and spatial variability. Case study from the Chernobyl fallouts illustrates the performance of the proposed model. It is shown that probability mapping, provided by the combination of ML data driven and geostatistical model based approaches, can be efficiently used in decision-making process. (C) 2003 Elsevier Ltd. All rights reserved.

Spatial predictions of land-use transitions and associated threats to biodiversity: the case of forest regrowth in mountain grasslands

Question Can we predict where forest regrowth caused by abandonment of agricultural activities is likely to occur? Can we assess how it may conflict with grassland diversity hotspots? Location Western Swiss Alps (4003210m a.s.l.). Methods We used statistical models to predict the location of land abandonment by farmers that is followed by forest regrowth in semi-natural grasslands of the Western Swiss Alps. Six modelling methods (GAM, GBM, GLM, RF, MDA, MARS) allowing binomial distribution were tested on two successive transitions occurring between three time periods. Models were calibrated using data on land-use change occurring between 1979 and 1992 as response, and environmental, accessibility and socio-economic variables as predictors, and these were validated for their capacity to predict the changes observed from 1992 to 2004. Projected probabilities of land-use change from an ensemble forecast of the six models were combined with a model of plant species richness based on a field inventory, allowing identification of critical grassland areas for the preservation of biodiversity. Results Models calibrated over the first land-use transition period predicted the second transition with reasonable accuracy. Forest regrowth occurs where cultivation costs are high and yield potential is low, i.e. on steeper slopes and at higher elevations. Overlaying species richness with land-use change predictions, we identified priority areas for the management and conservation of biodiversity at intermediate elevations. Conclusions Combining land-use change and biodiversity projections, we propose applied management measures for targeted/identified locations to limit the loss of biodiversity that could otherwise occur through loss of open habitats. The same approach could be applied to other types of land-use changes occurring in other ecosystems.

Personalized drug administrations using Support Vector Machine

The decision-making process regarding drug dose, regularly used in everyday medical practice, is critical to patients' health and recovery. It is a challenging process, especially for a drug with narrow therapeutic ranges, in which a medical doctor decides the quantity (dose amount) and frequency (dose interval) on the basis of a set of available patient features and doctor's clinical experience (a priori adaptation). Computer support in drug dose administration makes the prescription procedure faster, more accurate, objective, and less expensive, with a tendency to reduce the number of invasive procedures. This paper presents an advanced integrated Drug Administration Decision Support System (DADSS) to help clinicians/patients with the dose computing. Based on a support vector machine (SVM) algorithm, enhanced with the random sample consensus technique, this system is able to predict the drug concentration values and computes the ideal dose amount and dose interval for a new patient. With an extension to combine the SVM method and the explicit analytical model, the advanced integrated DADSS system is able to compute drug concentration-to-time curves for a patient under different conditions. A feedback loop is enabled to update the curve with a new measured concentration value to make it more personalized (a posteriori adaptation).

Insights into the biogeographical history of the Lower Guinea Forest Domain: evidence for the role of refugia in the intraspecific differentiation of Aucoumea klaineana.

Determining the biogeographical histories of rainforests is central to our understanding of the present distribution of tropical biodiversity. Ice age fragmentation of central African rainforests strongly influenced species distributions. Elevated areas characterized by higher species richness and endemism have been postulated to be Pleistocene forest refugia. However, it is often difficult to separate the effects of history and of present-day ecological conditions on diversity patterns at the interspecific level. Intraspecific genetic variation could yield new insights into history, because refugia hypotheses predict patterns not expected on the basis of contemporary environmental dynamics. Here, we test geographically explicit hypotheses of vicariance associated with the presence of putative refugia and provide clues about their location. We intensively sampled populations of Aucoumea klaineana, a forest tree sensitive to forest fragmentation, throughout its geographical range. Characterizing variation at 10 nuclear microsatellite loci, we were able to obtain phylogeographic data of unprecedented detail for this region. Using Bayesian clustering approaches, we demonstrated the presence of four differentiated genetic units. Their distribution matched that of forest refugia postulated from patterns of species richness and endemism. Our data also show differences in diversity dynamics at leading and trailing edges of the species' shifting distribution. Our results confirm predictions based on refugia hypotheses and cannot be explained on the basis of present-day ecological conditions.

The importance of sonographer experience and machine quality with regards to the role of musculoskeletal ultrasound in routine care of rheumatoid arthritis patients.

OBJECTIVES: Regarding recent progress, musculoskeletal ultrasound (US) will probably soon be integrated in standard care of patient with rheumatoid arthritis (RA). However, in daily care, quality of US machines and level of experience of sonographers are varied. We conducted a study to assess reproducibility and feasibility of an US scoring for RA, including US devices of different quality and rheumatologist with various levels of expertise in US as it would be in daily care. METHODS: The Swiss Sonography in Arthritis and Rheumatism (SONAR) group has developed a semi-quantitative score using OMERACT criteria for synovitis and erosion in RA. The score was taught to 108 rheumatologists trained in US. One year after the last workshop, 19 rheumatologists participated in the study. Scans were performed on 6 US machines ranging from low to high quality, each with a different patient. Weighted kappa was calculated for each pair of readers. RESULTS: Overall, the agreement was fair to moderate. Quality of device, experience of the sonographers and practice of the score before the study improved substantially the agreement. Agreement assessed on higher quality machine, among sonographers with good experience in US increased to substantial (median kappa for B-mode and Doppler: 0.64 and 0.41 for erosion). CONCLUSIONS: This study demonstrated feasibility and reproducibility of the Swiss US SONAR score for RA. Our results confirmed importance of the quality of US machine and the training of sonographers for the implementation of US scoring in the routine daily care of RA.

Correlated genetic effects on reproduction define a domestication syndrome in a forest tree.

Compared to natural selection, domestication implies a dramatic change in traits linked to fitness. A number of traits conferring fitness in the wild might be detrimental under domestication, and domesticated species typically differ from their ancestors in a set of traits known as the domestication syndrome. Specifically, trade-offs between growth and reproduction are well established across the tree of life. According to allocation theory, selection for growth rate is expected to indirectly alter life-history reproductive traits, diverting resources from reproduction to growth. Here we tested this hypothesis by examining the genetic change and correlated responses of reproductive traits as a result of selection for timber yield in the tree Pinus pinaster. Phenotypic selection was carried out in a natural population, and progenies from selected trees were compared with those of control trees in a common garden experiment. According to expectations, we detected a genetic change in important life-history traits due to selection. Specifically, threshold sizes for reproduction were much higher and reproductive investment relative to size significantly lower in the selected progenies just after a single artificial selection event. Our study helps to define the domestication syndrome in exploited forest trees and shows that changes affecting developmental pathways are relevant in domestication processes of long-lived plants.

Could machine learning improve the prediction of pelvic nodal status of prostate cancer patients? Preliminary results of a pilot study.

We tested and compared performances of Roach formula, Partin tables and of three Machine Learning (ML) based algorithms based on decision trees in identifying N+ prostate cancer (PC). 1,555 cN0 and 50 cN+ PC were analyzed. Results were also verified on an independent population of 204 operated cN0 patients, with a known pN status (187 pN0, 17 pN1 patients). ML performed better, also when tested on the surgical population, with accuracy, specificity, and sensitivity ranging between 48-86%, 35-91%, and 17-79%, respectively. ML potentially allows better prediction of the nodal status of PC, potentially allowing a better tailoring of pelvic irradiation.

Water availability predicts forest canopy height at the global scale.

The tendency of trees to grow taller with increasing water availability is common knowledge. Yet a robust, universal relationship between the spatial distribution of water availability and forest canopy height (H) is lacking. Here, we created a global water availability map by calculating an annual budget as the difference between precipitation (P) and potential evapotranspiration (PET) at a 1-km spatial resolution, and in turn correlated it with a global H map of the same resolution. Across forested areas over the globe, Hmean increased with P-PET, roughly: Hmean (m) = 19.3 + 0.077*(P-PET). Maximum forest canopy height also increased gradually from ~ 5 to ~ 50 m, saturating at ~ 45 m for P-PET > 500 mm. Forests were far from their maximum height potential in cold, boreal regions and in disturbed areas. The strong association between forest height and P-PET provides a useful tool when studying future forest dynamics under climate change, and in quantifying anthropogenic forest disturbance.

Forest Transition in Madagascar's Highlands: Initial Evidence and Implications

Madagascar is renowned for the loss of the forested habitat of lemurs and other species endemic to the island. Less well known is that in the highlands, a region often described as an environmental "basket-case" of fire-degraded, eroded grasslands, woody cover has been increasing for decades. Using information derived from publically available high- and medium-resolution satellites, this study characterizes tree cover dynamics in the highlands of Madagascar over the past two decades. Our results reveal heterogeneous patterns of increased tree cover on smallholder farms and village lands, spurred by a mix of endogenous and exogenous forces. The new trees play important roles in rural livelihoods, providing renewable supplies of firewood, charcoal, timber and other products and services, as well as defensible claims to land tenure in the context of a decline in the use of hillside commons for grazing. This study documents this nascent forest transition through Land Change Science techniques, and provides a prologue to political ecological analysis by setting these changes in their social and environmental context and interrogating the costs and benefits of the shift in rural livelihood strategies.