Machine learning for geospatial data : algorithms, software tools and case studies

Résumé Cette thèse est consacrée à l'analyse, la modélisation et la visualisation de données environnementales à référence spatiale à l'aide d'algorithmes d'apprentissage automatique (Machine Learning). L'apprentissage automatique peut être considéré au sens large comme une sous-catégorie de l'intelligence artificielle qui concerne particulièrement le développement de techniques et d'algorithmes permettant à une machine d'apprendre à partir de données. Dans cette thèse, les algorithmes d'apprentissage automatique sont adaptés pour être appliqués à des données environnementales et à la prédiction spatiale. Pourquoi l'apprentissage automatique ? Parce que la majorité des algorithmes d'apprentissage automatiques sont universels, adaptatifs, non-linéaires, robustes et efficaces pour la modélisation. Ils peuvent résoudre des problèmes de classification, de régression et de modélisation de densité de probabilités dans des espaces à haute dimension, composés de variables informatives spatialisées (« géo-features ») en plus des coordonnées géographiques. De plus, ils sont idéaux pour être implémentés en tant qu'outils d'aide à la décision pour des questions environnementales allant de la reconnaissance de pattern à la modélisation et la prédiction en passant par la cartographie automatique. Leur efficacité est comparable au modèles géostatistiques dans l'espace des coordonnées géographiques, mais ils sont indispensables pour des données à hautes dimensions incluant des géo-features. Les algorithmes d'apprentissage automatique les plus importants et les plus populaires sont présentés théoriquement et implémentés sous forme de logiciels pour les sciences environnementales. Les principaux algorithmes décrits sont le Perceptron multicouches (MultiLayer Perceptron, MLP) - l'algorithme le plus connu dans l'intelligence artificielle, le réseau de neurones de régression généralisée (General Regression Neural Networks, GRNN), le réseau de neurones probabiliste (Probabilistic Neural Networks, PNN), les cartes auto-organisées (SelfOrganized Maps, SOM), les modèles à mixture Gaussiennes (Gaussian Mixture Models, GMM), les réseaux à fonctions de base radiales (Radial Basis Functions Networks, RBF) et les réseaux à mixture de densité (Mixture Density Networks, MDN). Cette gamme d'algorithmes permet de couvrir des tâches variées telle que la classification, la régression ou l'estimation de densité de probabilité. L'analyse exploratoire des données (Exploratory Data Analysis, EDA) est le premier pas de toute analyse de données. Dans cette thèse les concepts d'analyse exploratoire de données spatiales (Exploratory Spatial Data Analysis, ESDA) sont traités selon l'approche traditionnelle de la géostatistique avec la variographie expérimentale et selon les principes de l'apprentissage automatique. La variographie expérimentale, qui étudie les relations entre pairs de points, est un outil de base pour l'analyse géostatistique de corrélations spatiales anisotropiques qui permet de détecter la présence de patterns spatiaux descriptible par une statistique. L'approche de l'apprentissage automatique pour l'ESDA est présentée à travers l'application de la méthode des k plus proches voisins qui est très simple et possède d'excellentes qualités d'interprétation et de visualisation. Une part importante de la thèse traite de sujets d'actualité comme la cartographie automatique de données spatiales. Le réseau de neurones de régression généralisée est proposé pour résoudre cette tâche efficacement. Les performances du GRNN sont démontrées par des données de Comparaison d'Interpolation Spatiale (SIC) de 2004 pour lesquelles le GRNN bat significativement toutes les autres méthodes, particulièrement lors de situations d'urgence. La thèse est composée de quatre chapitres : théorie, applications, outils logiciels et des exemples guidés. Une partie importante du travail consiste en une collection de logiciels : Machine Learning Office. Cette collection de logiciels a été développée durant les 15 dernières années et a été utilisée pour l'enseignement de nombreux cours, dont des workshops internationaux en Chine, France, Italie, Irlande et Suisse ainsi que dans des projets de recherche fondamentaux et appliqués. Les cas d'études considérés couvrent un vaste spectre de problèmes géoenvironnementaux réels à basse et haute dimensionnalité, tels que la pollution de l'air, du sol et de l'eau par des produits radioactifs et des métaux lourds, la classification de types de sols et d'unités hydrogéologiques, la cartographie des incertitudes pour l'aide à la décision et l'estimation de risques naturels (glissements de terrain, avalanches). Des outils complémentaires pour l'analyse exploratoire des données et la visualisation ont également été développés en prenant soin de créer une interface conviviale et facile à l'utilisation. Machine Learning for geospatial data: algorithms, software tools and case studies Abstract The thesis is devoted to the analysis, modeling and visualisation of spatial environmental data using machine learning algorithms. In a broad sense machine learning can be considered as a subfield of artificial intelligence. It mainly concerns with the development of techniques and algorithms that allow computers to learn from data. In this thesis machine learning algorithms are adapted to learn from spatial environmental data and to make spatial predictions. Why machine learning? In few words most of machine learning algorithms are universal, adaptive, nonlinear, robust and efficient modeling tools. They can find solutions for the classification, regression, and probability density modeling problems in high-dimensional geo-feature spaces, composed of geographical space and additional relevant spatially referenced features. They are well-suited to be implemented as predictive engines in decision support systems, for the purposes of environmental data mining including pattern recognition, modeling and predictions as well as automatic data mapping. They have competitive efficiency to the geostatistical models in low dimensional geographical spaces but are indispensable in high-dimensional geo-feature spaces. The most important and popular machine learning algorithms and models interesting for geo- and environmental sciences are presented in details: from theoretical description of the concepts to the software implementation. The main algorithms and models considered are the following: multi-layer perceptron (a workhorse of machine learning), general regression neural networks, probabilistic neural networks, self-organising (Kohonen) maps, Gaussian mixture models, radial basis functions networks, mixture density networks. This set of models covers machine learning tasks such as classification, regression, and density estimation. Exploratory data analysis (EDA) is initial and very important part of data analysis. In this thesis the concepts of exploratory spatial data analysis (ESDA) is considered using both traditional geostatistical approach such as_experimental variography and machine learning. Experimental variography is a basic tool for geostatistical analysis of anisotropic spatial correlations which helps to understand the presence of spatial patterns, at least described by two-point statistics. A machine learning approach for ESDA is presented by applying the k-nearest neighbors (k-NN) method which is simple and has very good interpretation and visualization properties. Important part of the thesis deals with a hot topic of nowadays, namely, an automatic mapping of geospatial data. General regression neural networks (GRNN) is proposed as efficient model to solve this task. Performance of the GRNN model is demonstrated on Spatial Interpolation Comparison (SIC) 2004 data where GRNN model significantly outperformed all other approaches, especially in case of emergency conditions. The thesis consists of four chapters and has the following structure: theory, applications, software tools, and how-to-do-it examples. An important part of the work is a collection of software tools - Machine Learning Office. Machine Learning Office tools were developed during last 15 years and was used both for many teaching courses, including international workshops in China, France, Italy, Ireland, Switzerland and for realizing fundamental and applied research projects. Case studies considered cover wide spectrum of the real-life low and high-dimensional geo- and environmental problems, such as air, soil and water pollution by radionuclides and heavy metals, soil types and hydro-geological units classification, decision-oriented mapping with uncertainties, natural hazards (landslides, avalanches) assessments and susceptibility mapping. Complementary tools useful for the exploratory data analysis and visualisation were developed as well. The software is user friendly and easy to use.

Soil compaction by machine traffic and least limiting water range related to soybean yield

The research aimed to evaluate machine traffic effect on soil compaction and the least limiting water range related to soybean cultivar yields, during two years, in a Haplustox soil. The six treatments were related to tractor (11 Mg weight) passes by the same place: T0, no compaction; and T1*, 1; T1, 1; T2, 2; T4, 4 and T6, 6. In the treatment T1*, the compaction occurred when soil was dried, in 2003/2004, and with a 4 Mg tractor in 2004/2005. Soybean yield was evaluated in relation to soil compaction during two agricultural years in completely randomized design (compaction levels); however, in the second year, there was a factorial scheme (compaction levels, with and without irrigation), with four replicates represented by 9 m² plots. In the first year, soybean [Glycine max (L.) Merr.] cultivar IAC Foscarim 31 was cultivated without irrigation; and in the second year, IAC Foscarim 31 and MG/BR 46 (Conquista) cultivars were cultivated with and without irrigation. Machine traffic causes compaction and reduces soybean yield for soil penetration resistance between 1.64 to 2.35 MPa, and bulk density between 1.50 to 1.53 Mg m-3. Soil bulk density from which soybean cultivar yields decrease is lower than the critical one reached at least limiting water range (LLWR =/ 0).

Towards Machine Consciousness: Grounding Abstract Models as pi-Processes

We present a two-level model of concurrent communicating systems (CCS) to serve as a basis formachine consciousness. A language implementing threads within logic programming is ¯rstintroduced. This high-level framework allows for the de¯nition of abstract processes that can beexecuted on a virtual machine. We then look for a possible grounding of these processes into thebrain. Towards this end, we map abstract de¯nitions (including logical expressions representingcompiled knowledge) into a variant of the pi-calculus. We illustrate this approach through aseries of examples extending from a purely reactive behavior to patterns of consciousness.

Environmental data mining and modeling based on machine learning algorithms and geostatistics

The paper presents some contemporary approaches to spatial environmental data analysis. The main topics are concentrated on the decision-oriented problems of environmental spatial data mining and modeling: valorization and representativity of data with the help of exploratory data analysis, spatial predictions, probabilistic and risk mapping, development and application of conditional stochastic simulation models. The innovative part of the paper presents integrated/hybrid model-machine learning (ML) residuals sequential simulations-MLRSS. The models are based on multilayer perceptron and support vector regression ML algorithms used for modeling long-range spatial trends and sequential simulations of the residuals. NIL algorithms deliver non-linear solution for the spatial non-stationary problems, which are difficult for geostatistical approach. Geostatistical tools (variography) are used to characterize performance of ML algorithms, by analyzing quality and quantity of the spatially structured information extracted from data with ML algorithms. Sequential simulations provide efficient assessment of uncertainty and spatial variability. Case study from the Chernobyl fallouts illustrates the performance of the proposed model. It is shown that probability mapping, provided by the combination of ML data driven and geostatistical model based approaches, can be efficiently used in decision-making process. (C) 2003 Elsevier Ltd. All rights reserved.

Shallow-transfer rule-based machine translation for Czech to Polish

This article describes the developmentof an Open Source shallow-transfer machine translation system from Czech to Polish in theApertium platform. It gives details ofthe methods and resources used in contructingthe system. Although the resulting system has quite a high error rate, it is still competitive with other systems.

Automatic acquisition of Named Entities for Rule-Based Machine Translation

This paper proposes to enrich RBMTdictionaries with Named Entities(NEs) automatically acquired fromWikipedia. The method is appliedto the Apertium English-Spanishsystem and its performance comparedto that of Apertium with and withouthandtagged NEs. The system withautomatic NEs outperforms the onewithout NEs, while results vary whencompared to a system with handtaggedNEs (results are comparable forSpanish to English but slightly worstfor English to Spanish). Apart fromthat, adding automatic NEs contributesto decreasing the amount of unknownterms by more than 10%.

Taking on new challenges in multi-word unit processing for Machine Translation

This paper discusses the qualitativecomparative evaluation performed on theresults of two machine translation systemswith different approaches to the processing ofmulti-word units. It proposes a solution forovercoming the difficulties multi-word unitspresent to machine translation by adopting amethodology that combines the lexicongrammar approach with OpenLogos ontologyand semantico-syntactic rules. The paper alsodiscusses the importance of a qualitativeevaluation metrics to correctly evaluate theperformance of machine translation engineswith regards to multi-word units.

A Widely Used Machine Translation Service and its Migration to a Free/Open-Source Solution : the Case of Softcatalà

Softcatalà is a non-profit associationcreated more than 10 years ago to fightthe marginalisation of the Catalan languagein information and communicationtechnologies. It has led the localisationof many applications and thecreation of a website which allows itsusers to translate texts between Spanishand Catalan using an external closed-sourcetranslation engine. Recently,the closed-source translation back-endhas been replaced by a free/open-sourcesolution completely managed by Softcatalà: the Apertium machine translationplatform and the ScaleMT web serviceframework. Thanks to the opennessof the new solution, it is possibleto take advantage of the huge amount ofusers of the Softcatalà translation serviceto improve it, using a series ofmethods presented in this paper. In addition,a study of the translations requestedby the users has been carriedout, and it shows that the translationback-end change has not affected theusage patterns.

Hybrid model of a double-fed induction machine and back-to-back converter

This report details the port interconnection of two subsystems: a power electronics subsystem (a back-to-back AC/AC converter (B2B), coupled to a phase of the power grid), and an electromechanical subsystem (a doubly-fed induction machine (DFIM), coupled mechanically to a flywheel and electrically to the power grid and to a local varying load). Both subsystems have been essentially described in previous reports (deliverables D 0.5 and D 4.3.1), although some previously unpublished details are presented here. The B2B is a variable structure system (VSS), due to the presence of control-actuated switches: however from a modelling and simulation, as well as a control-design, point of view, it is sensible to consider modulated transformers (MTF in the bond-graph language) instead of the pairs of complementary switches. The port-Hamiltonian models of both subsystems are presents and coupled through a power-preserving interconnection, and the Hamiltonian description of the whole system is obtained; detailed bond-graphs of all the subsystems and the complete system are provided.

Energy-based modelling and simulation of the interconnection of a back-to-back converter and a doubly-fed induction machine

This paper describes the port interconnection of two subsystems: a power electronics subsystem (a back-to-back AC/CA converter (B2B), coupled to a phase of the power grid), and an electromechanical subsystem (a doubly-fed induction machine (DFIM). The B2B is a variable structure system (VSS), due to presence of control-actuated switches: however, from a modelling simulation, as well as a control-design, point of view, it is sensible to consider modulated transformers (MTF in the bond graph language) instead of the pairs of complementary switches. The port-Hamiltonian models of both subsystems are presented and, using a power-preserving interconnection, the Hamiltonian description of the whole system is obtained; detailed bond graphs of all subsystems and the complete system are also provided. Using passivity-based controllers computed in the Hamiltonian formalism for both subsystems, the whole model is simulated; simulations are run to rest the correctness and efficiency of the Hamiltonian network modelling approach used in this work.

Impingement jet cooling of end windings in a high-speed electric machine

The rotational speed of high-speed electric machines is over 15 000 rpm. These machines are compact in size when compared to the power rate. As a consequence, the heat fluxes are at a high level and the adequacy of cooling becomes an important design criterion. In the high-speed machines, the air gap between the stator and rotor is a narrow flow channel. The cooling air is produced with a fan and the flow is then directed to the air gap. The flow in the gap does not provide sufficient cooling for the stator end windings, and therefore additional cooling is required. This study investigates the heat transfer and flow fields around the coil end windings when cooling jets are used. As a result, an innovative and new assembly is introduced for the cooling jets, with the benefits of a reduced amount of hot spots, a lower pressure drop, and hence a lower power need for the cooling fan. The gained information can also be applied to improve the cooling of electric machines through geometry modifications. The objective of the research is to determine the locations of the hot spots and to find out induced pressure losses with different jet alternatives. Several possibilities to arrange the extra cooling are considered. In the suggested approach cooling is provided by using a row of air jets. The air jets have three main tasks: to cool the coils effectively by direct impingement jets, to increase and cool down the flow that enters the coil end space through the air gap, and to ensure the correct distribution of the flow by forming an air curtain with additional jets. One important aim of this study is the arrangement of cooling jets in such manner that hot spots can be avoided to wide extent. This enables higher power density in high-speed motors. This cooling system can also be applied to the ordinary electric machines when efficient cooling is needed. The numerical calculations have been performed using a commercial Computational Fluid Dynamics software. Two geometries have been generated: cylindrical for the studied machine and Cartesian for the experimental model. The main parameters include the positions, arrangements and number of jets, the jet diameters, and the jet velocities. The investigated cases have been tested with two widely used turbulence models and using a computational grid of over 500 000 cells. The experimental tests have been made by using a simplified model for the end winding space with cooling jets. In the experiments, an emphasis has been given to flow visualisation. The computational analysis shows good agreement with the experimental results. Modelling of the cooling jet arrangement enables also a better understanding of the complex system of heat transfer at end winding space.

Pituusleikkurin turvaväylät ja koneturvallisuus

Turvallisuuteen liittyvän ohjausjärjestelmän tehtävänä on siirtää ja käsitellä turvallisuuskriittistä tietoa. Esimerkiksi anturin (turvalaitteen) havaitessa vaaravyöhykkeelle pyrkivän ihmisen tulee ensimmäisenä tiedon välittyä ohjausjärjestelmään. Ohjausjärjestelmän on muodostettava saapuneen tiedon perusteella käsky tehonohjauselimille. Tehonohjauselimillä säädellään koneen käyttöenergian syöttöä ja sitä kautta on mahdollista pysäyttää koneen liike ennen mahdollisen vahingon sattumista. Perinteiset turvaratkaisut ovat perustuneet pakkotoimintaisiin releisiin ja kahdennuksiin. Tällämenetelmällä on toteutettu varmatoimintaisia turvaratkaisuja, joissa yksittäiset viat ovat paljastuneet. Nykyisin on kuitenkin yhä enemmän tarvetta integroida turvatoiminnot automaatiojärjestelmään ja toteuttaa turvaratkaisut hajautetuillajärjestelmillä. Hajautetut järjestelmät sisältävät osittain ohjelmoitavien järjestelmien etuja ja haittoja, mutta tuovat mukanaan myös uudenlaisia vikoja. Työn tarkoitus oli selvittää, millaisia rajoituksia koneturvallisuus asettaa paperiteollisuuden pituusleikkurien turvallisuuteen liittyville ohjausjärjestelmille, sekä selvittää turvallisuuteen liittyvien järjestelmien rakennetta ja ominaisuuksia. Tässä työssä tutkittiin AS-i, EsaLan ja ProfiSafe turvajärjestelmiä teknisten tietojen perusteella. Tutkitut järjestelmät ovat erilaisia ja soveltuvat tämän vuoksi hieman erilaisiin kohteisiin. Kaikilla näillä järjestelmillä on kuitenkin mahdollista toteuttaa pituusleikkurin turvaväyläjärjestelmässä riittävä turvallisuuden taso koneturvallisuuden näkökulmasta. Tämä edellyttää oikein tehtyä riskianalyysiä ja oikeita suunnittelumenetelmiä. Teknisten tietojen perusteella otettiin testattavaksi AS-i safety at workja EsaLan:in Compact järjestelmät, joille suoritettiin sähkömagneettiseen yhteensopivuuteen (EMC) ja toiminnallisuuteen liittyviä testejä.

Laiteohjauksessamuodostuvan informaation jalostamisen mahdollisuudet

Teollisuuden automaatiojärjestelmät digitalisoituvat, samalla niiden tuottaman reaaliaikaisen datan määrä kasvaa ja etenkin sen saatavuus helpottuu. Samanaikaisesti laitevalmistajan liiketoimintamallit ovat muuttumassa perinteisestä konevalmistuksesta kohti palveluntarjontaa. Muuttuneessa tilanteessa laitteiden ohjauksessa käytettäviltä järjestelmiltä vaaditaan uusia ominaisuuksia. Informaation käsittely ja jalostaminen muodostuvat tärkeiksi kilpailu-tekijöiksi. Kirjallisuusosassa on tarkasteltu, miten data jalostuu informaatioksi ja siitä edelleen tietämykseksi. Työssä myös selvitetään, miten niitä voidaan hyödyntää liiketoiminnassa. Samalla perehdytään teollisuudesta löytyviin informaatio- ja tietämysjärjestelmiin. Kokeellisessa osassa esitellään toimiva tiedonkeruu- ja raportointijärjestelmä ja tutkitaan, miten sitä tulisi kehittää, jotta se sopisi paremmin muuttuviin liiketoimintamalleihin. Lopputuloksena kehitettiin mallijärjestelmä, jolla pystytään täyttämään laitevalmistajan ja loppukäyttäjän muuttuneet informaatiotarpeet osana laiteohjausta.