Machine learning for geospatial data : algorithms, software tools and case studies

Résumé Cette thèse est consacrée à l'analyse, la modélisation et la visualisation de données environnementales à référence spatiale à l'aide d'algorithmes d'apprentissage automatique (Machine Learning). L'apprentissage automatique peut être considéré au sens large comme une sous-catégorie de l'intelligence artificielle qui concerne particulièrement le développement de techniques et d'algorithmes permettant à une machine d'apprendre à partir de données. Dans cette thèse, les algorithmes d'apprentissage automatique sont adaptés pour être appliqués à des données environnementales et à la prédiction spatiale. Pourquoi l'apprentissage automatique ? Parce que la majorité des algorithmes d'apprentissage automatiques sont universels, adaptatifs, non-linéaires, robustes et efficaces pour la modélisation. Ils peuvent résoudre des problèmes de classification, de régression et de modélisation de densité de probabilités dans des espaces à haute dimension, composés de variables informatives spatialisées (« géo-features ») en plus des coordonnées géographiques. De plus, ils sont idéaux pour être implémentés en tant qu'outils d'aide à la décision pour des questions environnementales allant de la reconnaissance de pattern à la modélisation et la prédiction en passant par la cartographie automatique. Leur efficacité est comparable au modèles géostatistiques dans l'espace des coordonnées géographiques, mais ils sont indispensables pour des données à hautes dimensions incluant des géo-features. Les algorithmes d'apprentissage automatique les plus importants et les plus populaires sont présentés théoriquement et implémentés sous forme de logiciels pour les sciences environnementales. Les principaux algorithmes décrits sont le Perceptron multicouches (MultiLayer Perceptron, MLP) - l'algorithme le plus connu dans l'intelligence artificielle, le réseau de neurones de régression généralisée (General Regression Neural Networks, GRNN), le réseau de neurones probabiliste (Probabilistic Neural Networks, PNN), les cartes auto-organisées (SelfOrganized Maps, SOM), les modèles à mixture Gaussiennes (Gaussian Mixture Models, GMM), les réseaux à fonctions de base radiales (Radial Basis Functions Networks, RBF) et les réseaux à mixture de densité (Mixture Density Networks, MDN). Cette gamme d'algorithmes permet de couvrir des tâches variées telle que la classification, la régression ou l'estimation de densité de probabilité. L'analyse exploratoire des données (Exploratory Data Analysis, EDA) est le premier pas de toute analyse de données. Dans cette thèse les concepts d'analyse exploratoire de données spatiales (Exploratory Spatial Data Analysis, ESDA) sont traités selon l'approche traditionnelle de la géostatistique avec la variographie expérimentale et selon les principes de l'apprentissage automatique. La variographie expérimentale, qui étudie les relations entre pairs de points, est un outil de base pour l'analyse géostatistique de corrélations spatiales anisotropiques qui permet de détecter la présence de patterns spatiaux descriptible par une statistique. L'approche de l'apprentissage automatique pour l'ESDA est présentée à travers l'application de la méthode des k plus proches voisins qui est très simple et possède d'excellentes qualités d'interprétation et de visualisation. Une part importante de la thèse traite de sujets d'actualité comme la cartographie automatique de données spatiales. Le réseau de neurones de régression généralisée est proposé pour résoudre cette tâche efficacement. Les performances du GRNN sont démontrées par des données de Comparaison d'Interpolation Spatiale (SIC) de 2004 pour lesquelles le GRNN bat significativement toutes les autres méthodes, particulièrement lors de situations d'urgence. La thèse est composée de quatre chapitres : théorie, applications, outils logiciels et des exemples guidés. Une partie importante du travail consiste en une collection de logiciels : Machine Learning Office. Cette collection de logiciels a été développée durant les 15 dernières années et a été utilisée pour l'enseignement de nombreux cours, dont des workshops internationaux en Chine, France, Italie, Irlande et Suisse ainsi que dans des projets de recherche fondamentaux et appliqués. Les cas d'études considérés couvrent un vaste spectre de problèmes géoenvironnementaux réels à basse et haute dimensionnalité, tels que la pollution de l'air, du sol et de l'eau par des produits radioactifs et des métaux lourds, la classification de types de sols et d'unités hydrogéologiques, la cartographie des incertitudes pour l'aide à la décision et l'estimation de risques naturels (glissements de terrain, avalanches). Des outils complémentaires pour l'analyse exploratoire des données et la visualisation ont également été développés en prenant soin de créer une interface conviviale et facile à l'utilisation. Machine Learning for geospatial data: algorithms, software tools and case studies Abstract The thesis is devoted to the analysis, modeling and visualisation of spatial environmental data using machine learning algorithms. In a broad sense machine learning can be considered as a subfield of artificial intelligence. It mainly concerns with the development of techniques and algorithms that allow computers to learn from data. In this thesis machine learning algorithms are adapted to learn from spatial environmental data and to make spatial predictions. Why machine learning? In few words most of machine learning algorithms are universal, adaptive, nonlinear, robust and efficient modeling tools. They can find solutions for the classification, regression, and probability density modeling problems in high-dimensional geo-feature spaces, composed of geographical space and additional relevant spatially referenced features. They are well-suited to be implemented as predictive engines in decision support systems, for the purposes of environmental data mining including pattern recognition, modeling and predictions as well as automatic data mapping. They have competitive efficiency to the geostatistical models in low dimensional geographical spaces but are indispensable in high-dimensional geo-feature spaces. The most important and popular machine learning algorithms and models interesting for geo- and environmental sciences are presented in details: from theoretical description of the concepts to the software implementation. The main algorithms and models considered are the following: multi-layer perceptron (a workhorse of machine learning), general regression neural networks, probabilistic neural networks, self-organising (Kohonen) maps, Gaussian mixture models, radial basis functions networks, mixture density networks. This set of models covers machine learning tasks such as classification, regression, and density estimation. Exploratory data analysis (EDA) is initial and very important part of data analysis. In this thesis the concepts of exploratory spatial data analysis (ESDA) is considered using both traditional geostatistical approach such as_experimental variography and machine learning. Experimental variography is a basic tool for geostatistical analysis of anisotropic spatial correlations which helps to understand the presence of spatial patterns, at least described by two-point statistics. A machine learning approach for ESDA is presented by applying the k-nearest neighbors (k-NN) method which is simple and has very good interpretation and visualization properties. Important part of the thesis deals with a hot topic of nowadays, namely, an automatic mapping of geospatial data. General regression neural networks (GRNN) is proposed as efficient model to solve this task. Performance of the GRNN model is demonstrated on Spatial Interpolation Comparison (SIC) 2004 data where GRNN model significantly outperformed all other approaches, especially in case of emergency conditions. The thesis consists of four chapters and has the following structure: theory, applications, software tools, and how-to-do-it examples. An important part of the work is a collection of software tools - Machine Learning Office. Machine Learning Office tools were developed during last 15 years and was used both for many teaching courses, including international workshops in China, France, Italy, Ireland, Switzerland and for realizing fundamental and applied research projects. Case studies considered cover wide spectrum of the real-life low and high-dimensional geo- and environmental problems, such as air, soil and water pollution by radionuclides and heavy metals, soil types and hydro-geological units classification, decision-oriented mapping with uncertainties, natural hazards (landslides, avalanches) assessments and susceptibility mapping. Complementary tools useful for the exploratory data analysis and visualisation were developed as well. The software is user friendly and easy to use.

Binding free energy differences in a TCR-peptide-MHC complex induced by a peptide mutation: a simulation analysis.

Recognition by the T-cell receptor (TCR) of immunogenic peptides presented by class I major histocompatibility complexes (MHCs) is the determining event in the specific cellular immune response against virus-infected cells or tumor cells. It is of great interest, therefore, to elucidate the molecular principles upon which the selectivity of a TCR is based. These principles can in turn be used to design therapeutic approaches, such as peptide-based immunotherapies of cancer. In this study, free energy simulation methods are used to analyze the binding free energy difference of a particular TCR (A6) for a wild-type peptide (Tax) and a mutant peptide (Tax P6A), both presented in HLA A2. The computed free energy difference is 2.9 kcal/mol, in good agreement with the experimental value. This makes possible the use of the simulation results for obtaining an understanding of the origin of the free energy difference which was not available from the experimental results. A free energy component analysis makes possible the decomposition of the free energy difference between the binding of the wild-type and mutant peptide into its components. Of particular interest is the fact that better solvation of the mutant peptide when bound to the MHC molecule is an important contribution to the greater affinity of the TCR for the latter. The results make possible identification of the residues of the TCR which are important for the selectivity. This provides an understanding of the molecular principles that govern the recognition. The possibility of using free energy simulations in designing peptide derivatives for cancer immunotherapy is briefly discussed.

Environmental data mining and modeling based on machine learning algorithms and geostatistics

The paper presents some contemporary approaches to spatial environmental data analysis. The main topics are concentrated on the decision-oriented problems of environmental spatial data mining and modeling: valorization and representativity of data with the help of exploratory data analysis, spatial predictions, probabilistic and risk mapping, development and application of conditional stochastic simulation models. The innovative part of the paper presents integrated/hybrid model-machine learning (ML) residuals sequential simulations-MLRSS. The models are based on multilayer perceptron and support vector regression ML algorithms used for modeling long-range spatial trends and sequential simulations of the residuals. NIL algorithms deliver non-linear solution for the spatial non-stationary problems, which are difficult for geostatistical approach. Geostatistical tools (variography) are used to characterize performance of ML algorithms, by analyzing quality and quantity of the spatially structured information extracted from data with ML algorithms. Sequential simulations provide efficient assessment of uncertainty and spatial variability. Case study from the Chernobyl fallouts illustrates the performance of the proposed model. It is shown that probability mapping, provided by the combination of ML data driven and geostatistical model based approaches, can be efficiently used in decision-making process. (C) 2003 Elsevier Ltd. All rights reserved.

Practical algorithms for a family of waterfilling solutions

Many engineering problems that can be formulatedas constrained optimization problems result in solutionsgiven by a waterfilling structure; the classical example is thecapacity-achieving solution for a frequency-selective channel.For simple waterfilling solutions with a single waterlevel and asingle constraint (typically, a power constraint), some algorithmshave been proposed in the literature to compute the solutionsnumerically. However, some other optimization problems result insignificantly more complicated waterfilling solutions that includemultiple waterlevels and multiple constraints. For such cases, itmay still be possible to obtain practical algorithms to evaluate thesolutions numerically but only after a painstaking inspection ofthe specific waterfilling structure. In addition, a unified view ofthe different types of waterfilling solutions and the correspondingpractical algorithms is missing.The purpose of this paper is twofold. On the one hand, itoverviews the waterfilling results existing in the literature from aunified viewpoint. On the other hand, it bridges the gap betweena wide family of waterfilling solutions and their efficient implementationin practice; to be more precise, it provides a practicalalgorithm to evaluate numerically a general waterfilling solution,which includes the currently existing waterfilling solutions andothers that may possibly appear in future problems.

Distributed Energy Production in the North-West Region of Russia

The energy system of Russia is the world's fourth largest measured by installed power. The largest are that of the the United States of America, China and Japan. After 1990, the electricity consumption decreased as a result of the Russian industry crisis. The vivid economic growth during the latest few years explains the new increase in the demand for energy resources within the State. In 2005 the consumption of electricity achieved the maximum level of 1990 and continues to growth. In the 1980's, the renewal of power facilities was already very slow and practically stopped in the 1990's. At present, the energy system can be very much characterized as outdated, inefficient and uneconomic because of the old equipment, non-effective structure and large losses in the transmission lines. The aim of Russia's energy reform, which was started in 2001, is to achieve a market based energy policy by 2011. This would thus remove the significantly state-controlled monopoly in Russia's energy policy. The reform will stimulateto decrease losses, improve the energy system and employ energy-saving technologies. The Russian energy system today is still based on the use of fossil fuels, and it almost totally ignores the efficient use of renewable sources such as wind, solar, small hydro and biomass, despite of their significant resources in Russia. The main target of this project is to consider opportunities to apply renewable energy production in the North-West Federal Region of Russia to partly solve the above mentioned problems in the energy system.

The reform of the European Energy Tax Directive reform : effect on prices

[spa] Para hacer frente a los riesgos relacionados con la contaminación atmosférica, es ampliamente aceptada la necesidad de instrumentos de política encaminados a reducir las emisiones. La intervención tiene por objeto reducir las conductas contaminantes y incentivar una conducta más respetuosa y el uso de tecnologías más eficientes. La Unión Europea cuenta con dos importantes mecanismos económicos para el control de emisiones a escala europea: la directiva sobre los impuestos energéticos, un instrumento de fiscalidad ambiental aprobado en 2003 que afecta el precio de los productos energéticos, y el sistema de comercio de los derechos de emisiones, introducido en 2005, que afecta directamente a la cantidad de emisiones de CO2. En 2011, la Comisión Europea propuso una nueva versión de la directiva sobre los impuestos energéticos. El objetivo principal de la propuesta es aumentar la eficacia del instrumento a través de una mayor presión fiscal sobre los productos energéticos y de coordinar este instrumento de fiscalidad medioambiental con el sistema de comercio de los derechos de emisiones, para establecer una señal de precio de CO2 coherente para todos los sectores. Sin embargo, en mayo de 2012 el Parlamento Europeo bloqueó la propuesta de la nueva versión del impuesto, y el proceso de actualización se detuvo. La preocupación principal parecía ser el efecto de dicha propuesta en la competitividad, en particular para los sectores que serían los más afectados dado el uso intensivo de los productos energéticos, como el sector del transporte. El objetivo de este estudio es analizar el efecto que la reforma de la directiva sobre los impuestos energéticos podría tener sobre el nivel de precios, en particular en los países de la Unión Europea donde esta reforma implicaría un aumento de los impuestos energéticos. Utilizando datos del proyecto “World Input-Output Database”, la principal conclusión es que el nuevo sistema de impuestos energéticos tendría un impacto muy bajo sobre los precios. Por lo tanto, dado que los precios no serían fuertemente afectados por la reforma, no habrá inconvenientes para la competitividad y implicaciones en términos de distribución, pero, por otro lado, este resultado también implica una baja capacidad de esta reforma para provocar cambios en el consumo y la producción hacia menos presiones ambientales.

The reform of the European Energy Tax Directive reform : effect on prices

[spa] Para hacer frente a los riesgos relacionados con la contaminación atmosférica, es ampliamente aceptada la necesidad de instrumentos de política encaminados a reducir las emisiones. La intervención tiene por objeto reducir las conductas contaminantes y incentivar una conducta más respetuosa y el uso de tecnologías más eficientes. La Unión Europea cuenta con dos importantes mecanismos económicos para el control de emisiones a escala europea: la directiva sobre los impuestos energéticos, un instrumento de fiscalidad ambiental aprobado en 2003 que afecta el precio de los productos energéticos, y el sistema de comercio de los derechos de emisiones, introducido en 2005, que afecta directamente a la cantidad de emisiones de CO2. En 2011, la Comisión Europea propuso una nueva versión de la directiva sobre los impuestos energéticos. El objetivo principal de la propuesta es aumentar la eficacia del instrumento a través de una mayor presión fiscal sobre los productos energéticos y de coordinar este instrumento de fiscalidad medioambiental con el sistema de comercio de los derechos de emisiones, para establecer una señal de precio de CO2 coherente para todos los sectores. Sin embargo, en mayo de 2012 el Parlamento Europeo bloqueó la propuesta de la nueva versión del impuesto, y el proceso de actualización se detuvo. La preocupación principal parecía ser el efecto de dicha propuesta en la competitividad, en particular para los sectores que serían los más afectados dado el uso intensivo de los productos energéticos, como el sector del transporte. El objetivo de este estudio es analizar el efecto que la reforma de la directiva sobre los impuestos energéticos podría tener sobre el nivel de precios, en particular en los países de la Unión Europea donde esta reforma implicaría un aumento de los impuestos energéticos. Utilizando datos del proyecto “World Input-Output Database”, la principal conclusión es que el nuevo sistema de impuestos energéticos tendría un impacto muy bajo sobre los precios. Por lo tanto, dado que los precios no serían fuertemente afectados por la reforma, no habrá inconvenientes para la competitividad y implicaciones en términos de distribución, pero, por otro lado, este resultado también implica una baja capacidad de esta reforma para provocar cambios en el consumo y la producción hacia menos presiones ambientales.

Is altitude training an efficient treatment for obesity?

Worldwide, about half the adult population is considered overweight as defined by a body mass index (BMI - calculated by body weight divided by height squared) ratio in excess of 25 kg.m-2. Of these individuals, half are clinically obese (with a BMI in excess of 30) and these numbers are still increasing, notably in developing countries such as those of the Middle East region. Obesity is a disorder characterised by increased mass of adipose tissue (excessive fat accumulation) that is the result of a systemic imbalance between food intake and energy expenditure. Although factors such as family history, sedentary lifestyle, urbanisation, income and family diet patterns determine obesity prevalence, the main underlying causes are poor knowledge about food choice and lack of physical activity3. Current obesity treatments include dietary restriction, pharmacological interventions and ultimately, bariatric surgery. The beneficial effects of physical activity on weight loss through increased energy expenditure and appetite modulation are also firmly established. Another viable option to induce a negative energy balance, is to incorporate hypoxia per se or combine it with exercise in an individual's daily schedule. This article will present recent evidence suggesting that combining hypoxic exposure and exercise training might provide a cost-effective strategy for reducing body weight and improving cardio-metabolic health in obese individuals. The efficacy of this approach is further reinforced by epidemiological studies using large-scale databases, which evidence a negative relationship between altitude of habitation and obesity. In the United States, for instance, obesity prevalence is inversely associated with altitude of residence and urbanisation, after adjusting for temperature, diet, physical activity, smoking and demographic factors.

Efficient total variation algorithm for fetal MRI reconstruction

Fetal MRI reconstruction aims at finding a high-resolution image given a small set of low-resolution images. It is usually modeled as an inverse problem where the regularization term plays a central role in the reconstruction quality. Literature has considered several regularization terms s.a. Dirichlet/Laplacian energy [1], Total Variation (TV)based energies [2,3] and more recently non-local means [4]. Although TV energies are quite attractive because of their ability in edge preservation, standard explicit steepest gradient techniques have been applied to optimize fetal-based TV energies. The main contribution of this work lies in the introduction of a well-posed TV algorithm from the point of view of convex optimization. Specifically, our proposed TV optimization algorithm for fetal reconstruction is optimal w.r.t. the asymptotic and iterative convergence speeds O(1/n(2)) and O(1/root epsilon), while existing techniques are in O(1/n) and O(1/epsilon). We apply our algorithm to (1) clinical newborn data, considered as ground truth, and (2) clinical fetal acquisitions. Our algorithm compares favorably with the literature in terms of speed and accuracy.

Energy efficiency determinants: An empirical analysis of Spanish innovative firms

This paper examines the extent to which innovative Spanish firms pursue improvements in energy efficiency (EE) as an objective of innovation. The increase in energy consumption and its impact on greenhouse gas emissions justifies the greater attention being paid to energy efficiency and especially to industrial EE. The ability of manufacturing companies to innovate and improve their EE has a substantial influence on attaining objectives regarding climate change mitigation. Despite the effort to design more efficient energy policies, the EE determinants in manufacturing firms have been little studied in the empirical literature. From an exhaustive sample of Spanish manufacturing firms and using a logit model, we examine the energy efficiency determinants for those firms that have innovated. To carry out the econometric analysis, we use panel data from the Community Innovation Survey for the period 2008‐2011. Our empirical results underline the role of size among the characteristics of firms that facilitate energy efficiency innovation. Regarding company behaviour, firms that consider the reduction of environmental impacts to be an important objective of innovation and that have introduced organisational innovations are more likely to innovate with the objective of increasing energy efficiency. Keywords: energy efficiency, corporate targets, innovation, Community Innovation Survey. JEL Classification: Q40, Q55, O31

Cryptographic Energy Costs are Assumable in Ad Hoc Networks

Performance of symmetric and asymmetriccryptography algorithms in small devices is presented. Both temporaland energy costs are measured and compared with the basicfunctional costs of a device. We demonstrate that cryptographicpower costs are not a limiting factor of the autonomy of a deviceand explain how processing delays can be conveniently managedto minimize their impact.

TUNGSTOPHOSPHORIC ACID HETEROGENIZED ONTO NH4ZSM5 AS AN EFFICIENT AND RECYCLABLE CATALYST FOR THE PHOTOCATALYTIC DEGRADATION OF DYES

Materials based on tungstophosphoric acid (TPA) immobilized on NH4ZSM5 zeolite were prepared by wet impregnation of the zeolite matrix with TPA aqueous solutions. Their concentration was varied in order to obtain TPA contents of 5%, 10%, 20%, and 30% w/w in the solid. The materials were characterized by N2 adsorption-desorption isotherms, XRD, FT-IR, 31P MAS-NMR, TGA-DSC, DRS-UV-Vis, and the acidic behavior was studied by potentiometric titration with n-butylamine. The BET surface area (SBET) decreased when the TPA content was raised as a result of zeolite pore blocking. The X-ray diffraction patterns of the solids modified with TPA only presented the characteristic peaks of NH4ZSM5 zeolites, and an additional set of peaks assigned to the presence of (NH4)3PW12O40. According to the Fourier transform infrared and 31P magic angle spinning-nuclear magnetic resonance spectra, the main species present in the samples was the [PW12O40]3- anion, which was partially transformed into the [P2W21O71]6- anion during the synthesis and drying steps. The thermal stability of the NH4ZSM5TPA materials was similar to that of their parent zeolites. Moreover, the samples with the highest TPA content exhibited band gap energy values similar to those reported for TiO2. The immobilization of TPA on NH4ZSM5 zeolite allowed the obtention of catalysts with high photocatalytic activity in the degradation of methyl orange dye (MO) in water, at 25 ºC. These can be reused at least three times without any significant decrease in degree of degradation.

Structural and operational optimisation : applications in energy systems

In this work mathematical programming models for structural and operational optimisation of energy systems are developed and applied to a selection of energy technology problems. The studied cases are taken from industrial processes and from large regional energy distribution systems. The models are based on Mixed Integer Linear Programming (MILP), Mixed Integer Non-Linear Programming (MINLP) and on a hybrid approach of a combination of Non-Linear Programming (NLP) and Genetic Algorithms (GA). The optimisation of the structure and operation of energy systems in urban regions is treated in the work. Firstly, distributed energy systems (DES) with different energy conversion units and annual variations of consumer heating and electricity demands are considered. Secondly, district cooling systems (DCS) with cooling demands for a large number of consumers are studied, with respect to a long term planning perspective regarding to given predictions of the consumer cooling demand development in a region. The work comprises also the development of applications for heat recovery systems (HRS), where paper machine dryer section HRS is taken as an illustrative example. The heat sources in these systems are moist air streams. Models are developed for different types of equipment price functions. The approach is based on partitioning of the overall temperature range of the system into a number of temperature intervals in order to take into account the strong nonlinearities due to condensation in the heat recovery exchangers. The influence of parameter variations on the solutions of heat recovery systems is analysed firstly by varying cost factors and secondly by varying process parameters. Point-optimal solutions by a fixed parameter approach are compared to robust solutions with given parameter variation ranges. In the work enhanced utilisation of excess heat in heat recovery systems with impingement drying, electricity generation with low grade excess heat and the use of absorption heat transformers to elevate a stream temperature above the excess heat temperature are also studied.

Energy Efficiency Consideration in Electric Vehicle Transmission

This study is a survey of benefits and drawbacks of embedding a variable gearbox instead of a single reduction gear in electric vehicle powertrain from efficiency point of view. Losses due to a pair of spur gears meshing with involute teeth are modeled on the base of Coulomb’s law and fluid mechanics. The model for a variable gearbox is fulfilled and further employed in a complete vehicle simulation. Simulation model run for a single reduction gear then the results are taken as benchmark for other types of commonly used transmissions. Comparing power consumption, which is obtained from simulation model, shows that the extra load imposed by variable transmission components will shade the benefits of efficient operation of electric motor. The other accomplishment of this study is a combination of modified formulas that led to a new methodology for power loss prediction in gear meshing which is compatible with modern design and manufacturing technology.