Advanced mapping of environmental data: Geostatistics, Machine Learning and Bayesian Maximum Entropy

This book combines geostatistics and global mapping systems to present an up-to-the-minute study of environmental data. Featuring numerous case studies, the reference covers model dependent (geostatistics) and data driven (machine learning algorithms) analysis techniques such as risk mapping, conditional stochastic simulations, descriptions of spatial uncertainty and variability, artificial neural networks (ANN) for spatial data, Bayesian maximum entropy (BME), and more.

Evolution of decision-making and learning under fluctuating social environments

The capacity to learn to associate sensory perceptions with appropriate motor actions underlies the success of many animal species, from insects to humans. The evolutionary significance of learning has long been a subject of interest for evolutionary biologists who emphasize the bene¬fit yielded by learning under changing environmental conditions, where it is required to flexibly switch from one behavior to another. However, two unsolved questions are particularly impor¬tant for improving our knowledge of the evolutionary advantages provided by learning, and are addressed in the present work. First, because it is possible to learn the wrong behavior when a task is too complex, the learning rules and their underlying psychological characteristics that generate truly adaptive behavior must be identified with greater precision, and must be linked to the specific ecological problems faced by each species. A framework for predicting behavior from the definition of a learning rule is developed here. Learning rules capture cognitive features such as the tendency to explore, or the ability to infer rewards associated to unchosen actions. It is shown that these features interact in a non-intuitive way to generate adaptive behavior in social interactions where individuals affect each other's fitness. Such behavioral predictions are used in an evolutionary model to demonstrate that, surprisingly, simple trial-and-error learn¬ing is not always outcompeted by more computationally demanding inference-based learning, when population members interact in pairwise social interactions. A second question in the evolution of learning is its link with and relative advantage compared to other simpler forms of phenotypic plasticity. After providing a conceptual clarification on the distinction between genetically determined vs. learned responses to environmental stimuli, a new factor in the evo¬lution of learning is proposed: environmental complexity. A simple mathematical model shows that a measure of environmental complexity, the number of possible stimuli in one's environ¬ment, is critical for the evolution of learning. In conclusion, this work opens roads for modeling interactions between evolving species and their environment in order to predict how natural se¬lection shapes animals' cognitive abilities. - La capacité d'apprendre à associer des sensations perceptives à des actions motrices appropriées est sous-jacente au succès évolutif de nombreuses espèces, depuis les insectes jusqu'aux êtres hu¬mains. L'importance évolutive de l'apprentissage est depuis longtemps un sujet d'intérêt pour les biologistes de l'évolution, et ces derniers mettent l'accent sur le bénéfice de l'apprentissage lorsque les conditions environnementales sont changeantes, car dans ce cas il est nécessaire de passer de manière flexible d'un comportement à l'autre. Cependant, deux questions non résolues sont importantes afin d'améliorer notre savoir quant aux avantages évolutifs procurés par l'apprentissage. Premièrement, puisqu'il est possible d'apprendre un comportement incorrect quand une tâche est trop complexe, les règles d'apprentissage qui permettent d'atteindre un com¬portement réellement adaptatif doivent être identifiées avec une plus grande précision, et doivent être mises en relation avec les problèmes écologiques spécifiques rencontrés par chaque espèce. Un cadre théorique ayant pour but de prédire le comportement à partir de la définition d'une règle d'apprentissage est développé ici. Il est démontré que les caractéristiques cognitives, telles que la tendance à explorer ou la capacité d'inférer les récompenses liées à des actions non ex¬périmentées, interagissent de manière non-intuitive dans les interactions sociales pour produire des comportements adaptatifs. Ces prédictions comportementales sont utilisées dans un modèle évolutif afin de démontrer que, de manière surprenante, l'apprentissage simple par essai-et-erreur n'est pas toujours battu par l'apprentissage basé sur l'inférence qui est pourtant plus exigeant en puissance de calcul, lorsque les membres d'une population interagissent socialement par pair. Une deuxième question quant à l'évolution de l'apprentissage concerne son lien et son avantage relatif vis-à-vis d'autres formes plus simples de plasticité phénotypique. Après avoir clarifié la distinction entre réponses aux stimuli génétiquement déterminées ou apprises, un nouveau fac¬teur favorisant l'évolution de l'apprentissage est proposé : la complexité environnementale. Un modèle mathématique permet de montrer qu'une mesure de la complexité environnementale - le nombre de stimuli rencontrés dans l'environnement - a un rôle fondamental pour l'évolution de l'apprentissage. En conclusion, ce travail ouvre de nombreuses perspectives quant à la mo¬délisation des interactions entre les espèces en évolution et leur environnement, dans le but de comprendre comment la sélection naturelle façonne les capacités cognitives des animaux.

Local-global splitting for spatiotemporal-adaptive multiscale methods

We present a novel spatiotemporal-adaptive Multiscale Finite Volume (MsFV) method, which is based on the natural idea that the global coarse-scale problem has longer characteristic time than the local fine-scale problems. As a consequence, the global problem can be solved with larger time steps than the local problems. In contrast to the pressure-transport splitting usually employed in the standard MsFV approach, we propose to start directly with a local-global splitting that allows to locally retain the original degree of coupling. This is crucial for highly non-linear systems or in the presence of physical instabilities. To obtain an accurate and efficient algorithm, we devise new adaptive criteria for global update that are based on changes of coarse-scale quantities rather than on fine-scale quantities, as it is routinely done before in the adaptive MsFV method. By means of a complexity analysis we show that the adaptive approach gives a noticeable speed-up with respect to the standard MsFV algorithm. In particular, it is efficient in case of large upscaling factors, which is important for multiphysics problems. Based on the observation that local time stepping acts as a smoother, we devise a self-correcting algorithm which incorporates the information from previous times to improve the quality of the multiscale approximation. We present results of multiphase flow simulations both for Darcy-scale and multiphysics (hybrid) problems, in which a local pore-scale description is combined with a global Darcy-like description. The novel spatiotemporal-adaptive multiscale method based on the local-global splitting is not limited to porous media flow problems, but it can be extended to any system described by a set of conservation equations.

Interactions between learning and immunity in Drosophila melanogaster

Learning and immunity are two adaptive traits with roles in central aspects of an organism's life: learning allows adjusting behaviours in changing environments, while immunity protects the body integrity against parasites and pathogens. While we know a lot about how these two traits interact in vertebrates, the interactions between learning and immunity remain poorly explored in insects. During my PhD, I studied three possible ways in which these two traits interact in the model system Drosophila melanogaster, a model organism in the study of learning and in the study of immunity. Learning can affect the behavioural defences against parasites and pathogens through the acquisition of new aversions for contaminated food for instance. This type of learning relies on the ability to associate a food-related cue with the visceral sickness following ingestion of contaminated food. Despite its potential implication in infection prevention, the existence of pathogen avoidance learning has been rarely explored in invertebrates. In a first part of my PhD, I tested whether D. melanogaster, which feed on food enriched in microorganisms, innately avoid the orally-acquired 'novel' virulent pathogen Pseudomonas entomophila, and whether it can learn to avoid it. Although flies did not innately avoid this pathogen, they decreased their preference for contaminated food over time, suggesting the existence of a form of learning based likely on infection-induced sickness. I further found that flies may be able to learn to avoid an odorant which was previously associated with the pathogen, but this requires confirmation with additional data. If this is confirmed, this would be the first time, to my knowledge, that pathogen avoidance learning is reported in an insect. The detrimental effect of infection on cognition and more specifically on learning ability is well documented in vertebrates and in social insects. While the underlying mechanisms are described in detail in vertebrates, experimental investigations are lacking in invertebrates. In a second part of my PhD, I tested the effect of an oral infection with natural pathogens on associative learning of D. melanogaster. By contrast with previous studies in insects, I found that flies orally infected with the virulent P. entomophila learned better the association of an odorant with mechanical shock than uninfected flies. The effect seems to be specific to a gut infection, and so far I have not been able to draw conclusions on the respective contributions of the pathogen's virulence and of the flies' immune activity in this effect. Interestingly, infected flies may display an increased sensitivity to physical pain. If the learning improvement observed in infected flies was due partially to the activity of the immune system, my results would suggest the existence of physiological connections between the immune system and the nervous system. The basis of these connections would then need to be addressed. Learning and immunity are linked at the physiological level in social insects. Physiological links between traits often result from the expression of genetic links between these traits. However, in social insects, there is no evidence that learning and immunity may be involved in an evolutionary trade-off. I previously reported a positive effect of infection on learning in D. melanogaster. This might suggest that a positive genetic link could exist between learning and immunity. We tested this hypothesis with two approaches: the diallel cross design with inbred lines, and the isofemale lines design. The two approaches provided consistent results: we found no additive genetic correlation between learning and resistance to infection with the diallel cross, and no genetic correlation in flies which are not yet adapted to laboratory conditions in isofemale lines. Consistently with the literature, the two studies suggested that the positive effect of infection on learning I observed might not be reflected by a positive evolutionary link between learning and immunity. Nevertheless, the existence of complex genetic relationships between the two traits cannot be excluded. - L'apprentissage et l'immunité sont deux caractères à valeur adaptative impliqués dans des aspects centraux de la vie d'un organisme : l'apprentissage permet d'ajuster les comportements pour faire face aux changements de l'environnement, tandis que l'immunité protège l'intégrité corporelle contre les attaques des parasites et des pathogènes. Alors que les interactions entre l'apprentissage et l'immunité sont bien documentées chez les vertébrés, ces interactions ont été très peu étudiées chez les insectes. Pendant ma thèse, je me suis intéressée à trois aspects des interactions possibles entre l'apprentissage et l'immunité chez la mouche du vinaigre Drosophila melanogaster, qui est un organisme modèle dans l'étude à la fois de l'apprentissage et de l'immunité. L'apprentissage peut affecter les défenses comportementales contre les parasites et les pathogènes par l'acquisition de nouvelles aversions pour la nourriture contaminée par exemple. Ce type d'apprentissage repose sur la capacité à associer une caractéristique de la nourriture avec la maladie qui suit l'ingestion de cette nourriture. Malgré les implications potentielles pour la prévention des infections, l'évitement appris des pathogènes a été rarement étudié chez les invertébrés. Dans une première partie de ma thèse, j'ai testé si les mouches, qui se nourrissent sur des milieux enrichis en micro-organismes, évitent de façon innée un 'nouveau' pathogène virulent Pseudomonas entomophila, et si elles ont la capacité d'apprendre à l'éviter. Bien que les mouches ne montrent pas d'évitement inné pour ce pathogène, elles diminuent leur préférence pour de la nourriture contaminée dans le temps, suggérant l'existence d'une forme d'apprentissage basée vraisemblablement sur la maladie générée par l'infection. J'ai ensuite observé que les mouches semblent être capables d'apprendre à éviter une odeur qui était au préalable associée avec ce pathogène, mais cela reste à confirmer par la collecte de données supplémentaires. Si cette observation est confirmée, cela sera la première fois, à ma connaissance, que l'évitement appris des pathogènes est décrit chez un insecte. L'effet détrimental des infections sur la cognition et plus particulièrement sur les capacités d'apprentissage est bien documenté chez les vertébrés et les insectes sociaux. Alors que les mécanismes sous-jacents sont détaillés chez les vertébrés, des études expérimentales font défaut chez les insectes. Dans une seconde partie de ma thèse, j'ai mesuré les effets d'une infection orale par des pathogènes naturels sur les capacités d'apprentissage associatif de la drosophile. Contrairement aux études précédentes chez les insectes, j'ai trouvé que les mouches infectées par le pathogène virulent P. entomophila apprennent mieux à associer une odeur avec des chocs mécaniques que des mouches non infectées. Cet effet semble spécifique à l'infection orale, et jusqu'à présent je n'ai pas pu conclure sur les contributions respectives de la virulence du pathogène et de l'activité immunitaire des mouches dans cet effet. De façon intéressante, les mouches infectées pourraient montrer une plus grande réactivité à la douleur physique. Si l'amélioration de l'apprentissage observée chez les mouches infectées était due en partie à l'activité du système immunitaire, mes résultats suggéreraient l'existence de connections physiologiques entre le système immunitaire et le système nerveux. Les mécanismes de ces connections seraient à explorer. L'apprentissage et l'immunité sont liés sur un plan physiologique chez les insectes sociaux. Les liens physiologiques entre les caractères résultent souvent de l'expression de liens entre ces caractères au niveau génétique. Cependant, chez les insectes sociaux, il n'y a pas de preuve que l'apprentissage et l'immunité soient liés par un compromis évolutif. J'ai précédemment rapporté un effet positif de l'infection sur l'apprentissage chez la drosophile. Cela pourrait suggérer qu'une relation génétique positive existerait entre l'apprentissage et l'immunité. Nous avons testé cette hypothèse par deux approches : le croisement diallèle avec des lignées consanguines, et les lignées isofemelles. Les deux approches ont fournies des résultats similaires : nous n'avons pas détecté de corrélation génétique additive entre l'apprentissage et la résistance à l'infection avec le croisement diallèle, et pas de corrélation génétique chez des mouches non adaptées aux conditions de laboratoire avec les lignées isofemelles. En ligne avec la littérature, ces deux études suggèrent que l'effet positif de l'infection sur l'apprentissage que j'ai précédemment observé ne refléterait pas un lien évolutif positif entre l'apprentissage et l'immunité. Néanmoins, l'existence de relations génétiques complexes n'est pas exclue.

Activity in inferior parietal and medial prefrontal cortex signals the accumulation of evidence in a probability learning task.

In an uncertain environment, probabilities are key to predicting future events and making adaptive choices. However, little is known about how humans learn such probabilities and where and how they are encoded in the brain, especially when they concern more than two outcomes. During functional magnetic resonance imaging (fMRI), young adults learned the probabilities of uncertain stimuli through repetitive sampling. Stimuli represented payoffs and participants had to predict their occurrence to maximize their earnings. Choices indicated loss and risk aversion but unbiased estimation of probabilities. BOLD response in medial prefrontal cortex and angular gyri increased linearly with the probability of the currently observed stimulus, untainted by its value. Connectivity analyses during rest and task revealed that these regions belonged to the default mode network. The activation of past outcomes in memory is evoked as a possible mechanism to explain the engagement of the default mode network in probability learning. A BOLD response relating to value was detected only at decision time, mainly in striatum. It is concluded that activity in inferior parietal and medial prefrontal cortex reflects the amount of evidence accumulated in favor of competing and uncertain outcomes.

Information systems for water resources management improvement in developing countries

Water resources management, as also water service provision projects in developing countries have difficulties to take adequate decisions due to scarce reliable information, and a lack of proper information managing. Some appropriate tools need to be developed in order to improve decision making to improve water management and access of the poorest, through the design of Decision Support Systems (DSS). On the one side, a DSS for developing co-operation projects on water access improvement has been developed. Such a tool has specific context constrains (structure of the system, software requirements) and needs (Logical Framework Approach monitoring, organizational-learning, accountability and evaluation) that shall be considered for its design. Key aspects for its successful implementation have appeared to be a participatory design of the system and support of the managerial positions at the inception phase. A case study in Tanzania was conducted, together with the Spanish NGO ONGAWA – Ingeniería para el Desarrollo. On the other side, DSS are required also to improve decision making on water management resources in order to achieve a sustainable development that not only improves the living conditions of the population in developing countries, but that also does not hinder opportunities of the poorest on those context. A DSS made to fulfil these requirements shall be using information from water resources modelling, as also on the environment and the social context. Through the research, a case study has been conducted in the Central Rift Valley of Ethiopia, an endhorreic basin 160 km south of Addis Ababa. There, water has been modelled using ArcSWAT, a physically based model which can assess the impact of land management practices on large complex watersheds with varying soils, land use and management conditions over long periods of time. Moreover, governance on water and environment as also the socioeconomic context have been studied.

Machine learning algorithms for spatial data. Case studies: Environmental pollution, natural hazards, renewable resources

This paper presents general problems and approaches for the spatial data analysis using machine learning algorithms. Machine learning is a very powerful approach to adaptive data analysis, modelling and visualisation. The key feature of the machine learning algorithms is that they learn from empirical data and can be used in cases when the modelled environmental phenomena are hidden, nonlinear, noisy and highly variable in space and in time. Most of the machines learning algorithms are universal and adaptive modelling tools developed to solve basic problems of learning from data: classification/pattern recognition, regression/mapping and probability density modelling. In the present report some of the widely used machine learning algorithms, namely artificial neural networks (ANN) of different architectures and Support Vector Machines (SVM), are adapted to the problems of the analysis and modelling of geo-spatial data. Machine learning algorithms have an important advantage over traditional models of spatial statistics when problems are considered in a high dimensional geo-feature spaces, when the dimension of space exceeds 5. Such features are usually generated, for example, from digital elevation models, remote sensing images, etc. An important extension of models concerns considering of real space constrains like geomorphology, networks, and other natural structures. Recent developments in semi-supervised learning can improve modelling of environmental phenomena taking into account on geo-manifolds. An important part of the study deals with the analysis of relevant variables and models' inputs. This problem is approached by using different feature selection/feature extraction nonlinear tools. To demonstrate the application of machine learning algorithms several interesting case studies are considered: digital soil mapping using SVM, automatic mapping of soil and water system pollution using ANN; natural hazards risk analysis (avalanches, landslides), assessments of renewable resources (wind fields) with SVM and ANN models, etc. The dimensionality of spaces considered varies from 2 to more than 30. Figures 1, 2, 3 demonstrate some results of the studies and their outputs. Finally, the results of environmental mapping are discussed and compared with traditional models of geostatistics.

Open Learning Network : the evidence of OER impact

Much of the initial work on Open Educational Resources (OER) has inevitably concentrated on how to produce the resources themselves and to establish the idea in the community. It is now eight years since the term OER was first used and more than ten years since the concept of open content was described and a greater focus is now emerging on the way in which OER can influence policy and change the way in which educational systems help people learn. The Open University UK and Carnegie Mellon University are working in partnership on the OLnet (Open Learning Network), funded by The William and Flora Hewlett Foundation with the aims to search out the evidence for use and reuse of OER and to establish a network for information sharing about research in the field. This means both gathering evidence and developing approaches for how to research and understand ways to learn in a more open world, particularly linked to OER, but also looking at other influences.

A Classification of Tools for Learning Logic

A table showing a comparison and classification of tools (intelligent tutoring systems) for e-learning of Logic at a college level.

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Electronic voting system for computer supported collaborative learning

The EVS4CSCL project starts in the context of a Computer Supported Collaborative Learning environment (CSCL). Previous UOC projects created a CSCL generic platform (CLPL) to facilitate the development of CSCL applications. A discussion forum (DF) was the first application developed over the framework. This discussion forum was different from other products on the marketplace because of its focus on the learning process. The DF carried out the specification and elaboration phases from the discussion learning process but there was a lack in the consensus phase. The consensus phase in a learning environment is not something to be achieved but tested. Common tests are done by Electronic Voting System (EVS) tools, but consensus test is not an assessment test. We are not evaluating our students by their answers but by their discussion activity. Our educational EVS would be used as a discussion catalyst proposing a discussion about the results after an initial query or it would be used after a discussion period in order to manifest how the discussion changed the students mind (consensus). It should be also used by the teacher as a quick way to know where the student needs some reinforcement. That is important in a distance-learning environment where there is no direct contact between the teacher and the student and it is difficult to detect the learning lacks. In an educational environment, assessment it is a must and the EVS will provide direct assessment by peer usefulness evaluation, teacher marks on every query created and indirect assessment from statistics regarding the user activity.

An empirical study of machine learning techniques applied to image watermarking

A study of how the machine learning technique, known as gentleboost, could improve different digital watermarking methods such as LSB, DWT, DCT2 and Histogram shifting.