Absence of snapshot memory of the target view interferes with place navigation learning by rats in the water maze

Contribution of visual and nonvisual mechanisms to spatial behavior of rats in the Morris water maze was studied with a computerized infrared tracking system, which switched off the room lights when the subject entered the inner circular area of the pool with an escape platform. Naive rats trained under light-dark conditions (L-D) found the escape platform more slowly than rats trained in permanent light (L). After group members were swapped, the L-pretrained rats found under L-D conditions the same target faster and eventually approached latencies attained during L navigation. Performance of L-D-trained rats deteriorated in permanent darkness (D) but improved with continued D training. Thus L-D navigation improves gradually by procedural learning (extrapolation of the start-target azimuth into the zero-visibility zone) but remains impaired by lack of immediate visual feedback rather than by absence of the snapshot memory of the target view.

Spatial relational learning and memory abilities do not differ between men and women in a real-world, open-field environment

This study assesses gender differences in spatial and non-spatial relational learning and memory in adult humans behaving freely in a real-world, open-field environment. In Experiment 1, we tested the use of proximal landmarks as conditional cues allowing subjects to predict the location of rewards hidden in one of two sets of three distinct locations. Subjects were tested in two different conditions: (1) when local visual cues marked the potentially-rewarded locations, and (2) when no local visual cues marked the potentially-rewarded locations. We found that only 17 of 20 adults (8 males, 9 females) used the proximal landmarks to predict the locations of the rewards. Although females exhibited higher exploratory behavior at the beginning of testing, males and females discriminated the potentially-rewarded locations similarly when local visual cues were present. Interestingly, when the spatial and local information conflicted in predicting the reward locations, males considered both spatial and local information, whereas females ignored the spatial information. However, in the absence of local visual cues females discriminated the potentially-rewarded locations as well as males. In Experiment 2, subjects (9 males, 9 females) were tested with three asymmetrically-arranged rewarded locations, which were marked by local cues on alternate trials. Again, females discriminated the rewarded locations as well as males in the presence or absence of local cues. In sum, although particular aspects of task performance might differ between genders, we found no evidence that women have poorer allocentric spatial relational learning and memory abilities than men in a real-world, open-field environment.

Natural variation in learning ability in "Drosophila melanogaster"

Learning is the ability of an organism to adapt to the changes of its environment in response to its past experience. It is a widespread ability in the animal kingdom, but its evolutionary aspects are poorly known. Learning ability is supposedly advantageous under some conditions, when environmental conditions are not too stable - because in this case there is no need to learn to predict any event in the environment - and not changing too fast - otherwise environmental cues cannot be used because they are not reliable. Nevertheless, learning ability is also known to be costly in terms of energy needed for neuronal synthesis, memory formation, initial mistakes. During my PhD, I focused on the study of genetic variability of learning ability in natural populations. Genetic variability is the basis on which natural selection and genetic drift can act. How does learning ability vary in nature? What are the roles of additive genetic variation or maternal effects in this variation? Is it involved in evolutionary trade-offs with other fitness-related traits?¦I investigated a natural population of fruit fly, Drosophila melanogaster, as a model organism. Its learning ability is easy to measure with associative memory tests. I used two research tools: multiple inbred and isofemale lines derived from a natural population as a representative sample. My work was divided into three parts.¦First, I investigated the effects of inbreeding on aversive learning (avoidance of an odor previously associated with mechanical shock). While the inbred lines consistently showed reduced egg-to-adult viability by 28 %, the effects of inbreeding on learning performance was 18 % and varied among assays, with a trend to be most pronounced for intermediate conditioning intensity. Variation among inbred lines indicates that ample genetic variance for learning was segregating in the base population, and suggests that the inbreeding depression observed in learning performance was mostly due to dominance rather than overdominance. Across the inbred lines, learning performance was positively correlated with the egg-to-adult viability. This positive genetic correlation contradicts previous studies which observed a trade-off between learning ability and lifespan or larval competitive ability. It suggests that much of the genetic variation for learning is due to pleiotropic effects of genes affecting other functions related to survival. Together with the overall mild effects of inbreeding on learning performance, this suggests that genetic variation specifically affecting learning is either very low, or is due to alleles with mostly additive (semi-dominant) effects. It also suggests that alleles reducing learning performance are on average partially recessive, because their effect does not appear in the outbred base population. Moreover, overdominance seems unlikely as major cause of the inbreeding depression, because even if the overall mean of the inbred line is smaller than the outbred base population, some of the inbred lines show the same learning score as the outbred base population. If overdominance played an important part in inbreeding depression, then all the homozygous lines should show lower learning ability than¦outbred base population.¦In the second part of my project, I sampled the same natural population again and derived isofemale lines (F=0.25) which are less adapted to laboratory conditions and therefore are more representative of the variance of the natural population. They also showed some genetic variability for learning, and for three other fitness-related traits possibly related with learning: resistance to bacterial infection, egg-to-adult viability and developmental time. Nevertheless, the genetic variance of learning ability did not appear to be smaller than the variance of the other traits. The positive correlation previously observed between learning ability and egg- to-adult viability did not appear in isofemale lines (nor a negative correlation). It suggests that there was still genetic variability within isofemale lines and that they did not fix the highly deleterious pleiotropic alleles possibly responsible for the previous correlation.¦In order to investigate the relative amount of nuclear (additive and non-additive effects) and extra-nuclear (maternal and paternal effect) components of variance in learning ability and other fitness-related traits among the inbred lines tested in part one, I performed a diallel cross between them. The nuclear additive genetic variance was higher than other components for learning ability and survival to learning ability, but in contrast, maternal effects were more variable than other effects for developmental traits. This suggests that maternal effects, which reflects effects from mitochondrial DNA, epigenetic effects, or the amount of nutrients that are invested by the mother in the egg, are more important in the early stage of life, and less at the adult stage. There was no additive genetic correlation between learning ability and other traits, indicating that the correlation between learning ability and egg-to-adult viability observed in the first pat of my project was mostly due to recessive genes.¦Finally, my results showed that learning ability is genetically variable. The diallel experiment showed additive genetic variance was the most important component of the total variance. Moreover, every inbred or isofemale line showed some learning ability. This suggested that alleles impairing learning ability are eliminated by selection, and therefore that learning ability is under strong selection in natural populations of Drosophila. My results cannot alone explain the maintenance of the observed genetic variation. Even if I cannot eliminate the hypothesis of pleiotropy between learning ability and the other fitness-related traits I measured, there is no evidence for any trade-off between these traits and learning ability. This contradicts what has been observed between learning ability and other traits like lifespan and larval competitivity.¦L'apprentissage représente la capacité d'un organisme à s'adapter aux changement de son environnement au cours de sa vie, en réponse à son expérience passée. C'est une capacité très répandue dans le règne animal, y compris pour les animaux les plus petits et les plus simples, mais les aspects évolutifs de l'apprentissage sont encore mal connus. L'apprentissage est supposé avantageux dans certaines conditions, quand l'environnement n'est ni trop stable - dans ce cas, il n'y a rien à apprendre - ni trop variable - dans ce cas, les indices sur lesquels se reposer changent trop vite pour apprendre. D'un autre côté, l'apprentissage a aussi des coûts, en terme de synthèse neuronale, pour la formation de la mémoire, ou de coûts d'erreur initiale d'apprentissage. Pendant ma thèse, j'ai étudié la variabilité génétique naturelle des capacités d'apprentissage. Comment varient les capacités d'apprentissage dans la nature ? Quelle est la part de variation additive, l'impact des effets maternel ? Est-ce que l'apprentissage est impliqué dans des interactions, de type compromis évolutifs, avec d'autres traits liés à la fitness ?¦Afin de répondre à ces questions, je me suis intéressée à la mouche du vinaigre, ou drosophile, un organisme modèle. Ses capacités d'apprentissage sont facile à étudier avec un test de mémoire reposant sur l'association entre un choc mécanique et une odeur. Pour étudier ses capacités naturelles, j'ai dérivé de types de lignées d'une population naturelle: des lignées consanguines et des lignées isofemelles.¦Dans une première partie, je me suis intéressée aux effets de la consanguinité sur les capacités d'apprentissage, qui sont peu connues. Alors que les lignées consanguines ont montré une réduction de 28% de leur viabilité (proportion d'adultes émergeants d'un nombre d'oeufs donnés), leurs capacités d'apprentissage n'ont été réduites que de 18%, la plus forte diminution étant obtenue pour un conditionnement modéré. En outre, j'ai également observé que les capacités d'apprentissage était positivement corrélée à la viabilité entre les lignées. Cette corrélation est surprenante car elle est en contradiction avec les résultats obtenus par d'autres études, qui montrent l'existence de compromis évolutifs entre les capacités d'apprentissage et d'autres traits comme le vieillissement ou la compétitivité larvaire. Elle suggère que la variation génétique des capacités d'apprentissage est due aux effets pleiotropes de gènes récessifs affectant d'autres fonctions liées à la survie. Ces résultats indiquent que la variation pour les capacités d'apprentissage est réduite comparée à celle d'autres traits ou est due à des allèles principalement récessifs. L'hypothèse de superdominance semble peu vraisemblable, car certaines des lignées consanguines ont obtenu des scores d'apprentissage égaux à ceux de la population non consanguine, alors qu'en cas de superdominance, elles auraient toutes dû obtenir des scores inférieurs.¦Dans la deuxième partie de mon projet, j'ai mesuré les capacités d'apprentissage de lignées isofemelles issues de la même population initiale que les lignées consanguines. Ces lignées sont issues chacune d'un seul couple, ce qui leur donne un taux d'hétérozygosité supérieur et évite l'élimination de lignées par fixation d'allèles délétères rares. Elles sont ainsi plus représentatives de la variabilité naturelle. Leur variabilité génétique est significative pour les capacités d'apprentissage, et trois traits liés à la fois à la fitness et à l'apprentissage: la viabilité, la résistance à l'infection bactérienne et la vitesse de développement. Cependant, la variabilité des capacités d'apprentissage n'apparaît cette fois pas inférieure à celle des autres traits et aucune corrélation n'est constatée entre les capacité d'apprentissage et les autres traits. Ceci suggère que la corrélation observée auparavant était surtout due à la fixation d'allèles récessifs délétères également responsables de la dépression de consanguinité.¦Durant la troisième partie de mon projet, je me suis penchée sur la décomposition de la variance observée entre les lignées consanguines observée en partie 1. Quatre composants ont été examinés: la variance due à des effets nucléaires (additifs et non additifs), et due à des effets parentaux (maternels et paternels). J'ai réalisé un croisement diallèle de toutes les lignées. La variance additive nucléaire s'est révélée supérieure aux autres composants pour les capacités d'apprentissage et la résistance à l'infection bactérienne. Par contre, les effets maternels étaient plus importants que les autres composants pour les traits développementaux (viabilité et vitesse de développement). Ceci suggère que les effets maternels, dus à G ADN mitochondrial, à l'épistasie ou à la quantité de nutriments investis dans l'oeuf par la mère, sont plus importants dans les premiers stades de développement et que leur effet s'estompe à l'âge adulte. Il n'y a en revanche pas de corrélation statistiquement significative entre les effets additifs des capacités d'apprentissage et des autres traits, ce qui indique encore une fois que la corrélation observée entre les capacités d'apprentissage et la viabilité dans la première partie du projet était due à des effets d'allèles partiellement récessifs.¦Au, final, mes résultats montrent bien l'existence d'une variabilité génétique pour les capacités d'apprentissage, et l'expérience du diallèle montre que la variance additive de cette capacité est importante, ce qui permet une réponse à la sélection naturelle. Toutes les lignées, consanguines ou isofemelles, ont obtenu des scores d'apprentissage supérieurs à zéro. Ceci suggère que les allèles supprimant les capacités d'apprentissage sont fortement contre-sélectionnés dans la nature Néanmoins, mes résultats ne peuvent pas expliquer le maintien de cette variabilité génétique par eux-même. Même si l'hypothèse de pléiotropie entre les capacités d'apprentissage et l'un des traits liés à la fitness que j'ai mesuré ne peut être éliminée, il n'y a aucune preuve d'un compromis évolutif pouvant contribuer au maintien de la variabilité.

The neuronal monocarboxylate transporter MCT2: its regulation by BDNF and its role in learning and memory

Glucose has been considered the major, if not the exclusive, energy substrate for the brain. But under certain physiological and pathological conditions other substrates, namely monocarboxylates (lactate, pyruvate and ketone bodies), can contribute significantly to satisfy brain energy demands. These monocarboxylates need to be transported across the blood-brain barrier or out of astrocytes into the extracellular space and taken up into neurons. It has been shown that monocarboxylates are transported by a family of proton-linked transporters called monocarboxylate transporters (MCTs). In the central nervous system, MCT2 is the predominant neuronal isoform and little is known about the regulation of its expression. Noradrenaline (NA), insulin and IGF-1 were previously shown to enhance the expression of MCT2 in cultured cortical neurons via a translational mechanism. Here we demonstrate that the well known brain neurotrophic factor BDNF enhances MCT2 protein expression in cultured cortical neurons and in synaptoneurosome preparations in a time- and concentrationdependent manner without affecting MCT2 mRNA levels. We observed that BDNF induced MCT2 expression by activation of MAPK as well as PI3K/Akt/mTOR signaling pathways. Furthermore, we investigated the possible post-transcriptional regulation of MCT2 expression by a neuronal miRNA. Then, we demonstrated that BDNF enhanced MCT2 expression in the hippocampus in vivo, in parallel with some post-synaptic proteins such as PSD95 and AMPA receptor GluR2/3 subunits, and two immediate early genes Arc and Zif268 known to be expressed in conditions related to synaptic plasticity. In the last part, we demonstrated in vivo that a downregulation of hippocampal MCT2 via silencing with an appropriate lentiviral vector in mice caused an impairment of working memory without reference memory deficit. In conclusion, these results suggest that regulation of neuronal monocarboxylate transporter MCT2 expression could be a key event in the context of synaptic plasticity, allowing an adequate energy substrate supply in situations of altered synaptic efficacy. - Le glucose représente le substrat énergétique majeur pour le cerveau. Cependant, dans certaines conditions physiologiques ou pathologiques, le cerveau a la capacité d'utiliser des substrats énergéiques appartenant à la classe des monocarboxylates (lactate, pyruvate et corps cétoniques) afin de satisfaire ses besoins énergétiques. Ces monocarboxylates doivent être transportés à travers la barrière hématoencéphalique mais aussi hors des astrocytes vers l'espace extracellulaire puis re-captés par les neurones. Leur transport est assuré par une famillle de transporteurs aux monocarboxylates (MCTs). Dans le système nerveux central, les neurones expriment principalement l'isoforme MCT2 mais peu d'informations sont disponibles concernant la régulation de son expression. Il a été montré que la noradrénaline, l'insuline et l'IGF-1 induisent l'expression de MCT2 dans des cultures de neurones corticaux par un mécanisme traductionnel. Dans cette étude nous démontrons dans un premier temps que le facteur neurotrophique BDNF augmente l'expression de MCT2 à la fois dans des cultures de neurones corticaux et dans les préparations synaptoneurosomales selon un décours temporel et une gamme de concentrations propre. Aucun changement n'a été observé concernant les niveaux d'ARNm de MCT2. Nous avons observé que le BDNF induisait l'expression de MCT2 par l'activation simultanée des voies de signalisation MAPK et PI3K/Akt/mTOR. De plus, nous nous sommes intéressés à une potentielle régulation par les micro-ARNs de la synthèse de MCT2. Ensuite, nous avons démontré que le BDNF induit aussi l'expression de MCT2 dans l'hippocampe de la souris en parallèle avec d'autres protéines post-synaptiques telles que PSD95 et GluR2/3 et avec deux « immediate early genes » tels que Arc et Zif268 connus pour être exprimés dans des conditions de plasticité synaptique. Dans un dernier temps, nous avons démontré qu'une diminution d'expression de MCT2 induite par le biais d'un siRNA exprimé via un vecteur lentiviral dans l'hippocampe de souris générait des déficits de mémoire de travail sans affecter la mémoire de référence. En conclusion, ces résultats nous suggèrent que le transporteur aux monocarboxylates neuronal MCT2 serait essentiel pour l'apport énergétique du lactate pour les neurones dans des conditions de haute activité neuronale comme c'est le cas pendant les processus de plasticité synaptique.

Spatial learning by rats across visually disconnected environments

Two spatial tasks were designed to test specific properties of spatial representation in rats. In the first task, rats were trained to locate an escape hole at a fixed position in a visually homogeneous arena. This arena was connected with a periphery where a full view of the room environment existed. Therefore, rats were dependent on their memory trace of the previous position in the periphery to discriminate a position within the central region. Under these experimental conditions, the test animals showed a significant discrimination of the training position without a specific local view. In the second task, rats were trained in a radial maze consisting of tunnels that were transparent at their distal ends only. Because the central part of the maze was non-transparent, rats had to plan and execute appropriate trajectories without specific visual feedback from the environment. This situation was intended to encourage the reliance on prospective memory of the non-visited arms in selecting the following move. Our results show that acquisition performance was only slightly decreased compared to that shown in a completely transparent maze and considerably higher than in a translucent maze or in darkness. These two series of experiments indicate (1) that rats can learn about the relative position of different places with no common visual panorama, and (2) that they are able to plan and execute a sequence of visits to several places without direct visual feed-back about their relative position.

Machine learning for geospatial data : algorithms, software tools and case studies

Résumé Cette thèse est consacrée à l'analyse, la modélisation et la visualisation de données environnementales à référence spatiale à l'aide d'algorithmes d'apprentissage automatique (Machine Learning). L'apprentissage automatique peut être considéré au sens large comme une sous-catégorie de l'intelligence artificielle qui concerne particulièrement le développement de techniques et d'algorithmes permettant à une machine d'apprendre à partir de données. Dans cette thèse, les algorithmes d'apprentissage automatique sont adaptés pour être appliqués à des données environnementales et à la prédiction spatiale. Pourquoi l'apprentissage automatique ? Parce que la majorité des algorithmes d'apprentissage automatiques sont universels, adaptatifs, non-linéaires, robustes et efficaces pour la modélisation. Ils peuvent résoudre des problèmes de classification, de régression et de modélisation de densité de probabilités dans des espaces à haute dimension, composés de variables informatives spatialisées (« géo-features ») en plus des coordonnées géographiques. De plus, ils sont idéaux pour être implémentés en tant qu'outils d'aide à la décision pour des questions environnementales allant de la reconnaissance de pattern à la modélisation et la prédiction en passant par la cartographie automatique. Leur efficacité est comparable au modèles géostatistiques dans l'espace des coordonnées géographiques, mais ils sont indispensables pour des données à hautes dimensions incluant des géo-features. Les algorithmes d'apprentissage automatique les plus importants et les plus populaires sont présentés théoriquement et implémentés sous forme de logiciels pour les sciences environnementales. Les principaux algorithmes décrits sont le Perceptron multicouches (MultiLayer Perceptron, MLP) - l'algorithme le plus connu dans l'intelligence artificielle, le réseau de neurones de régression généralisée (General Regression Neural Networks, GRNN), le réseau de neurones probabiliste (Probabilistic Neural Networks, PNN), les cartes auto-organisées (SelfOrganized Maps, SOM), les modèles à mixture Gaussiennes (Gaussian Mixture Models, GMM), les réseaux à fonctions de base radiales (Radial Basis Functions Networks, RBF) et les réseaux à mixture de densité (Mixture Density Networks, MDN). Cette gamme d'algorithmes permet de couvrir des tâches variées telle que la classification, la régression ou l'estimation de densité de probabilité. L'analyse exploratoire des données (Exploratory Data Analysis, EDA) est le premier pas de toute analyse de données. Dans cette thèse les concepts d'analyse exploratoire de données spatiales (Exploratory Spatial Data Analysis, ESDA) sont traités selon l'approche traditionnelle de la géostatistique avec la variographie expérimentale et selon les principes de l'apprentissage automatique. La variographie expérimentale, qui étudie les relations entre pairs de points, est un outil de base pour l'analyse géostatistique de corrélations spatiales anisotropiques qui permet de détecter la présence de patterns spatiaux descriptible par une statistique. L'approche de l'apprentissage automatique pour l'ESDA est présentée à travers l'application de la méthode des k plus proches voisins qui est très simple et possède d'excellentes qualités d'interprétation et de visualisation. Une part importante de la thèse traite de sujets d'actualité comme la cartographie automatique de données spatiales. Le réseau de neurones de régression généralisée est proposé pour résoudre cette tâche efficacement. Les performances du GRNN sont démontrées par des données de Comparaison d'Interpolation Spatiale (SIC) de 2004 pour lesquelles le GRNN bat significativement toutes les autres méthodes, particulièrement lors de situations d'urgence. La thèse est composée de quatre chapitres : théorie, applications, outils logiciels et des exemples guidés. Une partie importante du travail consiste en une collection de logiciels : Machine Learning Office. Cette collection de logiciels a été développée durant les 15 dernières années et a été utilisée pour l'enseignement de nombreux cours, dont des workshops internationaux en Chine, France, Italie, Irlande et Suisse ainsi que dans des projets de recherche fondamentaux et appliqués. Les cas d'études considérés couvrent un vaste spectre de problèmes géoenvironnementaux réels à basse et haute dimensionnalité, tels que la pollution de l'air, du sol et de l'eau par des produits radioactifs et des métaux lourds, la classification de types de sols et d'unités hydrogéologiques, la cartographie des incertitudes pour l'aide à la décision et l'estimation de risques naturels (glissements de terrain, avalanches). Des outils complémentaires pour l'analyse exploratoire des données et la visualisation ont également été développés en prenant soin de créer une interface conviviale et facile à l'utilisation. Machine Learning for geospatial data: algorithms, software tools and case studies Abstract The thesis is devoted to the analysis, modeling and visualisation of spatial environmental data using machine learning algorithms. In a broad sense machine learning can be considered as a subfield of artificial intelligence. It mainly concerns with the development of techniques and algorithms that allow computers to learn from data. In this thesis machine learning algorithms are adapted to learn from spatial environmental data and to make spatial predictions. Why machine learning? In few words most of machine learning algorithms are universal, adaptive, nonlinear, robust and efficient modeling tools. They can find solutions for the classification, regression, and probability density modeling problems in high-dimensional geo-feature spaces, composed of geographical space and additional relevant spatially referenced features. They are well-suited to be implemented as predictive engines in decision support systems, for the purposes of environmental data mining including pattern recognition, modeling and predictions as well as automatic data mapping. They have competitive efficiency to the geostatistical models in low dimensional geographical spaces but are indispensable in high-dimensional geo-feature spaces. The most important and popular machine learning algorithms and models interesting for geo- and environmental sciences are presented in details: from theoretical description of the concepts to the software implementation. The main algorithms and models considered are the following: multi-layer perceptron (a workhorse of machine learning), general regression neural networks, probabilistic neural networks, self-organising (Kohonen) maps, Gaussian mixture models, radial basis functions networks, mixture density networks. This set of models covers machine learning tasks such as classification, regression, and density estimation. Exploratory data analysis (EDA) is initial and very important part of data analysis. In this thesis the concepts of exploratory spatial data analysis (ESDA) is considered using both traditional geostatistical approach such as_experimental variography and machine learning. Experimental variography is a basic tool for geostatistical analysis of anisotropic spatial correlations which helps to understand the presence of spatial patterns, at least described by two-point statistics. A machine learning approach for ESDA is presented by applying the k-nearest neighbors (k-NN) method which is simple and has very good interpretation and visualization properties. Important part of the thesis deals with a hot topic of nowadays, namely, an automatic mapping of geospatial data. General regression neural networks (GRNN) is proposed as efficient model to solve this task. Performance of the GRNN model is demonstrated on Spatial Interpolation Comparison (SIC) 2004 data where GRNN model significantly outperformed all other approaches, especially in case of emergency conditions. The thesis consists of four chapters and has the following structure: theory, applications, software tools, and how-to-do-it examples. An important part of the work is a collection of software tools - Machine Learning Office. Machine Learning Office tools were developed during last 15 years and was used both for many teaching courses, including international workshops in China, France, Italy, Ireland, Switzerland and for realizing fundamental and applied research projects. Case studies considered cover wide spectrum of the real-life low and high-dimensional geo- and environmental problems, such as air, soil and water pollution by radionuclides and heavy metals, soil types and hydro-geological units classification, decision-oriented mapping with uncertainties, natural hazards (landslides, avalanches) assessments and susceptibility mapping. Complementary tools useful for the exploratory data analysis and visualisation were developed as well. The software is user friendly and easy to use.

On learning dynamics underlying the evolution of learning rules.

In order to understand the development of non-genetically encoded actions during an animal's lifespan, it is necessary to analyze the dynamics and evolution of learning rules producing behavior. Owing to the intrinsic stochastic and frequency-dependent nature of learning dynamics, these rules are often studied in evolutionary biology via agent-based computer simulations. In this paper, we show that stochastic approximation theory can help to qualitatively understand learning dynamics and formulate analytical models for the evolution of learning rules. We consider a population of individuals repeatedly interacting during their lifespan, and where the stage game faced by the individuals fluctuates according to an environmental stochastic process. Individuals adjust their behavioral actions according to learning rules belonging to the class of experience-weighted attraction learning mechanisms, which includes standard reinforcement and Bayesian learning as special cases. We use stochastic approximation theory in order to derive differential equations governing action play probabilities, which turn out to have qualitative features of mutator-selection equations. We then perform agent-based simulations to find the conditions where the deterministic approximation is closest to the original stochastic learning process for standard 2-action 2-player fluctuating games, where interaction between learning rules and preference reversal may occur. Finally, we analyze a simplified model for the evolution of learning in a producer-scrounger game, which shows that the exploration rate can interact in a non-intuitive way with other features of co-evolving learning rules. Overall, our analyses illustrate the usefulness of applying stochastic approximation theory in the study of animal learning.

A model for the evolution of reinforcement learning in fluctuating games

Many species are able to learn to associate behaviours with rewards as this gives fitness advantages in changing environments. Social interactions between population members may, however, require more cognitive abilities than simple trial-and-error learning, in particular the capacity to make accurate hypotheses about the material payoff consequences of alternative action combinations. It is unclear in this context whether natural selection necessarily favours individuals to use information about payoffs associated with nontried actions (hypothetical payoffs), as opposed to simple reinforcement of realized payoff. Here, we develop an evolutionary model in which individuals are genetically determined to use either trial-and-error learning or learning based on hypothetical reinforcements, and ask what is the evolutionarily stable learning rule under pairwise symmetric two-action stochastic repeated games played over the individual's lifetime. We analyse through stochastic approximation theory and simulations the learning dynamics on the behavioural timescale, and derive conditions where trial-and-error learning outcompetes hypothetical reinforcement learning on the evolutionary timescale. This occurs in particular under repeated cooperative interactions with the same partner. By contrast, we find that hypothetical reinforcement learners tend to be favoured under random interactions, but stable polymorphisms can also obtain where trial-and-error learners are maintained at a low frequency. We conclude that specific game structures can select for trial-and-error learning even in the absence of costs of cognition, which illustrates that cost-free increased cognition can be counterselected under social interactions.

What can ecosystems learn? Expanding evolutionary ecology with learning theory.

BACKGROUND: The structure and organisation of ecological interactions within an ecosystem is modified by the evolution and coevolution of the individual species it contains. Understanding how historical conditions have shaped this architecture is vital for understanding system responses to change at scales from the microbial upwards. However, in the absence of a group selection process, the collective behaviours and ecosystem functions exhibited by the whole community cannot be organised or adapted in a Darwinian sense. A long-standing open question thus persists: Are there alternative organising principles that enable us to understand and predict how the coevolution of the component species creates and maintains complex collective behaviours exhibited by the ecosystem as a whole? RESULTS: Here we answer this question by incorporating principles from connectionist learning, a previously unrelated discipline already using well-developed theories on how emergent behaviours arise in simple networks. Specifically, we show conditions where natural selection on ecological interactions is functionally equivalent to a simple type of connectionist learning, 'unsupervised learning', well-known in neural-network models of cognitive systems to produce many non-trivial collective behaviours. Accordingly, we find that a community can self-organise in a well-defined and non-trivial sense without selection at the community level; its organisation can be conditioned by past experience in the same sense as connectionist learning models habituate to stimuli. This conditioning drives the community to form a distributed ecological memory of multiple past states, causing the community to: a) converge to these states from any random initial composition; b) accurately restore historical compositions from small fragments; c) recover a state composition following disturbance; and d) to correctly classify ambiguous initial compositions according to their similarity to learned compositions. We examine how the formation of alternative stable states alters the community's response to changing environmental forcing, and we identify conditions under which the ecosystem exhibits hysteresis with potential for catastrophic regime shifts. CONCLUSIONS: This work highlights the potential of connectionist theory to expand our understanding of evo-eco dynamics and collective ecological behaviours. Within this framework we find that, despite not being a Darwinian unit, ecological communities can behave like connectionist learning systems, creating internal conditions that habituate to past environmental conditions and actively recalling those conditions. REVIEWERS: This article was reviewed by Prof. Ricard V Solé, Universitat Pompeu Fabra, Barcelona and Prof. Rob Knight, University of Colorado, Boulder.