Evolution of decision-making and learning under fluctuating social environments

The capacity to learn to associate sensory perceptions with appropriate motor actions underlies the success of many animal species, from insects to humans. The evolutionary significance of learning has long been a subject of interest for evolutionary biologists who emphasize the bene¬fit yielded by learning under changing environmental conditions, where it is required to flexibly switch from one behavior to another. However, two unsolved questions are particularly impor¬tant for improving our knowledge of the evolutionary advantages provided by learning, and are addressed in the present work. First, because it is possible to learn the wrong behavior when a task is too complex, the learning rules and their underlying psychological characteristics that generate truly adaptive behavior must be identified with greater precision, and must be linked to the specific ecological problems faced by each species. A framework for predicting behavior from the definition of a learning rule is developed here. Learning rules capture cognitive features such as the tendency to explore, or the ability to infer rewards associated to unchosen actions. It is shown that these features interact in a non-intuitive way to generate adaptive behavior in social interactions where individuals affect each other's fitness. Such behavioral predictions are used in an evolutionary model to demonstrate that, surprisingly, simple trial-and-error learn¬ing is not always outcompeted by more computationally demanding inference-based learning, when population members interact in pairwise social interactions. A second question in the evolution of learning is its link with and relative advantage compared to other simpler forms of phenotypic plasticity. After providing a conceptual clarification on the distinction between genetically determined vs. learned responses to environmental stimuli, a new factor in the evo¬lution of learning is proposed: environmental complexity. A simple mathematical model shows that a measure of environmental complexity, the number of possible stimuli in one's environ¬ment, is critical for the evolution of learning. In conclusion, this work opens roads for modeling interactions between evolving species and their environment in order to predict how natural se¬lection shapes animals' cognitive abilities. - La capacité d'apprendre à associer des sensations perceptives à des actions motrices appropriées est sous-jacente au succès évolutif de nombreuses espèces, depuis les insectes jusqu'aux êtres hu¬mains. L'importance évolutive de l'apprentissage est depuis longtemps un sujet d'intérêt pour les biologistes de l'évolution, et ces derniers mettent l'accent sur le bénéfice de l'apprentissage lorsque les conditions environnementales sont changeantes, car dans ce cas il est nécessaire de passer de manière flexible d'un comportement à l'autre. Cependant, deux questions non résolues sont importantes afin d'améliorer notre savoir quant aux avantages évolutifs procurés par l'apprentissage. Premièrement, puisqu'il est possible d'apprendre un comportement incorrect quand une tâche est trop complexe, les règles d'apprentissage qui permettent d'atteindre un com¬portement réellement adaptatif doivent être identifiées avec une plus grande précision, et doivent être mises en relation avec les problèmes écologiques spécifiques rencontrés par chaque espèce. Un cadre théorique ayant pour but de prédire le comportement à partir de la définition d'une règle d'apprentissage est développé ici. Il est démontré que les caractéristiques cognitives, telles que la tendance à explorer ou la capacité d'inférer les récompenses liées à des actions non ex¬périmentées, interagissent de manière non-intuitive dans les interactions sociales pour produire des comportements adaptatifs. Ces prédictions comportementales sont utilisées dans un modèle évolutif afin de démontrer que, de manière surprenante, l'apprentissage simple par essai-et-erreur n'est pas toujours battu par l'apprentissage basé sur l'inférence qui est pourtant plus exigeant en puissance de calcul, lorsque les membres d'une population interagissent socialement par pair. Une deuxième question quant à l'évolution de l'apprentissage concerne son lien et son avantage relatif vis-à-vis d'autres formes plus simples de plasticité phénotypique. Après avoir clarifié la distinction entre réponses aux stimuli génétiquement déterminées ou apprises, un nouveau fac¬teur favorisant l'évolution de l'apprentissage est proposé : la complexité environnementale. Un modèle mathématique permet de montrer qu'une mesure de la complexité environnementale - le nombre de stimuli rencontrés dans l'environnement - a un rôle fondamental pour l'évolution de l'apprentissage. En conclusion, ce travail ouvre de nombreuses perspectives quant à la mo¬délisation des interactions entre les espèces en évolution et leur environnement, dans le but de comprendre comment la sélection naturelle façonne les capacités cognitives des animaux.

Spatial learning by rats across visually disconnected environments

Experiments were designed to examine some properties of spatial representations in rats. Adult subjects were trained to escape through a hole at a fixed position in a large circular arena (see Schenk 1989). The experiments were conducted in the dark, with a limited number of controlled visual light cues in order to assess the minimal cue requirement for place learning. Three identical light cues (shape, height and distance from the table) were used. Depending on the condition, they were either permanently on, or alternatively on or off, depending on the position of the rat in the field. Two questions were asked: a) how many identical visual cues were necessary for spatial discrimination in the dark, and b) could rats integrate the relative positions of separate cues, under conditions in which the rat was never allowed to perceive all three cues simultaneously. The results suggest that rats are able to achieve a place discrimination task even if the three cues necessary for efficient orientation can never be seen simultaneously. A dissociation between the discrimination of the spatial position of the goal and the capacity to reach it by a direct path suggests that a reduced number of cues might require prolonged locomotion to allow an accurate orientation in the environment.

Spatial learning by rats across visually disconnected environments

Two spatial tasks were designed to test specific properties of spatial representation in rats. In the first task, rats were trained to locate an escape hole at a fixed position in a visually homogeneous arena. This arena was connected with a periphery where a full view of the room environment existed. Therefore, rats were dependent on their memory trace of the previous position in the periphery to discriminate a position within the central region. Under these experimental conditions, the test animals showed a significant discrimination of the training position without a specific local view. In the second task, rats were trained in a radial maze consisting of tunnels that were transparent at their distal ends only. Because the central part of the maze was non-transparent, rats had to plan and execute appropriate trajectories without specific visual feedback from the environment. This situation was intended to encourage the reliance on prospective memory of the non-visited arms in selecting the following move. Our results show that acquisition performance was only slightly decreased compared to that shown in a completely transparent maze and considerably higher than in a translucent maze or in darkness. These two series of experiments indicate (1) that rats can learn about the relative position of different places with no common visual panorama, and (2) that they are able to plan and execute a sequence of visits to several places without direct visual feed-back about their relative position.

On solving fair exchange and related distributed problems in Byzantine environments

Abstract The solvability of the problem of fair exchange in a synchronous system subject to Byzantine failures is investigated in this work. The fair exchange problem arises when a group of processes are required to exchange digital items in a fair manner, which means that either each process obtains the item it was expecting or no process obtains any information on, the inputs of others. After introducing a novel specification of fair exchange that clearly separates safety and liveness, we give an overview of the difficulty of solving such a problem in the context of a fully-connected topology. On one hand, we show that no solution to fair exchange exists in the absence of an identified process that every process can trust a priori; on the other, a well-known solution to fair exchange relying on a trusted third party is recalled. These two results lead us to complete our system model with a flexible representation of the notion of trust. We then show that fair exchange is solvable if and only if a connectivity condition, named the reachable majority condition, is satisfied. The necessity of the condition is proven by an impossibility result and its sufficiency by presenting a general solution to fair exchange relying on a set of trusted processes. The focus is then turned towards a specific network topology in order to provide a fully decentralized, yet realistic, solution to fair exchange. The general solution mentioned above is optimized by reducing the computational load assumed by trusted processes as far as possible. Accordingly, our fair exchange protocol relies on trusted tamperproof modules that have limited communication abilities and are only required in key steps of the algorithm. This modular solution is then implemented in the context of a pedagogical application developed for illustrating and apprehending the complexity of fair exchange. This application, which also includes the implementation of a wide range of Byzantine behaviors, allows executions of the algorithm to be set up and monitored through a graphical display. Surprisingly, some of our results on fair exchange seem contradictory with those found in the literature of secure multiparty computation, a problem from the field of modern cryptography, although the two problems have much in common. Both problems are closely related to the notion of trusted third party, but their approaches and descriptions differ greatly. By introducing a common specification framework, a comparison is proposed in order to clarify their differences and the possible origins of the confusion between them. This leads us to introduce the problem of generalized fair computation, a generalization of fair exchange. Finally, a solution to this new problem is given by generalizing our modular solution to fair exchange

Interactions between learning and immunity in Drosophila melanogaster

Learning and immunity are two adaptive traits with roles in central aspects of an organism's life: learning allows adjusting behaviours in changing environments, while immunity protects the body integrity against parasites and pathogens. While we know a lot about how these two traits interact in vertebrates, the interactions between learning and immunity remain poorly explored in insects. During my PhD, I studied three possible ways in which these two traits interact in the model system Drosophila melanogaster, a model organism in the study of learning and in the study of immunity. Learning can affect the behavioural defences against parasites and pathogens through the acquisition of new aversions for contaminated food for instance. This type of learning relies on the ability to associate a food-related cue with the visceral sickness following ingestion of contaminated food. Despite its potential implication in infection prevention, the existence of pathogen avoidance learning has been rarely explored in invertebrates. In a first part of my PhD, I tested whether D. melanogaster, which feed on food enriched in microorganisms, innately avoid the orally-acquired 'novel' virulent pathogen Pseudomonas entomophila, and whether it can learn to avoid it. Although flies did not innately avoid this pathogen, they decreased their preference for contaminated food over time, suggesting the existence of a form of learning based likely on infection-induced sickness. I further found that flies may be able to learn to avoid an odorant which was previously associated with the pathogen, but this requires confirmation with additional data. If this is confirmed, this would be the first time, to my knowledge, that pathogen avoidance learning is reported in an insect. The detrimental effect of infection on cognition and more specifically on learning ability is well documented in vertebrates and in social insects. While the underlying mechanisms are described in detail in vertebrates, experimental investigations are lacking in invertebrates. In a second part of my PhD, I tested the effect of an oral infection with natural pathogens on associative learning of D. melanogaster. By contrast with previous studies in insects, I found that flies orally infected with the virulent P. entomophila learned better the association of an odorant with mechanical shock than uninfected flies. The effect seems to be specific to a gut infection, and so far I have not been able to draw conclusions on the respective contributions of the pathogen's virulence and of the flies' immune activity in this effect. Interestingly, infected flies may display an increased sensitivity to physical pain. If the learning improvement observed in infected flies was due partially to the activity of the immune system, my results would suggest the existence of physiological connections between the immune system and the nervous system. The basis of these connections would then need to be addressed. Learning and immunity are linked at the physiological level in social insects. Physiological links between traits often result from the expression of genetic links between these traits. However, in social insects, there is no evidence that learning and immunity may be involved in an evolutionary trade-off. I previously reported a positive effect of infection on learning in D. melanogaster. This might suggest that a positive genetic link could exist between learning and immunity. We tested this hypothesis with two approaches: the diallel cross design with inbred lines, and the isofemale lines design. The two approaches provided consistent results: we found no additive genetic correlation between learning and resistance to infection with the diallel cross, and no genetic correlation in flies which are not yet adapted to laboratory conditions in isofemale lines. Consistently with the literature, the two studies suggested that the positive effect of infection on learning I observed might not be reflected by a positive evolutionary link between learning and immunity. Nevertheless, the existence of complex genetic relationships between the two traits cannot be excluded. - L'apprentissage et l'immunité sont deux caractères à valeur adaptative impliqués dans des aspects centraux de la vie d'un organisme : l'apprentissage permet d'ajuster les comportements pour faire face aux changements de l'environnement, tandis que l'immunité protège l'intégrité corporelle contre les attaques des parasites et des pathogènes. Alors que les interactions entre l'apprentissage et l'immunité sont bien documentées chez les vertébrés, ces interactions ont été très peu étudiées chez les insectes. Pendant ma thèse, je me suis intéressée à trois aspects des interactions possibles entre l'apprentissage et l'immunité chez la mouche du vinaigre Drosophila melanogaster, qui est un organisme modèle dans l'étude à la fois de l'apprentissage et de l'immunité. L'apprentissage peut affecter les défenses comportementales contre les parasites et les pathogènes par l'acquisition de nouvelles aversions pour la nourriture contaminée par exemple. Ce type d'apprentissage repose sur la capacité à associer une caractéristique de la nourriture avec la maladie qui suit l'ingestion de cette nourriture. Malgré les implications potentielles pour la prévention des infections, l'évitement appris des pathogènes a été rarement étudié chez les invertébrés. Dans une première partie de ma thèse, j'ai testé si les mouches, qui se nourrissent sur des milieux enrichis en micro-organismes, évitent de façon innée un 'nouveau' pathogène virulent Pseudomonas entomophila, et si elles ont la capacité d'apprendre à l'éviter. Bien que les mouches ne montrent pas d'évitement inné pour ce pathogène, elles diminuent leur préférence pour de la nourriture contaminée dans le temps, suggérant l'existence d'une forme d'apprentissage basée vraisemblablement sur la maladie générée par l'infection. J'ai ensuite observé que les mouches semblent être capables d'apprendre à éviter une odeur qui était au préalable associée avec ce pathogène, mais cela reste à confirmer par la collecte de données supplémentaires. Si cette observation est confirmée, cela sera la première fois, à ma connaissance, que l'évitement appris des pathogènes est décrit chez un insecte. L'effet détrimental des infections sur la cognition et plus particulièrement sur les capacités d'apprentissage est bien documenté chez les vertébrés et les insectes sociaux. Alors que les mécanismes sous-jacents sont détaillés chez les vertébrés, des études expérimentales font défaut chez les insectes. Dans une seconde partie de ma thèse, j'ai mesuré les effets d'une infection orale par des pathogènes naturels sur les capacités d'apprentissage associatif de la drosophile. Contrairement aux études précédentes chez les insectes, j'ai trouvé que les mouches infectées par le pathogène virulent P. entomophila apprennent mieux à associer une odeur avec des chocs mécaniques que des mouches non infectées. Cet effet semble spécifique à l'infection orale, et jusqu'à présent je n'ai pas pu conclure sur les contributions respectives de la virulence du pathogène et de l'activité immunitaire des mouches dans cet effet. De façon intéressante, les mouches infectées pourraient montrer une plus grande réactivité à la douleur physique. Si l'amélioration de l'apprentissage observée chez les mouches infectées était due en partie à l'activité du système immunitaire, mes résultats suggéreraient l'existence de connections physiologiques entre le système immunitaire et le système nerveux. Les mécanismes de ces connections seraient à explorer. L'apprentissage et l'immunité sont liés sur un plan physiologique chez les insectes sociaux. Les liens physiologiques entre les caractères résultent souvent de l'expression de liens entre ces caractères au niveau génétique. Cependant, chez les insectes sociaux, il n'y a pas de preuve que l'apprentissage et l'immunité soient liés par un compromis évolutif. J'ai précédemment rapporté un effet positif de l'infection sur l'apprentissage chez la drosophile. Cela pourrait suggérer qu'une relation génétique positive existerait entre l'apprentissage et l'immunité. Nous avons testé cette hypothèse par deux approches : le croisement diallèle avec des lignées consanguines, et les lignées isofemelles. Les deux approches ont fournies des résultats similaires : nous n'avons pas détecté de corrélation génétique additive entre l'apprentissage et la résistance à l'infection avec le croisement diallèle, et pas de corrélation génétique chez des mouches non adaptées aux conditions de laboratoire avec les lignées isofemelles. En ligne avec la littérature, ces deux études suggèrent que l'effet positif de l'infection sur l'apprentissage que j'ai précédemment observé ne refléterait pas un lien évolutif positif entre l'apprentissage et l'immunité. Néanmoins, l'existence de relations génétiques complexes n'est pas exclue.

Selection on a eumelanic ornament is stronger in the tropics than in temperate zones in the worldwide-distributed barn owl.

Spatial variation in the pattern of natural selection can promote local adaptation and genetic differentiation between populations. Because heritable melanin-based ornaments can signal resistance to environmentally mediated elevation in glucocorticoids, to oxidative stress and parasites, populations may vary in the mean degree of melanic coloration if selection on these phenotypic aspects varies geographically. Within a population of Swiss barn owls (Tyto alba), the size of eumelanic spots is positively associated with survival, immunity and resistance to stress, but it is yet unknown whether Tyto species that face stressful environments evolved towards a darker eumelanic plumage. Because selection regimes vary along environmental gradients, we examined whether melanin-based traits vary clinally and are expressed to a larger extent in the tropics where parasites are more abundant than in temperate zones. To this end, we considered 39 barn owl species distributed worldwide. Barn owl species living in the tropics displayed larger eumelanic spots than those found in temperate zones. This was, however, verified in the northern hemisphere only. Parasites being particularly abundant in the tropics, they may promote the evolution of darker eumelanic ornaments.

Selection on a eumelanic ornament is stronger in the tropics than in temperate zones in the worldwide-distributed barn owl.

Spatial variation in the pattern of natural selection can promote local adaptation and genetic differentiation between populations. Because heritable melanin-based ornaments can signal resistance to environmentally mediated elevation in glucocorticoids, to oxidative stress and parasites, populations may vary in the mean degree of melanic coloration if selection on these phenotypic aspects varies geographically. Within a population of Swiss barn owls (Tyto alba), the size of eumelanic spots is positively associated with survival, immunity and resistance to stress, but it is yet unknown whether Tyto species that face stressful environments evolved towards a darker eumelanic plumage. Because selection regimes vary along environmental gradients, we examined whether melanin-based traits vary clinally and are expressed to a larger extent in the tropics where parasites are more abundant than in temperate zones. To this end, we considered 39 barn owl species distributed worldwide. Barn owl species living in the tropics displayed larger eumelanic spots than those found in temperate zones. This was, however, verified in the northern hemisphere only. Parasites being particularly abundant in the tropics, they may promote the evolution of darker eumelanic ornaments.

Decoding sequence learning from single-trial intracranial EEG in humans.

We propose and validate a multivariate classification algorithm for characterizing changes in human intracranial electroencephalographic data (iEEG) after learning motor sequences. The algorithm is based on a Hidden Markov Model (HMM) that captures spatio-temporal properties of the iEEG at the level of single trials. Continuous intracranial iEEG was acquired during two sessions (one before and one after a night of sleep) in two patients with depth electrodes implanted in several brain areas. They performed a visuomotor sequence (serial reaction time task, SRTT) using the fingers of their non-dominant hand. Our results show that the decoding algorithm correctly classified single iEEG trials from the trained sequence as belonging to either the initial training phase (day 1, before sleep) or a later consolidated phase (day 2, after sleep), whereas it failed to do so for trials belonging to a control condition (pseudo-random sequence). Accurate single-trial classification was achieved by taking advantage of the distributed pattern of neural activity. However, across all the contacts the hippocampus contributed most significantly to the classification accuracy for both patients, and one fronto-striatal contact for one patient. Together, these human intracranial findings demonstrate that a multivariate decoding approach can detect learning-related changes at the level of single-trial iEEG. Because it allows an unbiased identification of brain sites contributing to a behavioral effect (or experimental condition) at the level of single subject, this approach could be usefully applied to assess the neural correlates of other complex cognitive functions in patients implanted with multiple electrodes.

The influence of learning on evolution: a mathematical framework.

The Baldwin effect can be observed if phenotypic learning influences the evolutionary fitness of individuals, which can in turn accelerate or decelerate evolutionary change. Evidence for both learning-induced acceleration and deceleration can be found in the literature. Although the results for both outcomes were supported by specific mathematical or simulation models, no general predictions have been achieved so far. Here we propose a general framework to predict whether evolution benefits from learning or not. It is formulated in terms of the gain function, which quantifies the proportional change of fitness due to learning depending on the genotype value. With an inductive proof we show that a positive gain-function derivative implies that learning accelerates evolution, and a negative one implies deceleration under the condition that the population is distributed on a monotonic part of the fitness landscape. We show that the gain-function framework explains the results of several specific simulation models. We also use the gain-function framework to shed some light on the results of a recent biological experiment with fruit flies.

Electrical neuroimaging of memory discrimination based on single-trial multisensory learning.

Multisensory experiences influence subsequent memory performance and brain responses. Studies have thus far concentrated on semantically congruent pairings, leaving unresolved the influence of stimulus pairing and memory sub-types. Here, we paired images with unique, meaningless sounds during a continuous recognition task to determine if purely episodic, single-trial multisensory experiences can incidentally impact subsequent visual object discrimination. Psychophysics and electrical neuroimaging analyses of visual evoked potentials (VEPs) compared responses to repeated images either paired or not with a meaningless sound during initial encounters. Recognition accuracy was significantly impaired for images initially presented as multisensory pairs and could not be explained in terms of differential attention or transfer of effects from encoding to retrieval. VEP modulations occurred at 100-130ms and 270-310ms and stemmed from topographic differences indicative of network configuration changes within the brain. Distributed source estimations localized the earlier effect to regions of the right posterior temporal gyrus (STG) and the later effect to regions of the middle temporal gyrus (MTG). Responses in these regions were stronger for images previously encountered as multisensory pairs. Only the later effect correlated with performance such that greater MTG activity in response to repeated visual stimuli was linked with greater performance decrements. The present findings suggest that brain networks involved in this discrimination may critically depend on whether multisensory events facilitate or impair later visual memory performance. More generally, the data support models whereby effects of multisensory interactions persist to incidentally affect subsequent behavior as well as visual processing during its initial stages.

Influence of learning on range expansion and adaptation to novel habitats.

Learning has been postulated to 'drive' evolution, but its influence on adaptive evolution in heterogeneous environments has not been formally examined. We used a spatially explicit individual-based model to study the effect of learning on the expansion and adaptation of a species to a novel habitat. Fitness was mediated by a behavioural trait (resource preference), which in turn was determined by both the genotype and learning. Our findings indicate that learning substantially increases the range of parameters under which the species expands and adapts to the novel habitat, particularly if the two habitats are separated by a sharp ecotone (rather than a gradient). However, for a broad range of parameters, learning reduces the degree of genetically-based local adaptation following the expansion and facilitates maintenance of genetic variation within local populations. Thus, in heterogeneous environments learning may facilitate evolutionary range expansions and maintenance of the potential of local populations to respond to subsequent environmental changes.

Length of activity season drives geographic variation in body size of a widely distributed lizard

Understanding the factors that drive geographic variation in life history is an important challenge in evolutionary ecology. Here, we analyze what predicts geographic variation in life-history traits of the common lizard, Zootoca vivipara, which has the globally largest distribution range of all terrestrial reptile species. Variation in body size was predicted by differences in the length of activity season, while we found no effects of environmental temperature per se. Females experiencing relatively short activity season mature at a larger size and remain larger on average than females in populations with relatively long activity seasons. Interpopulation variation in fecundity was largely explained by mean body size of females and reproductive mode, with viviparous populations having larger clutch size than oviparous populations. Finally, body size-fecundity relationship differs between viviparous and oviparous populations, with relatively lower reproductive investment for a given body size in oviparous populations. While the phylogenetic signal was weak overall, the patterns of variation showed spatial effects, perhaps reflecting genetic divergence or geographic variation in additional biotic and abiotic factors. Our findings emphasize that time constraints imposed by the environment rather than ambient temperature play a major role in shaping life histories in the common lizard. This might be attributed to the fact that lizards can attain their preferred body temperature via behavioral thermoregulation across different thermal environments. Length of activity season, defining the maximum time available for lizards to maintain optimal performance, is thus the main environmental factor constraining growth rate and annual rates of mortality. Our results suggest that this factor may partly explain variation in the extent to which different taxa follow ecogeographic rules.