Evolution of decision-making and learning under fluctuating social environments

The capacity to learn to associate sensory perceptions with appropriate motor actions underlies the success of many animal species, from insects to humans. The evolutionary significance of learning has long been a subject of interest for evolutionary biologists who emphasize the bene¬fit yielded by learning under changing environmental conditions, where it is required to flexibly switch from one behavior to another. However, two unsolved questions are particularly impor¬tant for improving our knowledge of the evolutionary advantages provided by learning, and are addressed in the present work. First, because it is possible to learn the wrong behavior when a task is too complex, the learning rules and their underlying psychological characteristics that generate truly adaptive behavior must be identified with greater precision, and must be linked to the specific ecological problems faced by each species. A framework for predicting behavior from the definition of a learning rule is developed here. Learning rules capture cognitive features such as the tendency to explore, or the ability to infer rewards associated to unchosen actions. It is shown that these features interact in a non-intuitive way to generate adaptive behavior in social interactions where individuals affect each other's fitness. Such behavioral predictions are used in an evolutionary model to demonstrate that, surprisingly, simple trial-and-error learn¬ing is not always outcompeted by more computationally demanding inference-based learning, when population members interact in pairwise social interactions. A second question in the evolution of learning is its link with and relative advantage compared to other simpler forms of phenotypic plasticity. After providing a conceptual clarification on the distinction between genetically determined vs. learned responses to environmental stimuli, a new factor in the evo¬lution of learning is proposed: environmental complexity. A simple mathematical model shows that a measure of environmental complexity, the number of possible stimuli in one's environ¬ment, is critical for the evolution of learning. In conclusion, this work opens roads for modeling interactions between evolving species and their environment in order to predict how natural se¬lection shapes animals' cognitive abilities. - La capacité d'apprendre à associer des sensations perceptives à des actions motrices appropriées est sous-jacente au succès évolutif de nombreuses espèces, depuis les insectes jusqu'aux êtres hu¬mains. L'importance évolutive de l'apprentissage est depuis longtemps un sujet d'intérêt pour les biologistes de l'évolution, et ces derniers mettent l'accent sur le bénéfice de l'apprentissage lorsque les conditions environnementales sont changeantes, car dans ce cas il est nécessaire de passer de manière flexible d'un comportement à l'autre. Cependant, deux questions non résolues sont importantes afin d'améliorer notre savoir quant aux avantages évolutifs procurés par l'apprentissage. Premièrement, puisqu'il est possible d'apprendre un comportement incorrect quand une tâche est trop complexe, les règles d'apprentissage qui permettent d'atteindre un com¬portement réellement adaptatif doivent être identifiées avec une plus grande précision, et doivent être mises en relation avec les problèmes écologiques spécifiques rencontrés par chaque espèce. Un cadre théorique ayant pour but de prédire le comportement à partir de la définition d'une règle d'apprentissage est développé ici. Il est démontré que les caractéristiques cognitives, telles que la tendance à explorer ou la capacité d'inférer les récompenses liées à des actions non ex¬périmentées, interagissent de manière non-intuitive dans les interactions sociales pour produire des comportements adaptatifs. Ces prédictions comportementales sont utilisées dans un modèle évolutif afin de démontrer que, de manière surprenante, l'apprentissage simple par essai-et-erreur n'est pas toujours battu par l'apprentissage basé sur l'inférence qui est pourtant plus exigeant en puissance de calcul, lorsque les membres d'une population interagissent socialement par pair. Une deuxième question quant à l'évolution de l'apprentissage concerne son lien et son avantage relatif vis-à-vis d'autres formes plus simples de plasticité phénotypique. Après avoir clarifié la distinction entre réponses aux stimuli génétiquement déterminées ou apprises, un nouveau fac¬teur favorisant l'évolution de l'apprentissage est proposé : la complexité environnementale. Un modèle mathématique permet de montrer qu'une mesure de la complexité environnementale - le nombre de stimuli rencontrés dans l'environnement - a un rôle fondamental pour l'évolution de l'apprentissage. En conclusion, ce travail ouvre de nombreuses perspectives quant à la mo¬délisation des interactions entre les espèces en évolution et leur environnement, dans le but de comprendre comment la sélection naturelle façonne les capacités cognitives des animaux.

Evolutionary ecology of learning: insights from fruit flies

Ecologically and evolutionarily oriented research on learning has traditionally been carried out on vertebrates and bees. While less sophisticated than those animals, fruit flies (Drosophila) are capable of several forms of learning, and have an advantage of a short generation time, which makes them an ideal system for experimental evolution studies. This review summarizes the insights into evolutionary questions about learning gained in the last decade from evolutionary experiments on Drosophila. These experiments demonstrate that Drosophila have the genetic potential to evolve substantially improved learning performance in ecologically relevant learning tasks. In at least one set of selected populations the improved learning generalized to another task than that used to impose selection, involving a different behavior, different stimuli, and a different sensory channel for the aversive reinforcement. This improvement in learning ability was associated with reduction in other fitness-related traits, such as larval competitive ability and lifespan, pointing out to evolutionary trade-offs of improved learning. These trade-offs were confirmed by other evolutionary experiments where reduction in learning performance was observed as a correlated response to selection for tolerance to larval nutritional stress or for delayed aging. Such trade-offs could be one reason why fruit flies have not fully used up their evolutionary potential for learning ability. Finally, another evolutionary experiment with Drosophila provided the first direct evidence for the long-standing ideas that learning can under some circumstances accelerate and in other slow down genetically-based evolutionary change. These results demonstrate the usefulness of fruit flies as a model system to address evolutionary questions about learning.

New interactive resource feeds children's curiosity

The Public Health Agency and safefood today (Wednesday 22 June) officially launched Eat, Taste and Grow, a new interactive curriculum based education resource to help increase awareness among primary school children of the origins of their food, local produce and the role this plays in healthy eating. As research in 2006 showed, 18% of children aged 2-15 years in Northern Ireland were reported to be obese; and provisional data in 2008-2009 showed that 22.5% of children entering Year 1 were already overweight (17%) or obese (5%).* The launch of Eat, Taste and Grow is an innovative collaboration between the PHA and safefood that will provide children with lifelong lessons about the food they eat, healthy eating options and the benefits of an active lifestyle.The free teacher-led CD-ROM resource will be sent to every primary school across Northern Ireland by September 2011 and will help increase awareness among primary school children of the origins of their food and local produce, and the role this plays in healthy eating.Speaking at the event, Health Minister Edwin Poots said: "Being obese as a child can store up problems for the future, leading to a reduction in life expectancy and potentially causing other health problems such as increased risk of coronary heart disease, cancer and Type 2 Diabetes."Currently around one in four girls and one in six boys in Primary One (Year 1) are overweight or obese."Many of our children are not as physically active as they should be, nor do they have a healthy, balanced diet."This new resource will help teachers in our primary schools educate children on how to choose what foods are healthier for them which hopefully they will carry with them into adulthood."Dr Eddie Rooney, Chief Executive, PHA said: "The Public Health Agency recognises the need to give every child a healthy start in life. Schools play a vital role in contributing to the development of knowledge and skills necessary to make healthier food choices and laying the foundation for good eating habits which can then be carried through into adulthood. Eat, Taste and Grow is an excellent resource that will help equip teachers to carry out this role and in turn enable children to make healthier choices."Mr Martin Higgins, CEO safefood said: "We know and understand the challenges faced by parents to encourage children to eat healthy foods. As obesity among children continues to rise, this interactive resource will educate children in a fun and engaging way while providing them with the information to make informed, responsible choices throughout their lives."The Eat, Taste and Grow resource is an interactive CD-ROM for use on a whiteboard or computer and is divided into: Foundation, Key Stage 1 and Key Stage 2. Topics include 'Where does our food come from?', 'Who produces our food?', 'How food grows' and 'How food is produced and preserved'. Each topic has accompanying teacher's notes and includes ideas for classroom discussions, role play, games and quiz suggestions.

White Paper on Adult Education Learning for Life

This is Ireland’s first White Paper on Adult Education and marks the adoption of lifelong learning as the governing principle of educational policy. The Paper reflects on the role of adult education in society, builds on the consultation process following publication of the Green Paper, and sets out the Government’s policies and priorities for the future development of the sector. It does not aim to provide a policy blueprint for the training sector given that this work is being advanced through the National Employment Action Plans and previous publications, and the work of the Task Force on Lifelong Learning recently established by the Department of Enterprise, Trade and Employment. Rather, it seeks to ensure that there is a fit and complementarity between education and training provision, so as to ensure that learners can move progressively and incrementally within an over-arching co-ordinated and learner-centred framework.

Mise en oeuvre de protocoles spatiaux pour analyser la modulation de processus cognitifs en fonction de la dimension émotionnelle chez le rongeur

Résumé : Emotion et cognition sont deux termes généralement employés pour désigner des processus psychiques de nature opposée. C'est ainsi que les sciences cognitives se sont longtemps efforcées d'écarter la composante «chaude »des processus «froids »qu'elles visaient, si ce n'est pour montrer l'effet dévastateur de la première sur les seconds. Pourtant, les processus cognitifs (de collecte, maintien et utilisation d'information) et émotioAnels (d'activation subjective, physiologique et comportementale face à ce qui est attractif ou aversif) sont indissociables. Par l'approche neuro-éthologique, à l'interface entre le substrat biologique et les manifestations comportementales, nous nous sommes intéressés à une fonction cognitive essentielle, la fonction mnésique, classiquement exprimée chez le rongeur par l'orientation spatiale. Au niveau du substrat, McDonald et White (1993) ont montré la dissociation de trois systèmes de mémoire, avec les rôles de l'hippocampe, du néostriatum et de l'amygdale dans l'encodage des informations respectivement épisodiques, procédurales et émotionnelles. Nous nous sommes penchés sur l'interaction entre ces systèmes en fonction de la dimension émotionnelle par l'éclairage du comportement. L'état émotionnel de l'animal dépend de plusieurs facteurs, que nous avons tenté de contrôler indirectement en comparant leurs effets sur l'acquisition, dans diverses conditions, de la tâche de Morris (qui nécessite la localisation dans un bassin de la position d'une plate-forme submergée), ainsi que sur le style d'exploration de diverses arènes, ouvertes ou fermées, plus ou moins structurées par la présence de tunnels en plexiglas transparent. Nous avons d'abord exploré le rôle d'un composant du système adrénergique dans le rapport à la difficulté et au stress, à l'aide de souris knock-out pour le récepteur à la noradrénaline a-1 B dans un protocole avec 1 ou 4 points de départ dans un bassin partitionné. Ensuite, nous nous sommes penchés, chez le rat, sur les effets de renforcement intermittent dans différentes conditions expérimentales. Dans ces conditions, nous avons également tenté d'analyser en quoi la situation du but dans un paysage donné pouvait interférer avec les effets de certaines formes de stress. Finalement, nous avons interrogé les conséquences de perturbations passées, y compris le renforcement partiel, sur l'organisation des déplacements sur sol sec. Nos résultats montrent la nécessité, pour les souris cont~ô/es dont l'orientation repose sur l'hippocampe, de pouvoir varier les trajectoires, ce qui favoriserait la constitution d'une carte cognitive. Les souris a->B KO s'avèrent plus sensibles au stress et capables de bénéficier de la condition de route qui permet des réponses simples et automatisées, sous-tendues par l'activité du striatum. Chez les rats en bassin 100% renforcé, l'orientation apparaît basée sur l'hippocampe, relayée par le striatum pour le développement d'approches systématiques et rapides, avec réorientation efficace en nouvelle position par réactivation dépendant de l'hippocampe. A 50% de renforcement, on observe un effet du type de déroulement des sessions, transitoirement atténué par la motivation Lorsque les essais s'enchaînent sans pause intrasession, les latences diminuent régulièrement, ce qui suggère une prise en charge possible par des routines S-R dépendant du striatum. L'organisation des mouvements exploratoires apparaît dépendante du niveau d'insécurité, avec différents profils intermédiaires entre la différentiation maximale et la thigmotaxie, qui peuvent être mis en relation avec différents niveaux d'efficacité de l'hippocampe. Ainsi, notre travail encourage à la prise en compte de la dimension émotionnelle comme modulatrice du traitement d'information, tant en phase d'exploration de l'environnement que d'exploitation des connaissances spatiales. Abstract : Emotion and cognition are terms widely used to refer to opposite mental processes. Hence, cognitive science research has for a long time pushed "hot" components away from "cool" targeted processes, except for assessing devastating effects of the former upon the latter. However, cognitive processes (of information collection, preservation, and utilization) and emotional processes (of subjective, physiological, and behavioral activation roue to attraction or aversion) are inseparable. At the crossing between biological substrate and behavioral expression, we studied a chief cognitive function, memory, classically shown in animals through spatial orientation. At the substrate level, McDonald et White (1993) have shown a dissociation between three memory systems, with the hippocampus, neostriatum, and amygdala, encoding respectively episodic, habit, and emotional information. Through the behavior of laboratory rodents, we targeted the interaction between those systems and the emotional axis. The emotional state of an animal depends on different factors, that we tried to check in a roundabout way by the comparison of their effects on acquisition, in a variety of conditions, of the Morris task (in which the location of a hidden platform in a pool is required), as well as on the exploration profile in different apparatus, open-field and closed mazes, more or less organized by clear Plexiglas tunnels. We first tracked the role, under more or less difficult and stressful conditions, of an adrenergic component, with knock-out mice for the a-1 B receptor in a partitioned water maze with 1 or 4 start positions. With rats, we looked for the consequences of partial reinforcement in the water maze in different experimental conditions. In those conditions, we further analyzed how the situation of the goal in the landscape could interfere with the effect of a given stress. At last, we conducted experiments on solid ground, in an open-field and in radial mazes, in order to analyze the organization of spatial behavior following an aversive life event, such as partial reinforcement training in the water maze. Our results emphasize the reliance of normal mice to be able to vary approach trajectories. One of our leading hypotheses is that such strategies are hippocampus-dependent and are best developed for of a "cognitive map like" representation. Alpha-1 B KO mice appear more sensitive to stress and able to take advantage of the route condition allowing simple and automated responses, most likely striatum based. With rats in 100% reinforced water maze, the orientation strategy is predominantly hippocampus dependent (as illustrated by the impairment induced by lesions of this structure) and becomes progressively striatum dependent for the development of systematic and fast successful approaches. Training towards a new platform position requires a hippocampus based strategy. With a 50% reinforcement rate, we found a clear impairment related to intersession disruption, an effect transitorily minimized by motivation enhancement (cold water). When trials are given without intrasession interruption, latencies consistently diminish, suggesting a possibility for striatum dependent stimulus-response routine to occur. The organization of exploratory movements is shown to depend on the level of subjective security, with different intermediary profiles between maximum differentiation and thigmotaxy, which can be considered in parallel with different efficiency levels of the hippocampus dependent strategies. Thus, our work fosters the consideration of emotion as a cognitive treatment modulator, during spatial exploration as well as spatial learning. It leads to a model in which the predominance of hippocampus based exploration is challenged by training conditions of various nature.

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Visuo-motor and cognitive procedural learning in children with basal ganglia pathology.

We investigated procedural learning in 18 children with basal ganglia (BG) lesions or dysfunctions of various aetiologies, using a visuo-motor learning test, the Serial Reaction Time (SRT) task, and a cognitive learning test, the Probabilistic Classification Learning (PCL) task. We compared patients with early (<1 year old, n=9), later onset (>6 years old, n=7) or progressive disorder (idiopathic dystonia, n=2). All patients showed deficits in both visuo-motor and cognitive domains, except those with idiopathic dystonia, who displayed preserved classification learning skills. Impairments seem to be independent from the age of onset of pathology. As far as we know, this study is the first to investigate motor and cognitive procedural learning in children with BG damage. Procedural impairments were documented whatever the aetiology of the BG damage/dysfunction and time of pathology onset, thus supporting the claim of very early skill learning development and lack of plasticity in case of damage.

Spatial learning by rats across visually disconnected environments

Experiments were designed to examine some properties of spatial representations in rats. Adult subjects were trained to escape through a hole at a fixed position in a large circular arena (see Schenk 1989). The experiments were conducted in the dark, with a limited number of controlled visual light cues in order to assess the minimal cue requirement for place learning. Three identical light cues (shape, height and distance from the table) were used. Depending on the condition, they were either permanently on, or alternatively on or off, depending on the position of the rat in the field. Two questions were asked: a) how many identical visual cues were necessary for spatial discrimination in the dark, and b) could rats integrate the relative positions of separate cues, under conditions in which the rat was never allowed to perceive all three cues simultaneously. The results suggest that rats are able to achieve a place discrimination task even if the three cues necessary for efficient orientation can never be seen simultaneously. A dissociation between the discrimination of the spatial position of the goal and the capacity to reach it by a direct path suggests that a reduced number of cues might require prolonged locomotion to allow an accurate orientation in the environment.

Indirect effects of peri-and postnatal choline treatment on place-learning abilities in rat.

This work was aimed at analyzing the effects of perinatal choline supplementation on the development of spatial abilities and upon adult performance. Choline supplementation (3.5 g/L in 0.02 M saccharin solution in tap water) was maintained for two weeks before birth and for up to four weeks postnatally. Additional supplementation was maintained from the fifth to the tenth week postnatally. Spatial-learning capacities were studied at the ages of 26, 65, or 80 days in a circular swimming pool (Morris place-navigation task) and at the age of 7 months in a homing arena. Treatment effects were found in both juvenile and adult rats, and thus persisted for several months after the cessation of the supplementation. The choline supplementation improved the performance in the water maze in a very selective manner. The most consistent effect was a reduction in the latency to reach a cued platform at a fixed position in space, whereas the improvement was limited when the platform was invisible and had to be located relative to distant cues only. However, after removal of the goal cue, the treated rats showed a better retention of the training position than did the control rats. A similar effect was observed in a dry-land task conducted in the homing arena. The choline supplementation thus induced a significant improvement of spatial memory. But since this effect was only evident following training with a salient cue, it might be regarded as an indirect effect promoted by an optimal combination of cue guidance with a place strategy.

MENTAL IMAGERY IN PERCEPTUAL LEARNING

L'imagerie mentale est définie comme une expérience similaire à la perception mais se déroulant en l'absence d'une stimulation physique. Des recherches antérieures ont montré que l'imagerie mentale améliore la performance dans certains domaines, comme par exemple le domaine moteur. Cependant, son rôle dans l'apprentissage perceptif n'a pas encore été étudié. L'apprentissage perceptif correspond à l'amélioration permanente des performances suite à la répétition de la même tâche. Cette thèse présente une série des résultats empiriques qui montrent que l'apprentissage perceptif peut aussi être achevé en l'absence des stimuli physiques. En effet, imaginer des stimuli visuels amène à une meilleure performance avec les stimuli réels. Donc, les processus sous-jacents l'apprentissage perceptif ne sont pas uniquement déclenchés par les stimuli sensoriels, mais également par des signaux internes. En plus, l'apprentissage perceptif à travers l'imagerie mentale ne se réalise que seule-ment quand les stimuli ne sont pas (complètement) présents, mais gaiement quand les stimuli montrés ne sont pas utiles quant à la résolution de la tâche. - Mental imagery is described as an experience that resembles pereeptnal ex-perience but which occurs in the absence ef a physical stimulation. Despite its beneficial effects in, among others, motor performance, the role of mental imagery m perceptual learning has not yet been addressed. Here we focus on a specific sensory modality: vision. Perceptual learning is the ability to improve perception in a stable way through the repetition of a given task Here I demonstrate by a series of empirical results that a perceptual improve¬ment can also occur in the absence of a stimulation. Imagining visual stimuli is sufficient for successful perceptual learning. Hence, processes underlying perceptual learning are not only stimulus-driven but can also be driven by internally generated signals. Moreover, I also show that perceptual learning via mental imagery can occur not only when physical stimuli are (partially) absent, but also in conditions where stimuli are uninformative with respect to the task that has to be learned.

Evaluation of an online, case-based interactive approach to teaching pathophysiology

AIM: The aim of this study was to evaluate a new pedagogical approach in teaching fluid, electrolyte and acid-base pathophysiology in undergraduate students. METHODS: This approach comprises traditional lectures, the study of clinical cases on the web and a final interactive discussion of these cases in the classroom. When on the web, the students are asked to select laboratory tests that seem most appropriate to understand the pathophysiological condition underlying the clinical case. The percentage of students having chosen a given test is made available to the teacher who uses it in an interactive session to stimulate discussion with the whole class of students. The same teacher used the same case studies during 2 consecutive years during the third year of the curriculum. RESULTS: The majority of students answered the questions on the web as requested and evaluated positively their experience with this form of teaching and learning. CONCLUSIONS: Complementing traditional lectures with online case-based studies and interactive group discussions represents, therefore, a simple means to promote the learning and the understanding of complex pathophysiological mechanisms. This simple problem-based approach to teaching and learning may be implemented to cover all fields of medicine.