A model for the evolution of reinforcement learning in fluctuating games

Many species are able to learn to associate behaviours with rewards as this gives fitness advantages in changing environments. Social interactions between population members may, however, require more cognitive abilities than simple trial-and-error learning, in particular the capacity to make accurate hypotheses about the material payoff consequences of alternative action combinations. It is unclear in this context whether natural selection necessarily favours individuals to use information about payoffs associated with nontried actions (hypothetical payoffs), as opposed to simple reinforcement of realized payoff. Here, we develop an evolutionary model in which individuals are genetically determined to use either trial-and-error learning or learning based on hypothetical reinforcements, and ask what is the evolutionarily stable learning rule under pairwise symmetric two-action stochastic repeated games played over the individual's lifetime. We analyse through stochastic approximation theory and simulations the learning dynamics on the behavioural timescale, and derive conditions where trial-and-error learning outcompetes hypothetical reinforcement learning on the evolutionary timescale. This occurs in particular under repeated cooperative interactions with the same partner. By contrast, we find that hypothetical reinforcement learners tend to be favoured under random interactions, but stable polymorphisms can also obtain where trial-and-error learners are maintained at a low frequency. We conclude that specific game structures can select for trial-and-error learning even in the absence of costs of cognition, which illustrates that cost-free increased cognition can be counterselected under social interactions.

Extending body space in immersive virtual reality: a very long arm illusion

Recent studies have shown that a fake body part can be incorporated into human body representation through synchronous multisensory stimulation on the fake and corresponding real body part- the most famous example being the Rubber Hand Illusion. However, the extent to which gross asymmetries in the fake body can be assimilated remains unknown. Participants experienced, through a head-tracked stereo head-mounted display a virtual body coincident with their real body. There were 5 conditions in a between-groups experiment, with 10 participants per condition. In all conditions there was visuo-motor congruence between the real and virtual dominant arm. In an Incongruent condition (I), where the virtual arm length was equal to the real length, there was visuo-tactile incongruence. In four Congruent conditions there was visuo-tactile congruence, but the virtual arm lengths were either equal to (C1), double (C2), triple (C3) or quadruple (C4) the real ones. Questionnaire scores and defensive withdrawal movements in response to a threat showed that the overall level of ownership was high in both C1 and I, and there was no significant difference between these conditions. Additionally, participants experienced ownership over the virtual arm up to three times the length of the real one, and less strongly at four times the length. The illusion did decline, however, with the length of the virtual arm. In the C2-C4 conditions although a measure of proprioceptive drift positively correlated with virtual arm length, there was no correlation between the drift and ownership of the virtual arm, suggesting different underlying mechanisms between ownership and drift. Overall, these findings extend and enrich previous results that multisensory and sensorimotor information can reconstruct our perception of the body shape, size and symmetry even when this is not consistent with normal body proportions.

Learning-induced gene expression in the heads of two Nasonia species that differ in long-term memory formation.

BACKGROUND: Cellular processes underlying memory formation are evolutionary conserved, but natural variation in memory dynamics between animal species or populations is common. The genetic basis of this fascinating phenomenon is poorly understood. Closely related species of Nasonia parasitic wasps differ in long-term memory (LTM) formation: N. vitripennis will form transcription-dependent LTM after a single conditioning trial, whereas the closely-related species N. giraulti will not. Genes that were differentially expressed (DE) after conditioning in N. vitripennis, but not in N. giraulti, were identified as candidate genes that may regulate LTM formation. RESULTS: RNA was collected from heads of both species before and immediately, 4 or 24 hours after conditioning, with 3 replicates per time point. It was sequenced strand-specifically, which allows distinguishing sense from antisense transcripts and improves the quality of expression analyses. We determined conditioning-induced DE compared to naïve controls for both species. These expression patterns were then analysed with GO enrichment analyses for each species and time point, which demonstrated an enrichment of signalling-related genes immediately after conditioning in N. vitripennis only. Analyses of known LTM genes and genes with an opposing expression pattern between the two species revealed additional candidate genes for the difference in LTM formation. These include genes from various signalling cascades, including several members of the Ras and PI3 kinase signalling pathways, and glutamate receptors. Interestingly, several other known LTM genes were exclusively differentially expressed in N. giraulti, which may indicate an LTM-inhibitory mechanism. Among the DE transcripts were also antisense transcripts. Furthermore, antisense transcripts aligning to a number of known memory genes were detected, which may have a role in regulating these genes. CONCLUSION: This study is the first to describe and compare expression patterns of both protein-coding and antisense transcripts, at different time points after conditioning, of two closely related animal species that differ in LTM formation. Several candidate genes that may regulate differences in LTM have been identified. This transcriptome analysis is a valuable resource for future in-depth studies to elucidate the role of candidate genes and antisense transcription in natural variation in LTM formation.

Predicting diffusion of innovations with self-organisation and machine learning

The main subject of this master's thesis was predicting diffusion of innovations. The prediction was done in a special case: product has been available in some countries, and based on its diffusion in those countries the prediction is done for other countries. The prediction was based on finding similar countries with Self-Organizing Map~(SOM), using parameters of countries. Parameters included various economical and social key figures. SOM was optimised for different products using two different methods: (a) by adding diffusion information of products to the country parameters, and (b) by weighting the country parameters based on their importance for the diffusion of different products. A novel method using Differential Evolution (DE) was developed to solve the latter, highly non-linear optimisation problem. Results were fairly good. The prediction method seems to be on a solid theoretical foundation. The results based on country data were good. Instead, optimisation for different products did not generally offer clear benefit, but in some cases the improvement was clearly noticeable. The weights found for the parameters of the countries with the developed SOM optimisation method were interesting, and most of them could be explained by properties of the products.

Information, learning, and risk in financial markets

Cette thèse comprend trois essais qui abordent l'information le processus d'ap-prentissage ainsi que le risque dans les marchés finances. Elle se concentre d'abord sur les implications à l'équilibre de l'hétérogénéité des agents à travers un processus d'apprentissage comprtemental et de mise à jour de l'information. De plus, elle examine les effets du partage des risques dans un reseau entreprise-fournisseur. Le premier chapitre étudie les effets du biais de disponibili sur l'évaluation des actifs. Ce biais décrit le fait que les agents surestiment l'importance de l'information acquise via l'expérience personnelle. L'hétérogénéité restante des différentes perceptions individuelles amène à une volonté d'échanges. Conformé¬ment aux données empiriques, les jeunes agents échangent plus mais en même temps souffrent d'une performance inférieure. Le deuxième chapitre se penche sur l'impact qu'ont les différences de modelisation entre les agents sur leurs percevons individuelles du processus de prix, dans le contexte des projections de modèles. Les agents sujets à un biais de projection pensent être représentatifs et interprètent les opinions des autres agents comme du bruit. Les agents, avec des modèles plus persistants, perçoivent que les prix réagissent de façon excessive lors des périodes de turbulence. Le troisième chapitre analyse l'impact du partage des risques dans la relation entreprise-fournisseur sur la décision optimale de financement de l'entreprise. Il étudie l'impact sur l'optimisation de la structure du capital ainsi que sur le coût du capital. Les résultats indiquent en particulier qu'un fournisseur avec un effet de levier faible est utile pour le financement d'un nouveau projet d'investissement. Pour des projets très rentables et des fournisseurs à faible effet de levier, le coût des capitaux propres de l'entreprise peut diminuer.

Multisensory stimulation can induce an illusion of larger belly size in immersive virtual reality

Body change illusions have been of great interest in recent years for the understanding of how the brain represents the body. Appropriate multisensory stimulation can induce an illusion of ownership over a rubber or virtual arm, simple types of out-of-the-body experiences, and even ownership with respect to an alternate whole body. Here we use immersive virtual reality to investigate whether the illusion of a dramatic increase in belly size can be induced in males through (a) first person perspective position (b) synchronous visual-motor correlation between real and virtual arm movements, and (c) self-induced synchronous visual-tactile stimulation in the stomach area.

The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment

This paper reports an experiment that investigated people"s body ownership of an avatar that was observed in a virtual mirror. Twenty subjects were recruited in a within-groups study where 10 first experienced a virtual character that synchronously reflected their upper-body movements as seen in a virtual mirror, and then an asynchronous condition where the mirror avatar displayed prerecorded actions, unrelated to those of the participant. The other 10 subjects experienced the conditions in the opposite order. In both conditions the participant could carry out actions that led to elevation above ground level, as seen from their first person perspective and correspondingly in the mirror. A rotating virtual fan eventually descended to 2m above the ground. The hypothesis was that synchronous mirror reflection would result in higher subjective sense of ownership. A questionnaire analysis showed that the body ownership illusion was significantly greater for thesynchronous than asynchronous condition. Additionally participants in the synchronous condition avoided collision with the descending fan significantly more often than those in the asynchronous condition. The results of this experiment are put into context within similar experiments on multisensory correlation and body ownership within cognitive neuroscience.

Bystander responses to a violent incident in an immersive virtual environment

Under what conditions will a bystander intervene to try to stop a violent attack by one person on another? It is generally believed that the greater the size of the crowd of bystanders, the less the chance that any of them will intervene. A complementary model is that social identity is critical as an explanatory variable. For example, when the bystander shares common social identity with the victim the probability of intervention is enhanced, other things being equal. However, it is generally not possible to study such hypotheses experimentally for practical and ethical reasons. Here we show that an experiment that depicts a violent incident at life-size in immersive virtual reality lends support to the social identity explanation. 40 male supporters of Arsenal Football Club in England were recruited for a two-factor between-groups experiment: the victim was either an Arsenal supporter or not (in-group/out-group), and looked towards the participant for help or not during the confrontation. The response variables were the numbers of verbal and physical interventions by the participant during the violent argument. The number of physical interventions had a significantly greater mean in the ingroup condition compared to the out-group. The more that participants perceived that the Victim was looking to them for help the greater the number of interventions in the in-group but not in the out-group. These results are supported by standard statistical analysis of variance, with more detailed findings obtained by a symbolic regression procedure based on genetic programming. Verbal interventions made during their experience, and analysis of post-experiment interview data suggest that in-group members were more prone to confrontational intervention compared to the out-group who were more prone to make statements to try to diffuse the situation.

The determinants of CAP reform: learning from the European financial crisis and CAP 2013

There is an ongoing debate on which are the determinants of CAP reform. The economic environment has not been contemplated as a direct determinant of CAP reform but its proxy, the budget, has not only been looked at as such but underlined as a key cause of CAP reform. This paper argues, however, that the budget does not affect the modus operandi of the CAP. It affects the quantity of support each farmer is going to get and sometimes even the timing of the reform, but not the form it is going to receive it. Other CAP determinants and international negotiations in particular, have an impact on the substance of CAP reform. This hypothesis is not contradicted by an analysis of CAP 2013 changes.

M-Learning + Facebook: experiencias y percepciones de estudiantes de diseño

Durante los últimos años, diversas instituciones y universidades han comenzado a experimentar con el m-learning y Facebook a través de diferentes proyectos como parte de sus metodologías de aprendizaje y como una oportunidad para trabajar con los jóvenes. Sin embargo, poco se sabe de las percepciones y experiencias que pueden obtener estudiantes de diseño sobre este tema. En este estudio 24 estudian - tes han completado sus actividades de aprendizaje durante dos meses, utilizando un smarthphone y la popular red social Facebook. Al final del plazo, los estudiantes participaron además en un grupo de discusión para expresar sus experiencias. Los resultados indicaron que los estudiantes utilizaron Facebook como parte de su rutina diaria y que fueron creadores de contenido proporcionando estos a otros. Además los resultados indican que durante el primer mes perdieron mucho tiempo observando contenidos propuestos en Facebook, que después comentaron. El grupo en Facebook fue utilizado para la interacción social principalmente con otros estudiantes y el profesor, como un complemento a las sesiones presenciales. Los resultados obtenidos y el empleo de estrategias, puede ayudar a la concep - tualización del m-learning y mostrar como Facebook puede funcionar como un entorno de aprendizaje para apoyar la enseñanza y aprendizaje en el área del diseño.

From bird to sparrow: Learning-induced modulations in fine-grained semantic discrimination.

Recognition of environmental sounds is believed to proceed through discrimination steps from broad to more narrow categories. Very little is known about the neural processes that underlie fine-grained discrimination within narrow categories or about their plasticity in relation to newly acquired expertise. We investigated how the cortical representation of birdsongs is modulated by brief training to recognize individual species. During a 60-minute session, participants learned to recognize a set of birdsongs; they improved significantly their performance for trained (T) but not control species (C), which were counterbalanced across participants. Auditory evoked potentials (AEPs) were recorded during pre- and post-training sessions. Pre vs. post changes in AEPs were significantly different between T and C i) at 206-232ms post stimulus onset within a cluster on the anterior part of the left superior temporal gyrus; ii) at 246-291ms in the left middle frontal gyrus; and iii) 512-545ms in the left middle temporal gyrus as well as bilaterally in the cingulate cortex. All effects were driven by weaker activity for T than C species. Thus, expertise in discriminating T species modulated early stages of semantic processing, during and immediately after the time window that sustains the discrimination between human vs. animal vocalizations. Moreover, the training-induced plasticity is reflected by the sharpening of a left lateralized semantic network, including the anterior part of the temporal convexity and the frontal cortex. Training to identify birdsongs influenced, however, also the processing of C species, but at a much later stage. Correct discrimination of untrained sounds seems to require an additional step which results from lower-level features analysis such as apperception. We therefore suggest that the access to objects within an auditory semantic category is different and depends on subject's level of expertise. More specifically, correct intra-categorical auditory discrimination for untrained items follows the temporal hierarchy and transpires in a late stage of semantic processing. On the other hand, correct categorization of individually trained stimuli occurs earlier, during a period contemporaneous with human vs. animal vocalization discrimination, and involves a parallel semantic pathway requiring expertise.

How we experience immersive virtual environments: the concept of presence and its measurement

This paper reviews the concept of presence in immersive virtual environments, the sense of being there signalled by people acting and responding realistically to virtual situations and events. We argue that presence is a unique phenomenon that must be distinguished from the degree of engagement, involvement in the portrayed environment. We argue that there are three necessary conditions for presence: the (a) consistent low latency sensorimotor loop between sensory data and proprioception; (b) statistical plausibility: images must be statistically plausible in relation to the probability distribution of images over natural scenes. A constraint on this plausibility is the level of immersion;(c) behaviour-response correlations: Presence may be enhanced and maintained over time by appropriate correlations between the state and behaviour of participants and responses within the environment, correlations that show appropriate responses to the activity of the participants. We conclude with a discussion of methods for assessing whether presence occurs, and in particular recommend the approach of comparison with ground truth and give some examples of this.