A model for the evolution of reinforcement learning in fluctuating games

Many species are able to learn to associate behaviours with rewards as this gives fitness advantages in changing environments. Social interactions between population members may, however, require more cognitive abilities than simple trial-and-error learning, in particular the capacity to make accurate hypotheses about the material payoff consequences of alternative action combinations. It is unclear in this context whether natural selection necessarily favours individuals to use information about payoffs associated with nontried actions (hypothetical payoffs), as opposed to simple reinforcement of realized payoff. Here, we develop an evolutionary model in which individuals are genetically determined to use either trial-and-error learning or learning based on hypothetical reinforcements, and ask what is the evolutionarily stable learning rule under pairwise symmetric two-action stochastic repeated games played over the individual's lifetime. We analyse through stochastic approximation theory and simulations the learning dynamics on the behavioural timescale, and derive conditions where trial-and-error learning outcompetes hypothetical reinforcement learning on the evolutionary timescale. This occurs in particular under repeated cooperative interactions with the same partner. By contrast, we find that hypothetical reinforcement learners tend to be favoured under random interactions, but stable polymorphisms can also obtain where trial-and-error learners are maintained at a low frequency. We conclude that specific game structures can select for trial-and-error learning even in the absence of costs of cognition, which illustrates that cost-free increased cognition can be counterselected under social interactions.

Learning-induced gene expression in the heads of two Nasonia species that differ in long-term memory formation.

BACKGROUND: Cellular processes underlying memory formation are evolutionary conserved, but natural variation in memory dynamics between animal species or populations is common. The genetic basis of this fascinating phenomenon is poorly understood. Closely related species of Nasonia parasitic wasps differ in long-term memory (LTM) formation: N. vitripennis will form transcription-dependent LTM after a single conditioning trial, whereas the closely-related species N. giraulti will not. Genes that were differentially expressed (DE) after conditioning in N. vitripennis, but not in N. giraulti, were identified as candidate genes that may regulate LTM formation. RESULTS: RNA was collected from heads of both species before and immediately, 4 or 24 hours after conditioning, with 3 replicates per time point. It was sequenced strand-specifically, which allows distinguishing sense from antisense transcripts and improves the quality of expression analyses. We determined conditioning-induced DE compared to naïve controls for both species. These expression patterns were then analysed with GO enrichment analyses for each species and time point, which demonstrated an enrichment of signalling-related genes immediately after conditioning in N. vitripennis only. Analyses of known LTM genes and genes with an opposing expression pattern between the two species revealed additional candidate genes for the difference in LTM formation. These include genes from various signalling cascades, including several members of the Ras and PI3 kinase signalling pathways, and glutamate receptors. Interestingly, several other known LTM genes were exclusively differentially expressed in N. giraulti, which may indicate an LTM-inhibitory mechanism. Among the DE transcripts were also antisense transcripts. Furthermore, antisense transcripts aligning to a number of known memory genes were detected, which may have a role in regulating these genes. CONCLUSION: This study is the first to describe and compare expression patterns of both protein-coding and antisense transcripts, at different time points after conditioning, of two closely related animal species that differ in LTM formation. Several candidate genes that may regulate differences in LTM have been identified. This transcriptome analysis is a valuable resource for future in-depth studies to elucidate the role of candidate genes and antisense transcription in natural variation in LTM formation.

Predicting diffusion of innovations with self-organisation and machine learning

The main subject of this master's thesis was predicting diffusion of innovations. The prediction was done in a special case: product has been available in some countries, and based on its diffusion in those countries the prediction is done for other countries. The prediction was based on finding similar countries with Self-Organizing Map~(SOM), using parameters of countries. Parameters included various economical and social key figures. SOM was optimised for different products using two different methods: (a) by adding diffusion information of products to the country parameters, and (b) by weighting the country parameters based on their importance for the diffusion of different products. A novel method using Differential Evolution (DE) was developed to solve the latter, highly non-linear optimisation problem. Results were fairly good. The prediction method seems to be on a solid theoretical foundation. The results based on country data were good. Instead, optimisation for different products did not generally offer clear benefit, but in some cases the improvement was clearly noticeable. The weights found for the parameters of the countries with the developed SOM optimisation method were interesting, and most of them could be explained by properties of the products.

Information, learning, and risk in financial markets

Cette thèse comprend trois essais qui abordent l'information le processus d'ap-prentissage ainsi que le risque dans les marchés finances. Elle se concentre d'abord sur les implications à l'équilibre de l'hétérogénéité des agents à travers un processus d'apprentissage comprtemental et de mise à jour de l'information. De plus, elle examine les effets du partage des risques dans un reseau entreprise-fournisseur. Le premier chapitre étudie les effets du biais de disponibili sur l'évaluation des actifs. Ce biais décrit le fait que les agents surestiment l'importance de l'information acquise via l'expérience personnelle. L'hétérogénéité restante des différentes perceptions individuelles amène à une volonté d'échanges. Conformé¬ment aux données empiriques, les jeunes agents échangent plus mais en même temps souffrent d'une performance inférieure. Le deuxième chapitre se penche sur l'impact qu'ont les différences de modelisation entre les agents sur leurs percevons individuelles du processus de prix, dans le contexte des projections de modèles. Les agents sujets à un biais de projection pensent être représentatifs et interprètent les opinions des autres agents comme du bruit. Les agents, avec des modèles plus persistants, perçoivent que les prix réagissent de façon excessive lors des périodes de turbulence. Le troisième chapitre analyse l'impact du partage des risques dans la relation entreprise-fournisseur sur la décision optimale de financement de l'entreprise. Il étudie l'impact sur l'optimisation de la structure du capital ainsi que sur le coût du capital. Les résultats indiquent en particulier qu'un fournisseur avec un effet de levier faible est utile pour le financement d'un nouveau projet d'investissement. Pour des projets très rentables et des fournisseurs à faible effet de levier, le coût des capitaux propres de l'entreprise peut diminuer.

The determinants of CAP reform: learning from the European financial crisis and CAP 2013

There is an ongoing debate on which are the determinants of CAP reform. The economic environment has not been contemplated as a direct determinant of CAP reform but its proxy, the budget, has not only been looked at as such but underlined as a key cause of CAP reform. This paper argues, however, that the budget does not affect the modus operandi of the CAP. It affects the quantity of support each farmer is going to get and sometimes even the timing of the reform, but not the form it is going to receive it. Other CAP determinants and international negotiations in particular, have an impact on the substance of CAP reform. This hypothesis is not contradicted by an analysis of CAP 2013 changes.

M-Learning + Facebook: experiencias y percepciones de estudiantes de diseño

Durante los últimos años, diversas instituciones y universidades han comenzado a experimentar con el m-learning y Facebook a través de diferentes proyectos como parte de sus metodologías de aprendizaje y como una oportunidad para trabajar con los jóvenes. Sin embargo, poco se sabe de las percepciones y experiencias que pueden obtener estudiantes de diseño sobre este tema. En este estudio 24 estudian - tes han completado sus actividades de aprendizaje durante dos meses, utilizando un smarthphone y la popular red social Facebook. Al final del plazo, los estudiantes participaron además en un grupo de discusión para expresar sus experiencias. Los resultados indicaron que los estudiantes utilizaron Facebook como parte de su rutina diaria y que fueron creadores de contenido proporcionando estos a otros. Además los resultados indican que durante el primer mes perdieron mucho tiempo observando contenidos propuestos en Facebook, que después comentaron. El grupo en Facebook fue utilizado para la interacción social principalmente con otros estudiantes y el profesor, como un complemento a las sesiones presenciales. Los resultados obtenidos y el empleo de estrategias, puede ayudar a la concep - tualización del m-learning y mostrar como Facebook puede funcionar como un entorno de aprendizaje para apoyar la enseñanza y aprendizaje en el área del diseño.

From bird to sparrow: Learning-induced modulations in fine-grained semantic discrimination.

Recognition of environmental sounds is believed to proceed through discrimination steps from broad to more narrow categories. Very little is known about the neural processes that underlie fine-grained discrimination within narrow categories or about their plasticity in relation to newly acquired expertise. We investigated how the cortical representation of birdsongs is modulated by brief training to recognize individual species. During a 60-minute session, participants learned to recognize a set of birdsongs; they improved significantly their performance for trained (T) but not control species (C), which were counterbalanced across participants. Auditory evoked potentials (AEPs) were recorded during pre- and post-training sessions. Pre vs. post changes in AEPs were significantly different between T and C i) at 206-232ms post stimulus onset within a cluster on the anterior part of the left superior temporal gyrus; ii) at 246-291ms in the left middle frontal gyrus; and iii) 512-545ms in the left middle temporal gyrus as well as bilaterally in the cingulate cortex. All effects were driven by weaker activity for T than C species. Thus, expertise in discriminating T species modulated early stages of semantic processing, during and immediately after the time window that sustains the discrimination between human vs. animal vocalizations. Moreover, the training-induced plasticity is reflected by the sharpening of a left lateralized semantic network, including the anterior part of the temporal convexity and the frontal cortex. Training to identify birdsongs influenced, however, also the processing of C species, but at a much later stage. Correct discrimination of untrained sounds seems to require an additional step which results from lower-level features analysis such as apperception. We therefore suggest that the access to objects within an auditory semantic category is different and depends on subject's level of expertise. More specifically, correct intra-categorical auditory discrimination for untrained items follows the temporal hierarchy and transpires in a late stage of semantic processing. On the other hand, correct categorization of individually trained stimuli occurs earlier, during a period contemporaneous with human vs. animal vocalization discrimination, and involves a parallel semantic pathway requiring expertise.

Dysregulated ADAM10-Mediated Processing of APP during a Critical Time Window Leads to Synaptic Deficits in Fragile X Syndrome.

The Fragile X mental retardation protein (FMRP) regulates neuronal RNA metabolism, and its absence or mutations leads to the Fragile X syndrome (FXS). The β-amyloid precursor protein (APP) is involved in Alzheimer's disease, plays a role in synapse formation, and is upregulated in intellectual disabilities. Here, we show that during mouse synaptogenesis and in human FXS fibroblasts, a dual dysregulation of APP and the α-secretase ADAM10 leads to the production of an excess of soluble APPα (sAPPα). In FXS, sAPPα signals through the metabotropic receptor that, activating the MAP kinase pathway, leads to synaptic and behavioral deficits. Modulation of ADAM10 activity in FXS reduces sAPPα levels, restoring translational control, synaptic morphology, and behavioral plasticity. Thus, proper control of ADAM10-mediated APP processing during a specific developmental postnatal stage is crucial for healthy spine formation and function(s). Downregulation of ADAM10 activity at synapses may be an effective strategy for ameliorating FXS phenotypes.

Call for uniform neuropsychological assessment after aneurysmal subarachnoid hemorrhage: Swiss recommendations.

BACKGROUND: In a high proportion of patients with favorable outcome after aneurysmal subarachnoid hemorrhage (aSAH), neuropsychological deficits, depression, anxiety, and fatigue are responsible for the inability to return to their regular premorbid life and pursue their professional careers. These problems often remain unrecognized, as no recommendations concerning a standardized comprehensive assessment have yet found entry into clinical routines. METHODS: To establish a nationwide standard concerning a comprehensive assessment after aSAH, representatives of all neuropsychological and neurosurgical departments of those eight Swiss centers treating acute aSAH have agreed on a common protocol. In addition, a battery of questionnaires and neuropsychological tests was selected, optimally suited to the deficits found most prevalent in aSAH patients that was available in different languages and standardized. RESULTS: We propose a baseline inpatient neuropsychological screening using the Montreal Cognitive Assessment (MoCA) between days 14 and 28 after aSAH. In an outpatient setting at 3 and 12 months after bleeding, we recommend a neuropsychological examination, testing all relevant domains including attention, speed of information processing, executive functions, verbal and visual learning/memory, language, visuo-perceptual abilities, and premorbid intelligence. In addition, a detailed assessment capturing anxiety, depression, fatigue, symptoms of frontal lobe affection, and quality of life should be performed. CONCLUSIONS: This standardized neuropsychological assessment will lead to a more comprehensive assessment of the patient, facilitate the detection and subsequent treatment of previously unrecognized but relevant impairments, and help to determine the incidence, characteristics, modifiable risk factors, and the clinical course of these impairments after aSAH.

Could machine learning improve the prediction of pelvic nodal status of prostate cancer patients? Preliminary results of a pilot study.

We tested and compared performances of Roach formula, Partin tables and of three Machine Learning (ML) based algorithms based on decision trees in identifying N+ prostate cancer (PC). 1,555 cN0 and 50 cN+ PC were analyzed. Results were also verified on an independent population of 204 operated cN0 patients, with a known pN status (187 pN0, 17 pN1 patients). ML performed better, also when tested on the surgical population, with accuracy, specificity, and sensitivity ranging between 48-86%, 35-91%, and 17-79%, respectively. ML potentially allows better prediction of the nodal status of PC, potentially allowing a better tailoring of pelvic irradiation.

Dendritic Spine Abnormalities in Hippocampal CA1 Pyramidal Neurons Underlying Memory Deficits in the SAMP8 Mouse Model of Alzheimer's Disease

SAMP8 is a strain of mice with accelerated senescence. These mice have recently been the focus of attention as they show several alterations that have also been described in Alzheimer"s disease (AD) patients. The number of dendritic spines, spine plasticity, and morphology are basic to memory formation. In AD, the density of dendritic spines is severely decreased. We studied memory alterations using the object recognition test. We measured levels of synaptophysin as a marker of neurotransmission and used Golgi staining to quantify and characterize the number and morphology of dendritic spines in SAMP8 mice and in SAMR1 as control animals. While there were no memory differences at 3 months of age, the memory of both 6- and 9-month-old SAMP8 mice was impaired in comparison with age-matched SAMR1 mice or young SAMP8 mice. In addition, synaptophysin levels were not altered in young SAMP8 animals, but SAMP8 aged 6 and 9 months had less synaptophysin than SAMR1 controls and also less than 3-month-old SAMP8 mice. Moreover, while spine density remained stable with age in SAMR1 mice, the number of spines started to decrease in SAMP8 animals at 6 months, only to get worse at 9 months. Our results show that from 6 months onwards SAMP8 mice show impaired memory. This age coincides with that at which the levels of synaptophysin and spine density decrease. Thus, we conclude that together with other studies that describe several alterations at similar ages, SAMP8 mice are a very suitable model for studying AD.

Long-Lasting Metabolic Imbalance Related to Obesity Alters Olfactory Tissue Homeostasis and Impairs Olfactory-Driven Behaviors.

Obesity is associated with chronic food intake disorders and binge eating. Food intake relies on the interaction between homeostatic regulation and hedonic signals among which, olfaction is a major sensory determinant. However, its potential modulation at the peripheral level by a chronic energy imbalance associated to obese status remains a matter of debate. We further investigated the olfactory function in a rodent model relevant to the situation encountered in obese humans, where genetic susceptibility is juxtaposed on chronic eating disorders. Using several olfactory-driven tests, we compared the behaviors of obesity-prone Sprague-Dawley rats (OP) fed with a high-fat/high-sugar diet with those of obese-resistant ones fed with normal chow. In OP rats, we reported 1) decreased odor threshold, but 2) poor olfactory performances, associated with learning/memory deficits, 3) decreased influence of fasting, and 4) impaired insulin control on food seeking behavior. Associated with these behavioral modifications, we found a modulation of metabolism-related factors implicated in 1) electrical olfactory signal regulation (insulin receptor), 2) cellular dynamics (glucorticoids receptors, pro- and antiapoptotic factors), and 3) homeostasis of the olfactory mucosa and bulb (monocarboxylate and glucose transporters). Such impairments might participate to the perturbed daily food intake pattern that we observed in obese animals.