Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Acid-sensing ion channels : modulation by serine-proteases and involvement in acid-induced action potential generation in hippocampal neurons

SUMMARY Acid-sensing ion channels (ASICs) are non-voltage gated sodium channels. They are activated by rapid extracellular acidification and generate an inactivating inward current. Four ASIC genes have been cloned: ASIC1, 2, 3 and 4, with variants a and b for ASIC1and AS1C2. ASICs are expressed in neurons of the central (CNS) and peripheral nervous system (PNS). In the CNS, ASICs have a role in learning, memory, as well as in neuronal death in ischemia. In the PNS, ASICs are involved in the perception of acid-induced pain, as well as in mechanoperception. In one part of my thesis project, we addressed the question of the mechanism of regulation of ASIC1 a by the serine protease trypsin at the molecular level. Trypsin modifies the function of ASIC1 a but not of ASIC1b. In order to identify the channel region responsible for this effect, we created chimeras between ASIC1 a and 1b. Subsequently, to identify the exact trypsin target(s), we mutated predicted trypsin sites in the region identified by the chimera. In the second part of a project, we investigated the role of ASICs at the cellular level, in neuronal signaling. Using the whole-cell patch clamp in hippocampal neuronal culture, we studied the potential involvement of ASICs in action potential (AP) generation. In the first part of the thesis work, we showed that trypsin modifies ASIC1a function: it shifts the pH activation and the steady-state inactivation curve towards more acidic values and accelerates the time course of the channel recovery from inactivation. We also showed that trypsin cleaves ASIC1a and that the functional effect and a channel cleavage correlate. In the inactivated state, channels cannot be modified by trypsin. Cleavage occurs in a channel region that is also important for inactivation of all ASICs; a part of this region is critical for the inhibition of ASIC1 a by the spider toxin Psalmotoxin1. In the second part of the thesis work, we showed that ASIC activity can modulate AP generation. ASIC activity by itself can induce trains of APs. In situations in which this activity by itself is not sufficient to induce APs, it can contribute to AP generation. During high neuronal activity, ASIC activity can block already existing trains of APs. In conclusion, depending on the activity of neuron in a particular moment, ASICs can differently modulate AP generation; they can induce, facilitate or inhibit APs. We also showed that trypsin changes the capability of ASICs to modulate AP generation by shifting the pH dependence to more acidic values, which adapts channel gating to pH conditions which may occur in pathological conditions such as ischemia. Our finding that trypsin modifies ASIC1 a function identifies a novel pharmacological tool, and proposes a mechanism of ASIC1a regulation that may have a physiological importance. The identification of the exact site of trypsin action gives insight to the molecular mechanisms of ASIC regulation. This work proposes a role in modulation of AP generation for ASICs in the CNS. RESUME Les canaux ASIC sont les canaux ioniques activés par l'acidification rapide extracellulaire. Activés, ils génèrent un courant entrant qui inactive en présence de stimulus acide. Quatre gènes ASIC ont été clonés, ASIC1, 2, 3 et 4, avec les variants a et b pour ASIC1 et 2. Les ASICs sont exprimés dans les neurones du système nerveux central (SNC) et périphérique (SNP). Dans le SNC, les ASIC ont un rôle dans le mémoire, apprentissage et la mort neuronale dans t'ischémie. Dans le SNP, ils ont un rôle dans la perception de la douleur et méchanosensation. Dans une partie de mon projet de thèse, nous avons étudié les mécanismes de la régulation d'ASIC1a par la sérine-protéase trypsine au niveau moléculaire. La trypsine modifie la fonction d'ASIC1a et pas ASIC1b. Nous avons créé les chimères entre ASIC1 a et 1 b, afin d'identifier la région du canal responsable pour l'effet. Pour identifier le(s) site(s) exactes de l'action de la trypsine, nous avons muté les sites potentiels de la trypsine dans la région identifiée par les chimères. Dans la deuxième partie du projet, nous avons étudié le rôle des ASICs au niveau cellulaire. En utilisant la technique du patch clamp dans les cultures des neurones de l'hippocampe, nous avons étudié l'implication des ASICs dans la génération des potentiels d'action (PA). Nous avons montré que la trypsine agit sur le canal ASIC1a ; elle décale l'activation et « steady-state » inactivation vers les valeurs plus acides, et elle raccourcit le temps du « recovery » du canal. La trypsine coupe ASIC1a sur le résidu K145 et l'effet fonctionnel et la coupure corrèlent. Nous avons identifié la région du canal responsable pour l'inactivation de tous les ASICs ; une partie de cette région est responsable pour ['inhibition d'ASIC1 a par la Psalmotoxinel . Nous avons montré que les ASICs peuvent moduler la génération des PAs. L'activité des ASICs peut induire les trains des PAs. Quand l'activité des ASICs n'est pas suffisante pour induire le PA, elle peut contribuer à sa génération. Pendant l'activité neuronale forte, l'activité des ASICs peut bloquer les trains des PAs qui existent déjà. En conclusion, dépendant de l'activité neuronale, les ASICs peuvent moduler la génération des PAs différemment ; ils peuvent induire, faciliter ou inhiber les PAs. La trypsine change la capacité des ASICs de moduler les PAs. Après l'action de la trypsine, les ASICs peuvent moduler la génération des PAs dans les conditions légèrement acides, suivies par les fluctuations du pH acide, qui peuvent exister dans l'ischémie. Le fait que la trypsine agit sur ASIC1a définit l'outil pharmacologique et propose le mécanisme de la régulation d'ASICI a qui pourrait avoir l'importance physiologique. L'identification du site de l'action de la trypsine éclaircit les mécanismes moléculaires de la régulation des ASICs. Cette étude propose un rôle des ASICs dans la modulation de la génération des PAs. Résumé pour le public large Les neurones sont les cellules de système nerveux dont la fonction est la signalisation. Comme toutes les autres cellules, les neurones ont une membrane qui sépare l'intérieur du milieu extérieur. Cette membrane est imperméable pour des particules chargées (ions). Dans cette membrane existent les protéines spécifiques, « canaux », qui permettent le transport des ions d'un côté de la membrane à l'autre, comme réponse aux stimuli différents. Ce transport des ions à travers la membrane génère un courant, qu'on peut mesurer. Ce courant est la base de la communication entre les neurones, ou, ce qu'on appelle la signalisation neuronale. Quand ce courant est suffisamment grand, il permet la génération du potentiel d'action, qui est le message principal de communication neuronale. Les canaux ASIC (acid-sensing ion channel), que nous étudions dans le laboratoire, sont activés par les acides. Les acides sont relâchés dans beaucoup de situations dans le système nerveux. Les ASIC ont été découverts récemment (en 1996), et nous ne connaissons pas encore très bien toutes les fonctions de ces canaux. Nous savons qu'ils ont un rôle dans le mémoire, apprentissage, la sensation de la douleur et l'infarctus cérébral. Dans la première partie de ce projet de thèse, nous avons voulu mieux comprendre comment fonctionnent ces canaux. Pour faire ça, nous avons étudié la régulation des ASICs par une protéine, trypsine, qui coupe le canal ASIC. Nous avons étudié ou exactement la trypsine coupe le canal et quels effets ça produit sur la fonction du canal. Dans la deuxième partie du projet de thèse, nous avons voulu mieux connaître comment le canal fonctionne au niveau de la cellule, comment il interagit avec les autres canaux et si il a un rôle dans la génération des potentiels d'action. Nous avons pu montrer que la trypsine change la fonction du canal, ce qui lui permet de fonctionner différemment. Nous avons aussi déterminé ou exactement ta trypsine coupe le canal. Au niveau de la cellule, nous avons montré que les ASIC peuvent moduler la génération des potentiels d'action, étant, dépendant de l'activité du neurone, soit activateurs, soit inhibiteurs. La trypsine est une molécule qui peut être libérée dans le système nerveux pendant certaines conditions, comme l'infarctus cérébral. A cause de ça, les connaissances que la trypsine agit sur le anal ASIC pourraient être important physiologiquement. La connaissance de l'endroit exacte ou la trypsine coupe le canal nous aide à mieux comprendre la relation structure-fonction du canal. La modulation de la génération des potentiels d'actions par les ASIC indique que ces canaux peuvent avoir un rôle important dans la signalisation neuronale.

Modes of acquisition of health literacy skills in informal learning contexts

In this article we try to analyze the learning processes of health literacy skills in informal contexts. We intend to broaden the understanding of the learning process beyond the formal contexts, thus contributing to the elucidation of health professionals on how individuals acquire and manage their knowledge in health matters. Given our goal, we use an analytic corpus constituted by one hundred autobiographical narratives written between 2006 and 2011, in educational contexts but with recognized potential for use in different scientific fields, including health. The results reveal the existence of three different types of modes of learning health literacy skills in informal context: : i) learning that takes place in action, in achieving daily tasks; ii) learning processes that result from problem solving; iii) learning that occurs in an unplanned manner, resulting from accidental circumstances and, in some cases, devoid of intentionality.

Novel action of a wake-promoting neuropeptide in hippocampus : inhibition of nmda receptor function at excitatory synapses through orexin-a

One of the most intriguing functions of the brain is the ability to learn and memorize. The mechanism through which memory and learning are expressed requires the activation of NMDA receptors (NMDARs). These molecular entities are placed at the postsynaptic density of excitatory synapses and their function is tightly controlled by the actions of several modulators at the extracellular, intracellular and pore sites. A large part of the intracellular modulation comes from the action of G-protein coupled receptors (GPCRs). Through intracellular cascades typically involving kinases and phosphatases, GPCRs potentiate or inhibit NMDARs, controlling the conductive state but also the trafficking within the synapse. The GPCRs are involved in the modulation of a variety of brain functions. Many of them control cognition, memory and learning performance, therefore, their effects on NMDARs are extensively studied. The orexinergic system signals through GPCRs and it is well known for the regulation of waking, feeding, reward and autonomic functions. Moreover, it is involved in potentiating hippocampus-related cognitive tasks. Orexin receptors and fibers are present within the hippocampus, but whether these directly modulate hippocampal cells and synapses has not yet been determined. During my thesis, I studied orexinergic actions on excitatory synaptic transmission via whole-cell patch-clamp recordings in rat acute hippocampal slices. I observed that exogenously applied orexin-A (ox-A) exerted a strong inhibitory action on NMDAR-mediated synaptic potentials at mossy fiber (MF)-CA3 synapses, by postsynaptically activating orexin-2 receptors, a minor inhibition at Schaffer collateral-CAl synapses and did not affect other synapses with the CA3 area. Moreover, I demonstrated that the susceptibility of NMDARs to ox- A depends on the tone of endogenous orexin known to fluctuate during the day-night cycle. In fact, in slices prepared during the active period of the rats, when endogenous orexin levels are high, NMDAR-currents were not affected by exogenously applied ox-A. The inhibitory effect of ox-A was, however, reverted when interfering with the orexinergic system through intraperitoneal injections of almorexant, a dual orexin receptor antagonist, during the active phase prior to slice preparation. This thesis work suggests that the orexinergic system regulates NMDAR-dependent information flow through select hippocampal pathways depending on the time-of-day. The specific orexinergic modulation of NMDARs at MFs dampens the excitability of the hippocampal circuit and could impede the mechanisms related to memory formation, possibly also following extended periods of waking. -- La capacité d'apprentissage et de mémorisation est une des fonctions les plus intrigantes de notre cerveau. Il a été montré qu'elles requièrent l'activation des récepteurs NMDA (NMDARs). Ces entités moléculaires sont présentes au niveau de la densité post-synaptique des synapses excitatrices et leur fonction est étroitement contrôlée par l'action de nombreux modulateurs au niveau extracellulaire, intracellulaire et membranaire de ces récepteurs. Une grande partie de la modulation intracellulaire s'effectue via l'action de récepteurs couplés aux protéines G (GPCRs). Grace à leurs cascades intracellulaires typiquement impliquant des kinases et des phosphatases, les GPCRs favorisent l'activation ou l'inhibition des NMDARs, contrôlant ainsi leur perméabilité mais aussi leur mouvement à la synapse. Les GPCRs sont impliquées dans de nombreuses fonctions cérébrales telles que la cognition, la mémoire ainsi que la capacité d'apprentissage c'est pour cela que leurs effets sur les NMDARs sont très étudiés. Le système orexinergique fait intervenir ces GPCRs et est connu par son rôle dans la régulation de fonctions physiologiques telles que l'éveil, la prise alimentaire, la récompense ainsi que d'autres fonctions du système nerveux autonome. De plus, ce système est impliqué dans la régulation de tâches cognitives liées à l'hippocampe. Bien que les fibres et les récepteurs à l'orexine soient présents dans l'hippocampe, leur mécanisme d'action sur les cellules et les synapses de l'hippocampe n'a pas encore été élucidé. Durant ma thèse, je me suis intéressée aux effets de l'orexine sur la transmission synaptique excitatrice en utilisant la méthode d'enregistrement en patch-clamp en configuration cellule entière sur des tranches aiguës d'hippocampes de rats. J'ai observé que l'application exogène d'orexine A d'une part inhibe fortement les courants synaptiques dépendants de l'activation des NMDARs au niveau de la synapse entre les fibres moussues et CA3 via l'activation post-synaptique des orexine récepteurs 2 mais d'autre part n'inhibe que de façon mineure la synapse entre les collatérales de Schaffer et CAI et n'affecte pas les autres synapses impliquant CA3. J'ai également démontré que la sensibilité des NMDARs à l'orexine A dépend de sa concentration endogène qui fluctue durant le cycle éveil-sommeil. En effet, lorsque les coupes d'hippocampes sont préparées durant la période active de l'animal correspondant à un niveau endogène d'orexine élevé, l'application exogène d'orexine A n'a aucun effet sur les courants dépendants de l'activation des NMDARs. Cependant, l'injection dans le péritoine, durant la phase active de l'animal, d'un antagoniste des orexine récepteurs, l'almorexant, va supprimer l'effet inhibiteur de l'orexine A. Les résultats de ma thèse suggèrent donc que le système orexinergique module les informations véhiculées par les NMDARs via des voies de signalisation sélectives de l'hippocampe en fonction du moment de la journée. La modulation orexinergique des NMDARs au niveau des fibres moussues diminue ainsi l'excitabilité du circuit hippocampal et pourrait entraver les mécanismes liés à la formation de la mémoire, potentiellement après de longues périodes d'éveil.

The handaxe and the microscope: individual and social learning in a multidimensional model of adaptation

When individuals learn by trial-and-error, they perform randomly chosen actions and then reinforce those actions that led to a high payoff. However, individuals do not always have to physically perform an action in order to evaluate its consequences. Rather, they may be able to mentally simulate actions and their consequences without actually performing them. Such fictitious learners can select actions with high payoffs without making long chains of trial-and-error learning. Here, we analyze the evolution of an n-dimensional cultural trait (or artifact) by learning, in a payoff landscape with a single optimum. We derive the stochastic learning dynamics of the distance to the optimum in trait space when choice between alternative artifacts follows the standard logit choice rule. We show that for both trial-and-error and fictitious learners, the learning dynamics stabilize at an approximate distance of root n/(2 lambda(e)) away from the optimum, where lambda(e) is an effective learning performance parameter depending on the learning rule under scrutiny. Individual learners are thus unlikely to reach the optimum when traits are complex (n large), and so face a barrier to further improvement of the artifact. We show, however, that this barrier can be significantly reduced in a large population of learners performing payoff-biased social learning, in which case lambda(e) becomes proportional to population size. Overall, our results illustrate the effects of errors in learning, levels of cognition, and population size for the evolution of complex cultural traits. (C) 2013 Elsevier Inc. All rights reserved.

On learning dynamics underlying the evolution of learning rules.

In order to understand the development of non-genetically encoded actions during an animal's lifespan, it is necessary to analyze the dynamics and evolution of learning rules producing behavior. Owing to the intrinsic stochastic and frequency-dependent nature of learning dynamics, these rules are often studied in evolutionary biology via agent-based computer simulations. In this paper, we show that stochastic approximation theory can help to qualitatively understand learning dynamics and formulate analytical models for the evolution of learning rules. We consider a population of individuals repeatedly interacting during their lifespan, and where the stage game faced by the individuals fluctuates according to an environmental stochastic process. Individuals adjust their behavioral actions according to learning rules belonging to the class of experience-weighted attraction learning mechanisms, which includes standard reinforcement and Bayesian learning as special cases. We use stochastic approximation theory in order to derive differential equations governing action play probabilities, which turn out to have qualitative features of mutator-selection equations. We then perform agent-based simulations to find the conditions where the deterministic approximation is closest to the original stochastic learning process for standard 2-action 2-player fluctuating games, where interaction between learning rules and preference reversal may occur. Finally, we analyze a simplified model for the evolution of learning in a producer-scrounger game, which shows that the exploration rate can interact in a non-intuitive way with other features of co-evolving learning rules. Overall, our analyses illustrate the usefulness of applying stochastic approximation theory in the study of animal learning.

Action Research : the first steps to start up a pilot experiment in heritage education

Peer-reviewed

A model for the evolution of reinforcement learning in fluctuating games

Many species are able to learn to associate behaviours with rewards as this gives fitness advantages in changing environments. Social interactions between population members may, however, require more cognitive abilities than simple trial-and-error learning, in particular the capacity to make accurate hypotheses about the material payoff consequences of alternative action combinations. It is unclear in this context whether natural selection necessarily favours individuals to use information about payoffs associated with nontried actions (hypothetical payoffs), as opposed to simple reinforcement of realized payoff. Here, we develop an evolutionary model in which individuals are genetically determined to use either trial-and-error learning or learning based on hypothetical reinforcements, and ask what is the evolutionarily stable learning rule under pairwise symmetric two-action stochastic repeated games played over the individual's lifetime. We analyse through stochastic approximation theory and simulations the learning dynamics on the behavioural timescale, and derive conditions where trial-and-error learning outcompetes hypothetical reinforcement learning on the evolutionary timescale. This occurs in particular under repeated cooperative interactions with the same partner. By contrast, we find that hypothetical reinforcement learners tend to be favoured under random interactions, but stable polymorphisms can also obtain where trial-and-error learners are maintained at a low frequency. We conclude that specific game structures can select for trial-and-error learning even in the absence of costs of cognition, which illustrates that cost-free increased cognition can be counterselected under social interactions.

Policy learning in Europe : social policy and labour market reforms in 11 countries

This report compares policy learning processes in 11 European countries. Based on the country reports that were produced by the national teams of the INSPIRES project, this paper develops an argument that connects problem pressure and politicization to learning in different labor market innovations. In short, we argue that learning efforts are most likely to impact on policy change if there is a certain problem pressure that clearly necessitates political action. On the other hand, if problem pressure is very low, or so high that governments need to react immediately, chances are low that learning impacts on policy change. The second part of our argument contends that learning impacts on policy change especially if a problem is not very politicized, i.e. there are no main conflicts concerning a reform, because then, solutions are wound up in the search for a compromise. Our results confirm our first hypothesis regarding the connection between problem pressure and policy learning. Governments learn indeed up to a certain degree of problem pressure. However, once political action becomes really urgent, i.e. in anti-crisis policies, there is no time and room for learning. On the other hand, learning occurred independently from the politicization of problem. In fact, in countries that have a consensual political system, learning occurred before the decision on a reform, whereas in majoritarian systems, learning happened after the adoption of a policy during the process of implementation.

CRM and higher education : developing a monitoring system to improve relationships in e-learning environments

Peer-reviewed

A virtual mathematics learning environment for engineering students

Teachers of the course Introduction to Mathematics for Engineers at the UOC, an online distance-learning university, have designed,developed and tested an online studymaterial. It includes basic pre-university mathematics, indications for correct follow-up of this content and recommendations for finding appropriate support and complementarymaterials. Many different resources are used,depending on the characteristics of thecontents: Flash sequences, interactive applets, WIRIS calculators and PDF files.During the last semester, the new study material has been tested with 119 students. The academic results and student satisfaction have allowed us to outline and prioritise future lines of action.

The FaceUOC Project : An Open Social Learning experience by the Open University of Catalonia

The possibilities and expansion of the use of Web 2.0 has opened up a world of possibilities in online learning. In spite of the integration of these tools in education major changes are required in the educational design of instructional processes.This paper presents an educational experience conducted by the Open University of Catalonia using the social network Facebook for the purpose of testing a learning model that uses a participation and collaboration methodology among users based on the use of open educational resources.- The aim of the experience is to test an Open Social Learning (OSL) model, understood to be a virtual learning environment open to the Internet community, based on the use of open resources and on a methodology focused on the participation and collaboration of users in the construction of knowledge.- The topic chosen for this experience in Facebook was 2.0 Journeys: online tools and resources. The objective of this 5 weeks course was to provide students with resources for managing the various textual, photographic, audiovisual and multimedia materials resulting from a journey.- The most important changes in the design and development of a course based on OSL are the role of the teacher, the role of the student, the type of content and the methodology:- The teacher mixes with the participants, guiding them and offering the benefit of his/her experience and knowledge.- Students learn through their participation and collaboration with a mixed group of users.- The content is open and editable under different types of license that specify the level of accessibility.- The methodology of the course was based on the creation of a learning community able to self-manage its learning process. For this a facilitator was needed and also a central activity was established for people to participate and contribute in the community.- We used an ethnographic methodology and also questionnaires to students in order to acquire results regarding the quality of this type of learning experience.- Some of the data obtained raised questions to consider for future designs of educational situations based on OSL:- Difficulties in breaking the facilitator-centred structure- Change in the time required to adapt to the system and to achieve the objectives- Lack of commitment with free courses- The trend to return to traditional ways of learning- Accreditation- This experience has taught all of us that education can happen any time and in any place but not in any way.

eLab, a Personal Learning Environment for the Green Open Innovation Platform

Open Innovation is a relatively new concept which involves a change of paradigm in the R+D+i processes of companies whose aim is to create new technologies or new processes. If to this change, we add the need for innovation in the new green and sustainability economy, and we set out to create a collaborative platform with a learning space where this can happen, we will be facing an overwhelming challenge which requires the application of intelligent programming technologies and languages at the service of education.The aim of the Green IDI (Green Open Innovation) ¿ Economic development and job creation vector in SMEs, based on the environment and sustainability project is to create a platform where companies and individual researchers can perform open innovation processes in the field of sustainability and the environment.The Green IDI (Green Open Innovation) project is funded under the program INNPACTO by the Ministry of Science and Innovation of Spain and is being developed through a consortium formed by the following institutions: GRUPO ICA; COMPARTIA; GRUPO INTERCOM; CETAQUA and the Instituto de Investigación en Inteligencia Artificial (IIIA) from Consejo Superior de Investigaciones Científicas (CSIC). Also the consortium include FUNDACIÓ PRIVADA BARCELONA DIGITAL; PIMEC and UNIVERSITAT OBERTA DE CATALUNYA (UOC).Sustainability and positive action for the environment are considered the principle vector of economic development for companies. As Nicolás Scoli says (2007) ¿in short, preventing unnecessary consumption and the efficient consumption of resources means producing greater wealth with less. Both effects lead to reduced pollution linked to production and consumption¿.The Spanish Sustainable Development Strategy (EEDS) plan defends consumption and sustainable production linked to social and economic development by adhering to the commitment not to endanger ecosystems and abolishing the idea that economic growth is directly proportional to the deterioration of the environment.Uniting the Open Innovation and New Green Economy concepts leads to the "Green Open Innovation¿ Platform creation project.This article analyses the concept of open innovation and defines the importance of the new green and sustainable economy. Lastly, it proposes the creation of eLab. The eLab is defined as an Open Green Innovation Platform personal and collaborative education space which is fed by the interactions of users and which enables innovation processes based on new green economy concepts to be carried out.The creation of a personal learning environment such as eLab on the Green Open Innovation Platform meets the need to offer a collaborative space where platform users can improve their skills regarding the environment and sustainability based on collaborative synergies through Information and Communication Technologies.

Neural correlates of language learning in adults

The human language-learning ability persists throughout life, indicating considerable flexibility at the cognitive and neural level. This ability spans from expanding the vocabulary in the mother tongue to acquisition of a new language with its lexicon and grammar. The present thesis consists of five studies that tap both of these aspects of adult language learning by using magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) during language processing and language learning tasks. The thesis shows that learning novel phonological word forms, either in the native tongue or when exposed to a foreign phonology, activates the brain in similar ways. The results also show that novel native words readily become integrated in the mental lexicon. Several studies in the thesis highlight the left temporal cortex as an important brain region in learning and accessing phonological forms. Incidental learning of foreign phonological word forms was reflected in functionally distinct temporal lobe areas that, respectively, reflected short-term memory processes and more stable learning that persisted to the next day. In a study where explicitly trained items were tracked for ten months, it was found that enhanced naming-related temporal and frontal activation one week after learning was predictive of good long-term memory. The results suggest that memory maintenance is an active process that depends on mechanisms of reconsolidation, and that these process vary considerably between individuals. The thesis put special emphasis on studying language learning in the context of language production. The neural foundation of language production has been studied considerably less than that of perceptive language, especially on the sentence level. A well-known paradigm in language production studies is picture naming, also used as a clinical tool in neuropsychology. This thesis shows that accessing the meaning and phonological form of a depicted object are subserved by different neural implementations. Moreover, a comparison between action and object naming from identical images indicated that the grammatical class of the retrieved word (verb, noun) is less important than the visual content of the image. In the present thesis, the picture naming was further modified into a novel paradigm in order to probe sentence-level speech production in a newly learned miniature language. Neural activity related to grammatical processing did not differ between the novel language and the mother tongue, but stronger neural activation for the novel language was observed during the planning of the upcoming output, likely related to more demanding lexical retrieval and short-term memory. In sum, the thesis aimed at examining language learning by combining different linguistic domains, such as phonology, semantics, and grammar, in a dynamic description of language processing in the human brain.