Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Reshaping Rural Extension: Learning for Sustainability (LforS): An Integrative and Learning-Based Advisory Approach for Rural Extension with Small-Scale Farmers

This publication offers concrete suggestions for implementing an integrative and learning-oriented approach to agricultural extension with the goal of fostering sustainable development. It targets governmental and non-governmental organisations, development agencies, and extension staff working in the field of rural development. The book looks into the conditions and trends that influence extension today, and outlines new challenges and necessary adaptations. It offers a basic reflection on the goals, the criteria for success and the form of a state-of-the-art approach to extension. The core of the book consists of a presentation of Learning for Sustainability (LforS), an example of an integrative, learning-oriented approach that is based on three crucial elements: stakeholder dialogue, knowledge management, and organizational development. Awareness raising and capacity building, social mobilization, and monitoring & evaluation are additional building blocks. The structure and organisation of the LforS approach as well as a selection of appropriate methods and tools are presented. The authors also address key aspects of developing and managing a learning-oriented extension approach. The book illustrates how LforS can be implemented by presenting two case studies, one from Madagascar and one from Mongolia. It addresses conceptual questions and at the same time it is practice-oriented. In contrast to other extension approaches, LforS does not limit its focus to production-related aspects and the development of value chains: it also addresses livelihood issues in a broad sense. With its focus on learning processes LforS seeks to create a better understanding of the links between different spheres and different levels of decision-making; it also seeks to foster integration of the different actors’ perspectives.

Perceptual Learning via Modification of Cortical Top-Down Signals

The primary visual cortex (V1) is pre-wired to facilitate the extraction of behaviorally important visual features. Collinear edge detectors in V1, for instance, mutually enhance each other to improve the perception of lines against a noisy background. The same pre-wiring that facilitates line extraction, however, is detrimental when subjects have to discriminate the brightness of different line segments. How is it possible to improve in one task by unsupervised practicing, without getting worse in the other task? The classical view of perceptual learning is that practicing modulates the feedforward input stream through synaptic modifications onto or within V1. However, any rewiring of V1 would deteriorate other perceptual abilities different from the trained one. We propose a general neuronal model showing that perceptual learning can modulate top-down input to V1 in a task-specific way while feedforward and lateral pathways remain intact. Consistent with biological data, the model explains how context-dependent brightness discrimination is improved by a top-down recruitment of recurrent inhibition and a top-down induced increase of the neuronal gain within V1. Both the top-down modulation of inhibition and of neuronal gain are suggested to be universal features of cortical microcircuits which enable perceptual learning.

Interference during the implicit learning of two different motor sequences

It has been demonstrated that learning a second motor task after having learned a first task may interfere with the long-term consolidation of the first task. However, little is known about immediate changes in the representation of the motor memory in the early acquisition phase within the first minutes of the learning process. Therefore, we investigated such early interference effects with an implicit serial reaction time task in 55 healthy subjects. Each subject performed either a sequence learning task involving two different sequences, or a random control task. The results showed that learning the first sequence led to only a slight, short-lived interference effect in the early acquisition phase of the second sequence. Overall, learning of neither sequence was impaired. Furthermore, the two processes, sequence-unrelated task learning (i.e. general motor training) and the sequence learning itself did not appear to interfere with each other. In conclusion, although the long-term consolidation of a motor memory has been shown to be sensitive to other interfering memories, the present study suggests that the brain is initially able to acquire more than one new motor sequence within a short space of time without significant interference.

Evaluating retraining rules for semi-supervised learning in neural network based cursive word recognition

Training a system to recognize handwritten words is a task that requires a large amount of data with their correct transcription. However, the creation of such a training set, including the generation of the ground truth, is tedious and costly. One way of reducing the high cost of labeled training data acquisition is to exploit unlabeled data, which can be gathered easily. Making use of both labeled and unlabeled data is known as semi-supervised learning. One of the most general versions of semi-supervised learning is self-training, where a recognizer iteratively retrains itself on its own output on new, unlabeled data. In this paper we propose to apply semi-supervised learning, and in particular self-training, to the problem of cursive, handwritten word recognition. The special focus of the paper is on retraining rules that define what data are actually being used in the retraining phase. In a series of experiments it is shown that the performance of a neural network based recognizer can be significantly improved through the use of unlabeled data and self-training if appropriate retraining rules are applied.

When language of instruction and language of application differ: Cognitive costs of bilingual mathematics learning

Bilingual education programs implicitly assume that the acquired knowledge is represented in a language-independent way. This assumption, however, stands in strong contrast to research findings showing that information may be represented in a way closely tied to the specific language of instruction and learning. The present study aims to examine whether and to which extent cognitive costs appear during arithmetic learning when language of instruction and language of retrieving differ. Thirty-nine high school students participating in a bilingual education program underwent a four-day training on multiplication and subtraction problems in one language (German or French), followed by a test session in which they had to solve trained as well as untrained problems in both languages. We found that cognitive costs related to language switching appeared for both arithmetic operations. Implications of our findings are discussed with respect to bilingual education as well as to cognitive mechanisms underlying different arithmetic operations.

Social skills as predictors of communication, performance and quality of collaboration in project-based learning

The assumption that social skills are necessary ingredients of collaborative learning is well established but rarely empirically tested. In addition, most theories on collaborative learning focus on social skills only at the personal level, while the social skill configurations within a learning group might be of equal importance. Using the integrative framework, this study investigates which social skills at the personal level and at the group level are predictive of task-related e-mail communication, satisfaction with performance and perceived quality of collaboration. Data collection took place in a technology-enhanced long-term project-based learning setting for pre-service teachers. For data collection, two questionnaires were used, one at the beginning and one at the end of the learning cycle which lasted 3 months. During the project phase, the e-mail communication between group members was captured as well. The investigation of 60 project groups (N = 155 for the questionnaires; group size: two or three students) and 33 groups for the e-mail communication (N = 83) revealed that personal social skills played only a minor role compared to group level configurations of social skills in predicting satisfaction with performance, perceived quality of collaboration and communication behaviour. Members from groups that showed a high and/or homogeneous configuration of specific social skills (e.g., cooperation/compromising, leadership) usually were more satisfied and saw their group as more efficient than members from groups with a low and/or heterogeneous configuration of skills.

Early adolescents' enjoyment experienced in learning situations and its relation to student achievement

While many studies confirm that positive emotions, including enjoyment, lead to better student achievement, less empirical evidence exists about possible mediator variables that link achievement to enjoyment. It is proposed that achievement and enjoyment form a circular dependency; enjoyment in learning leads to higher achievement but a degree of achievement is required to enjoy learning. This study provides insight into the reverse of the much studied enjoyment to achievement link and provides practical recommendations on how to use these findings. Founded in Control-value theory, which suggests that control and value cognitions are important variables that mediate the connection between enjoyment and achievement, this study explores the reciprocal achievement-cognition-enjoyment link. The reciprocal link was investigated by applying a one year longitudinal design to students of grade 6 and 7 (N = 356). This age group was chosen because early adolescence represents a critical period during which a strong decrease in positive learning emotions is observed. Part of the work involved identifying factors that might be responsible for this negative development. Results of cross-lagged path analysis identified reciprocal effects between student achievement and enjoyment with control and value cognitions functioning as partial mediators. High achievement goes with high control and value cognitions, which in turn positively affect enjoyment. However, cross-lagged correlations could only be partly confirmed. The results are discussed in terms of theoretical and practical implications.

DAT1 polymorphism determines L-DOPA effects on learning about others' prosociality.

Despite that a wealth of evidence links striatal dopamine to individualś reward learning performance in non-social environments, the neurochemical underpinnings of such learning during social interaction are unknown. Here, we show that the administration of 300 mg of the dopamine precursor L-DOPA to 200 healthy male subjects influences learning about a partners' prosocial preferences in a novel social interaction task, which is akin to a repeated trust game. We found learning to be modulated by a well-established genetic marker of striatal dopamine levels, the 40-bp variable number tandem repeats polymorphism of the dopamine transporter (DAT1 polymorphism). In particular, we found that L-DOPA improves learning in 10/10R genoype subjects, who are assumed to have lower endogenous striatal dopamine levels and impairs learning in 9/10R genotype subjects, who are assumed to have higher endogenous dopamine levels. These findings provide first evidence for a critical role of dopamine in learning whether an interaction partner has a prosocial or a selfish personality. The applied pharmacogenetic approach may open doors to new ways of studying psychiatric disorders such as psychosis, which is characterized by distorted perceptions of others' prosocial attitudes.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

OBJECTIVES The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace-based assessment. In this study, we aimed to analyse how often trainer–student pairs identified corresponding LNs in mini-clinical evaluation exercise (mini-CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini-CEX was conducted. METHODS Retrospective analyses of adapted mini-CEX forms (trainers’ and students’ assessments) completed by all Year 4 medical students during clerkships were performed. Learning needs were defined by the lowest score(s) assigned to one or more of the mini-CEX domains. Learning goals were categorised qualitatively according to their correspondence with the six mini-CEX domains (e.g. history taking, professionalism). Following descriptive analyses of LNs and LGs, multi-level logistic regression models were used to predict LGs by identified LNs and social context variables. RESULTS A total of 512 trainers and 165 students conducted 1783 mini-CEXs (98% completion rate). Concordantly, trainer–student pairs most often identified LNs in the domains of ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation/efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710). Of the 446 LGs noted in total, the most frequently noted were ‘physical examination’ (49%) and ‘history taking’ (21%). Corresponding LNs as well as social context factors (e.g. clinic size) were found to be predictors of these LGs. CONCLUSIONS Although trainer–student pairs often agreed in the LNs they identified, many assessments did not result in aligned LGs. The sparseness of LGs, their dependency on social context and their partial non-alignment with students’ LNs raise questions about how the full potential of the mini-CEX as not only a ‘diagnostic’ but also an ‘educational’ tool can be exploited.

Alignment between learning needs and learning goals of Mini-CEX in clerkships

Background: Defining learning goals (LG) in alignment with learning needs (LN) is one of the key purposes of formative workplace-based assessment, but studies about this topic are scarce. Summary of Work: We analysed quantitatively and qualitatively how often trainer-student pairs identified the same LN during Mini Clinical Evaluation Exercises (Mini-CEX) in clerkships and to what degree those LNs were in line with the recorded LGs. Multilevel logistic regression models were used to predict LGs by identified LNs, controlling for context variables. Summary of Results: 512 trainers and 165 students conducted 1783 Mini-CEX (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation / efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710), with a total of 446 LGs. Of these, the most frequent LGs were ‘physical examination’ (49% of 446 LGs) and ‘history taking’ (21%); corresponding LNs as well as context variables (e.g. clinic size) were found to be predictors of these LGs. Discussion and Conclusions: Although trainer-student pairs often agreed in their identified LNs, many assessments did not result in an aligned LG or a LG at all. Interventions are needed to enhance the proportion of (aligned) LGs in Mini-CEX in order to tap into its full potential not only as a ‘diagnostic’ but also as an ‘educational tool’. Take-home messages: The sparseness of LGs, their dependency on context variables and their partial non-alignment with students’ LNs raise the question of how the effectiveness of Mini-CEX can be further enhanced.

Early adolescents’ enjoyment experienced in learning situations at school and its relation to student achievement

While many studies confirm that positive emotions, including enjoyment, lead to better student achievement, less empirical evidence exists about possible mediator variables that link achievement to enjoyment. It is proposed that achievement and enjoyment form a circular dependency; enjoyment in learning leads to higher achievement but a degree of achievement is required to enjoy learning. This study provides insight into the reverse of the much studied enjoyment to achievement link and provides practical recommendations on how to use these findings. Founded in Control-value theory, which suggests that control and value cognitions are important variables that mediate the connection between enjoyment and achievement, this study explores the reciprocal achievement-cognition-enjoyment link. The reciprocal link was investigated by applying a one year longitudinal design to students of grade 6 and 7 (N = 356). This age group was chosen because early adolescence represents a critical period during which a strong decrease in positive learning emotions is observed. Part of the work involved identifying factors that might be responsible for this negative development. Results of cross-lagged path analysis identified reciprocal effects between student achievement and enjoyment with control and value cognitions functioning as partial mediators. High achievement goes with high control and value cognitions, which in turn positively affect enjoyment. However, cross-lagged correlations could only be partly confirmed. The results are discussed in terms of theoretical and practical implications

CONMICOM – conflict mitigation in communities: LforS learning and simulation game

Three extended families live around a lake. One family are rice farmers, the second family are vegetable farmers, and the third are a family of livestock herders. All of them depend on the use of lake water for their production, and all of them need large quantities of water. All are dependent on the use of the lake water to secure their livelihood. In the game, the families are represented by their councils of elders. Each of the councils has to find means and ways to increase production in order to keep up with the growth of its family and their demands. This puts more and more pressure on the water resources, increasing the risk of overuse. Conflicts over water are about to emerge between the families. Each council of elders must try to pursue its families interests, while at the same time preventing excessive pressure on the water resources. Once a council of elders is no longer able to meet the needs of its family, it is excluded from the game. Will the parties cooperate or compete? To face the challenge of balancing economic well-being, sustainable resource management, and individual and collective interests, the three parties have a set of options for action at hand. These include power play to safeguard their own interests, communication and cooperation to negotiate with neighbours, and searching for alternatives to reduce pressure on existing water resources. During the game the players can experience how tensions may arise, increase and finally escalate. They realise what impact power play has and how alliances form, and the importance of trust-building measures, consensus and cooperation. From the insights gained, important conflict prevention and mitigation measures are derived in a debriefing session. The game is facilitated by a moderator, and lasts for 3-4 hours. Aim of the game: Each family pursues the objective of serving its own interests and securing its position through appropriate strategies and skilful negotiation, while at the same time optimising use of the water resources in a way that prevents their degradation. The end of the game is open. While the game may end by one or two families dropping out because they can no longer secure their subsistence, it is also possible that the three families succeed in creating a situation that allows them to meet their own needs as well as the requirements for sustainable water use in the long term. Learning objectives The game demonstrates how tension builds up, increases, and finally escalates; it shows how power positions work and alliances are formed; and it enables the players to experience the great significance of mutual agreement and cooperation. During the game and particularly during the debriefing and evaluation session it is important to link experiences made during the game to the players’ real-life experiences, and to discuss these links in the group. The resulting insights will provide a basis for deducing important conflict prevention and transformation measures.