Spatio-temporal credit assignment in population learning

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

Researchers' roles in knowledge co-production: experience from sustainability research in Kenya, Switzerland, Bolivia and Nepal

Co-production of knowledge between academic and non-academic communities is a prerequisite for research aiming at more sustainable development paths. Sustainability researchers face three challenges in such co-production: (a) addressing power relations; (b) interrelating different perspectives on the issues at stake; and (c) promoting a previously negotiated orientation towards sustainable development. A systematic comparison of four sustainability research projects in Kenya (vulnerability to drought), Switzerland (soil protection), Bolivia and Nepal (conservation vs. development) shows how the researchers intuitively adopted three different roles to face these challenges: the roles of reflective scientist, intermediary, and facilitator of a joint learning process. From this systematized and iterative self-reflection on the roles that a researcher can assume in the indeterminate social space where knowledge is co-produced, we draw conclusions regarding training.

Spike-based decision learning of nash equilibria in two-player games

Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.

Social Learning Processes in Swiss Soil Protection? The 'From Farmer - To Farmer' Project

Social learning approaches have become a prominent focus in studies related to sustainable agriculture. In order to better understand the potential of social learning for more sustainable development, the present study assessed the processes, effects and facilitating elements of interaction related to social learning in the context of Swiss soil protection and the innovative ‘From Farmer - To Farmer’ project. The study reveals that social learning contributes to fundamental transformations of patterns of interactions. However, the study also demonstrates that a learning-oriented understanding of sustainable development implies including analysis of the institutional environments in which the organizations of the individual representatives of face-to-face-based social learning processes are operating. This has shown to be a decisive element when face-to-face-based learning processes of the organisations’ representatives are translated into organisational learning. Moreover, the study revealed that this was achieved not directly through formalisation of new lines of institutionalised cooperation but by establishing links in a ‘boundary space’ trying out new forms of collaboration, aiming at social learning and co-production of knowledge. It is argued that further research on social learning processes should give greater emphasis to this intermediary level of ‘boundary spaces’.

Envisioning sustainable water futures in a transdisciplinary learning process: combining normative, explorative, and participatory scenario approaches

Competing water demands for household consumption as well as the production of food, energy, and other uses pose challenges for water supply and sustainable development in many parts of the world. Designing creative strategies and learning processes for sustainable water governance is thus of prime importance. While this need is uncontested, suitable approaches still have to be found. In this article we present and evaluate a conceptual approach to scenario building aimed at transdisciplinary learning for sustainable water governance. The approach combines normative, explorative, and participatory scenario elements. This combination allows for adequate consideration of stakeholders’ and scientists’ systems, target, and transformation knowledge. Application of the approach in the MontanAqua project in the Swiss Alps confirmed its high potential for co-producing new knowledge and establishing a meaningful and deliberative dialogue between all actors involved. The iterative and combined approach ensured that stakeholders’ knowledge was adequately captured, fed into scientific analysis, and brought back to stakeholders in several cycles, thereby facilitating learning and co-production of new knowledge relevant for both stakeholders and scientists. However, the approach also revealed a number of constraints, including the enormous flexibility required of stakeholders and scientists in order for them to truly engage in the co-production of new knowledge. Overall, the study showed that shifts from strategic to communicative action are possible in an environment of mutual trust. This ultimately depends on creating conditions of interaction that place scientists’ and stakeholders’ knowledge on an equal footing.

Effects of Nosema apis, N. ceranae, and coinfections on honey bee (Apis mellifera) learning and memory

Western honey bees (Apis mellifera) face an increasing number of challenges that in recent years have led to significant economic effects on apiculture, with attendant consequences for agriculture. Nosemosis is a fungal infection of honey bees caused by either Nosema apis or N. ceranae. The putative greater virulence of N. ceranae has spurred interest in understanding how it differs from N. apis. Little is known of effects of N. apis or N. ceranae on honey bee learning and memory. Following a Pavlovian model that relies on the proboscis extension reflex, we compared acquisition learning and long-term memory recall of uninfected (control) honey bees versus those inoculated with N. apis, N. ceranae, or both. We also tested whether spore intensity was associated with variation in learning and memory. Neither learning nor memory differed among treatments. There was no evidence of a relationship between spore intensity and learning, and only limited evidence of a negative effect on memory; this occurred only in the co-inoculation treatment. Our results suggest that if Nosema spp. are contributing to unusually high colony losses in recent years, the mechanism by which they may affect honey bees is probably not related to effects on learning or memory, at least as assessed by the proboscis extension reflex.