Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Large deletions in the CFTR gene: clinics and genetics in Swiss patients with CF

Cystic fibrosis (CF) is the most common life-shortening autosomal recessive disorder in Caucasians, and is associated with at least one mutation on each CF transmembrane conductance regulator (CFTR) allele. Some patients, however, with only one identifiable point mutation carry on the other allele, a large deletion that is not detected by conventional screening methods. The overall frequency of large deletions in patients with CF is estimated to be 1-3%. Using the CFTR Multiplex Ligation dependent Probe Amplification Kit (MRC-Holland, Amsterdam, Netherlands) that allows the exact detection of copy numbers from all 27 exons in the CFTR gene, we screened 50 patients with only one identified mutation for large deletions in the CFTR gene. Each detected deletion was confirmed using our real-time polymerase chain reaction (PCR) assay and deletion-specific PCR reactions using junction fragment primers. We detected large deletions in eight patients (16%). These eight CF alleles belong to four different deletion types (CFTRindel2, CFTRdele14b-17b, CFTRdele17a-17b and CFTRdele 2-9) whereof the last is novel. Comparing detailed clinical data of all these patients with CF and the molecular genetic findings, we were able to elaborate criteria for deletion screenings and possible genotype-phenotype associations. In conclusion, we agree with other authors that deletion screenings should be implemented in routine genetic diagnostics of CF.

Adapting public agricultural extension services to climate change: Insights from Kenya

This paper analyses the adaptiveness of the Public Agricultural Extension Services (PAES) to climate change. Existing literature, interviews and group discussions among PAES actors in larger Makueni district, Kenya, provided the data for the analyses. The findings show that the PAES already have various elements of adaptiveness in its policies, approaches and methods of extension provision. However, the hierarchical structure of the PAES does not augur well for self-organisation at local levels of extension provision, especially under conditions of abrupt change which climate change might trigger. Most importantly, adpativeness presupposes adaptive capacity but the lack of resources in terms of funding for extension, limited mobility of extension officers, the low extension staff/farmer ratio, the aging of extension staff and significant dependence on donor funding limits the adaptiveness of the PAES. Accordingly criteria and indicators were identified in literature with which an initial assessement of the adaptiiveneess of PAES was conducted. However this assessment framework needs to be improved and future steps will integrate more specific inputs from actors in PAES in order to make the framework operational.