Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Spindle burst like spike discharge oscillations in organotypic cultures of neonatal rat cortex

Early network oscillations and spindle bursts are typical patterns of spontaneous rhythmic activity in cortical networks of neonatal rodents in vivo and in vitro. The latter can also be triggered in vivo by stimulation of afferent inputs. The mechanisms underlying such oscillations undergo profound developmental changes in the first postnatal weeks. Their possible role in cortical development is postulated but not known in detail. We have studied spontaneous and evoked patterns of activity in organotypic cultures of slices from neonatal rat cortex grown on multielectrode arrays (MEAs) for extracellular single- and multi-unit recording. Episodes of spontaneous spike discharge oscillations at 7 - 25 Hz lasting for 0.6 - 3 seconds appeared in about half of these cultures spontaneously and could be triggered by electrical stimulation of few distinct electrodes. These oscillations usually covered only restricted areas of the slices. Besides oscillations, single population bursts that spread in a wavelike manner over the whole slice also appeared spontaneously and were triggered by electrical stimulation. In most but not all cultures, population bursts preceded the oscillations. Both population bursts and spike discharge oscillations required intact glutamatergic synaptic transmission since they were suppressed by the AMPA/kainate glutamate receptor antagonist CNQX. The NMDA antagonist d-APV suppressed the oscillations but not the population bursts, suggesting an involvement of NMDA receptors in the oscillations. These findings show that spindle burst like cortical rhythms are reproduced in organotypic cultures of neonatal cortex. The culture model thus allows investigating the role of such rhythms in cortical circuit formation. Supported by SNF grant No. 3100A0-107641/1.

Leader–member exchange (LMX), job autonomy, and creative work involvement

Researchers have claimed that high quality of supervisor–employee relationships (i.e., leader– member exchange; LMX) fosters creativity at work. Moreover, researchers have acknowledged that this relationship is not clear-cut but rather complex. The present study focused on the moderating role of job autonomy in the LMX-creative work involvement relationship. Longitu-dinal field survey data (N = 144) collected in a high-technology firm revealed a positive asso-ciation between LMX and creative work involvement and confirmed our assumptions on the moderating role of job autonomy. The positive relationship between LMX and creative work involvement was stronger when employees experienced greater job autonomy. Our findings point to the importance of considering the interplay of both, the leader–member relationship and job design issues for increasing employees' creative work involvement.