Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).

Spindle burst like spike discharge oscillations in organotypic cultures of neonatal rat cortex

Early network oscillations and spindle bursts are typical patterns of spontaneous rhythmic activity in cortical networks of neonatal rodents in vivo and in vitro. The latter can also be triggered in vivo by stimulation of afferent inputs. The mechanisms underlying such oscillations undergo profound developmental changes in the first postnatal weeks. Their possible role in cortical development is postulated but not known in detail. We have studied spontaneous and evoked patterns of activity in organotypic cultures of slices from neonatal rat cortex grown on multielectrode arrays (MEAs) for extracellular single- and multi-unit recording. Episodes of spontaneous spike discharge oscillations at 7 - 25 Hz lasting for 0.6 - 3 seconds appeared in about half of these cultures spontaneously and could be triggered by electrical stimulation of few distinct electrodes. These oscillations usually covered only restricted areas of the slices. Besides oscillations, single population bursts that spread in a wavelike manner over the whole slice also appeared spontaneously and were triggered by electrical stimulation. In most but not all cultures, population bursts preceded the oscillations. Both population bursts and spike discharge oscillations required intact glutamatergic synaptic transmission since they were suppressed by the AMPA/kainate glutamate receptor antagonist CNQX. The NMDA antagonist d-APV suppressed the oscillations but not the population bursts, suggesting an involvement of NMDA receptors in the oscillations. These findings show that spindle burst like cortical rhythms are reproduced in organotypic cultures of neonatal cortex. The culture model thus allows investigating the role of such rhythms in cortical circuit formation. Supported by SNF grant No. 3100A0-107641/1.

Coreleased orexin and glutamate evoke nonredundant spike outputs and computations in histamine neurons.

Stable wakefulness requires orexin/hypocretin neurons (OHNs) and OHR2 receptors. OHNs sense diverse environmental cues and control arousal accordingly. For unknown reasons, OHNs contain multiple excitatory transmitters, including OH peptides and glutamate. To analyze their cotransmission within computational frameworks for control, we optogenetically stimulated OHNs and examined resulting outputs (spike patterns) in a downstream arousal regulator, the histamine neurons (HANs). OHR2s were essential for sustained HAN outputs. OHR2-dependent HAN output increased linearly during constant OHN input, suggesting that the OHN→HAN(OHR2) module may function as an integral controller. OHN stimulation evoked OHR2-dependent slow postsynaptic currents, similar to midnanomolar OH concentrations. Conversely, glutamate-dependent output transiently communicated OHN input onset, peaking rapidly then decaying alongside OHN→HAN glutamate currents. Blocking glutamate-driven spiking did not affect OH-driven spiking and vice versa, suggesting isolation (low cross-modulation) of outputs. Therefore, in arousal regulators, cotransmitters may translate distinct features of OHN activity into parallel, nonredundant control signals for downstream effectors.

Decision making in beach volleyball defense: Crucial factors derived from interviews with top-level experts

Objectives: In fast ball sports like beach volleyball, decision-making skills are a determining factor for excellent performance. The current investigation aimed to identify factors that influence the decisionmaking process in top-level beach volleyball defense in order to find relevant aspects for further research. For this reason, focused interviews with top players in international beach volleyball were conducted and analyzed with respect to decision-making characteristics. Design: Nineteen world-tour beach volleyball defense players, including seven Olympic or world champions, were interviewed, focusing on decision-making factors, gaze behavior, and interactions between the two. Methods: Verbal data were analyzed by inductive content analysis according to Mayring (2008). This approach allows categories to emerge from the interview material itself instead of forcing data into preset classifications and theoretical concepts. Results: The data analysis showed that, for top-level beach volleyball defense, decision making depends on opponent specifics, external context, situational context, opponent's movements, and intuition. Information on gaze patterns and visual cues revealed general tendencies indicating optimal gaze strategies that support excellent decision making. Furthermore, the analysis highlighted interactions between gaze behavior, visual information, and domain-specific knowledge. Conclusions: The present findings provide information on visual perception, domain-specific knowledge, and interactions between the two that are relevant for decision making in top-level beach volleyball defense. The results can be used to inform sports practice and to further untangle relevant mechanisms underlying decision making in complex game situations.

Effectiveness of the Call in Beach Volleyball Attacking Play

In beach volleyball the setter has the opportunity to give her or his hitter a “call”. The call intends that the setter suggests to her or his partner where to place the attack in the opponent’s court. The effectiveness of a call is still unknown. We investigated the women’s and men’s Swiss National Beach Volleyball Championships in 2011 and analyzed 2185 attacks. We found large differences between female and male players. While men called in only 38.4% of attacks, women used calls in 85.5% of attacks. If the male players followed a given call, 63% of the attacks were successful. The success rate of attacks without any call was 55.8% and 47.6% when the call was ignored. These differences were not significant (χ2(2) = 4.55, p = 0.103). In women’s beach volleyball, the rate of successful attacks was 61.5% when a call was followed, 35% for attacks without a call, and 42.6% when a call was ignored. The differences were highly significant (χ2(2) = 23.42, p < 0.0005). Taking into account the findings of the present study, we suggested that the call was effective in women’s beach volleyball, while its effect in men’s game was unclear. Considering the quality of calls we indicate that there is a significant potential to increase the effectiveness of a call.

Perceptual Training in Beach Volleyball Defence: Different Effects of Gaze-Path Cueing on Gaze and Decision-Making

For perceptual-cognitive skill training, a variety of intervention methods has been proposed, including the so-called “color-cueing method” which aims on superior gaze-path learning by applying visual markers. However, recent findings challenge this method, especially, with regards to its actual effects on gaze behavior. Consequently, after a preparatory study on the identification of appropriate visual cues for life-size displays, a perceptual-training experiment on decision-making in beach volleyball was conducted, contrasting two cueing interventions (functional vs. dysfunctional gaze path) with a conservative control condition (anticipation-related instructions). Gaze analyses revealed learning effects for the dysfunctional group only. Regarding decision-making, all groups showed enhanced performance with largest improvements for the control group followed by the functional and the dysfunctional group. Hence, the results confirm cueing effects on gaze behavior, but they also question its benefit for enhancing decision-making. However, before completely denying the method’s value, optimisations should be checked regarding, for instance, cueing-pattern characteristics and gaze-related feedback.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.