Learning Spike-Based Population Codes by Reward and Population Feedback

We investigate a recently proposed model for decision learning in a population of spiking neurons where synaptic plasticity is modulated by a population signal in addition to reward feedback. For the basic model, binary population decision making based on spike/no-spike coding, a detailed computational analysis is given about how learning performance depends on population size and task complexity. Next, we extend the basic model to n-ary decision making and show that it can also be used in conjunction with other population codes such as rate or even latency coding.

Intermittent spike-wave dynamics in a heterogeneous, spatially extended neural mass model

Generalised epileptic seizures are frequently accompanied by sudden, reversible transitions from low amplitude, irregular background activity to high amplitude, regular spike-wave discharges (SWD) in the EEG. The underlying mechanisms responsible for SWD generation and for the apparently spontaneous transitions to SWD and back again are still not fully understood. Specifically, the role of spatial cortico-cortical interactions in ictogenesis is not well studied. We present a macroscopic, neural mass model of a cortical column which includes two distinct time scales of inhibition. This model can produce both an oscillatory background and a pathological SWD rhythm. We demonstrate that coupling two of these cortical columns can lead to a bistability between out-of-phase, low amplitude background dynamics and in-phase, high amplitude SWD activity. Stimuli can cause state-dependent transitions from background into SWD. In an extended local area of cortex, spatial heterogeneities in a model parameter can lead to spontaneous reversible transitions from a desynchronised background to synchronous SWD due to intermittency. The deterministic model is therefore capable of producing absence seizure-like events without any time dependent adjustment of model parameters. The emergence of such mechanisms due to spatial coupling demonstrates the importance of spatial interactions in modelling ictal dynamics, and in the study of ictogenesis.

A triplet spike-timing–dependent plasticity model generalizes the Bienenstock–Cooper–Munro rule to higher-order spatiotemporal correlations

Synaptic strength depresses for low and potentiates for high activation of the postsynaptic neuron. This feature is a key property of the Bienenstock–Cooper–Munro (BCM) synaptic learning rule, which has been shown to maximize the selectivity of the postsynaptic neuron, and thereby offers a possible explanation for experience-dependent cortical plasticity such as orientation selectivity. However, the BCM framework is rate-based and a significant amount of recent work has shown that synaptic plasticity also depends on the precise timing of presynaptic and postsynaptic spikes. Here we consider a triplet model of spike-timing–dependent plasticity (STDP) that depends on the interactions of three precisely timed spikes. Triplet STDP has been shown to describe plasticity experiments that the classical STDP rule, based on pairs of spikes, has failed to capture. In the case of rate-based patterns, we show a tight correspondence between the triplet STDP rule and the BCM rule. We analytically demonstrate the selectivity property of the triplet STDP rule for orthogonal inputs and perform numerical simulations for nonorthogonal inputs. Moreover, in contrast to BCM, we show that triplet STDP can also induce selectivity for input patterns consisting of higher-order spatiotemporal correlations, which exist in natural stimuli and have been measured in the brain. We show that this sensitivity to higher-order correlations can be used to develop direction and speed selectivity.

Synaptic regulation of spike firing in interneurons of the striatum in vivo

The striatum, the major input nucleus of the basal ganglia, is numerically dominated by a single class of principal neurons, the GABAergic spiny projection neuron (SPN) that has been extensively studied both in vitro and in vivo. Much less is known about the sparsely distributed interneurons, principally the cholinergic interneuron (CIN) and the GABAergic fast-spiking interneuron (FSI). Here, we summarize results from two recent studies on these interneurons where we used in vivo intracellular recording techniques in urethane-anaesthetized rats (Schulz et al., J Neurosci 31[31], 2011; J Physiol, in press). Interneurons were identified by their characteristic responses to intracellular current steps and spike waveforms. Spontaneous spiking contained a high proportion (~45%) of short inter-spike intervals (ISI) of <30 ms in FSIs, but virtually none in CINs. Spiking patterns in CINs covered a broad spectrum ranging from regular tonic spiking to phasic activity despite very similar unimodal membrane potential distributions across neurons. In general, phasic spiking activity occurred in phase with the slow ECoG waves, whereas CINs exhibiting tonic regular spiking were little affected by afferent network activity. In contrast, FSIs exhibited transitions between Down and Up states very similar to SPNs. Compared to SPNs, the FSI Up state membrane potential was noisier and power spectra exhibited significantly larger power at frequencies in the gamma range (55-95 Hz). Cortical-evoked inputs had faster dynamics in FSIs than SPNs and the membrane potential preceding spontaneous spike discharge exhibited short and steep trajectories, suggesting that fast input components controlled spike output in FSIs. Intrinsic resonance mechanisms may have further enhanced the sensitivity of FSIs to fast oscillatory inputs. Induction of an activated ECoG state by local ejection of bicuculline into the superior colliculus, resulted in increased spike frequency in both interneuron classes without changing the overall distribution of ISIs. This manipulation also made CINs responsive to a light flashed into the contralateral eye. Typically, the response consisted of an excitation at short latency followed by a pause in spike firing, via an underlying depolarization-hyperpolarization membrane sequence. These results highlight the differential sensitivity of striatal interneurons to afferent synaptic signals and support a model where CINs modulate the striatal network in response to salient sensory bottom-up signals, while FSIs serve gating of top-down signals from the cortex during action selection and reward-related learning.

Spike-triggered reaction-time EEG as a possible assessment tool for driving ability

The impact of interictal epileptic activity (IEA) on driving is a rarely investigated issue. We analyzed the impact of IEA on reaction time in a pilot study. Reactions to simple visual stimuli (light flash) in the Flash test or complex visual stimuli (obstacle on a road) in a modified car driving computer game, the Steer Clear, were measured during IEA bursts and unremarkable electroencephalography (EEG) periods. Individual epilepsy patients showed slower reaction times (RTs) during generalized IEA compared to RTs during unremarkable EEG periods. RT differences were approximately 300 ms (p < 0.001) in the Flash test and approximately 200 ms (p < 0.001) in the Steer Clear. Prior work suggested that RT differences >100 ms may become clinically relevant. This occurred in 40% of patients in the Flash test and in up to 50% in the Steer Clear. When RT were pooled, mean RT differences were 157 ms in the Flash test (p < 0.0001) and 116 ms in the Steer Clear (p < 0.0001). Generalized IEA of short duration seems to impair brain function, that is, the ability to react. The reaction-time EEG could be used routinely to assess driving ability.

Spike-based decision learning of nash equilibria in two-player games

Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.

Predicting spike timing of neocortical pyramidal neurons by simple threshold models

Neurons generate spikes reliably with millisecond precision if driven by a fluctuating current--is it then possible to predict the spike timing knowing the input? We determined parameters of an adapting threshold model using data recorded in vitro from 24 layer 5 pyramidal neurons from rat somatosensory cortex, stimulated intracellularly by a fluctuating current simulating synaptic bombardment in vivo. The model generates output spikes whenever the membrane voltage (a filtered version of the input current) reaches a dynamic threshold. We find that for input currents with large fluctuation amplitude, up to 75% of the spike times can be predicted with a precision of +/-2 ms. Some of the intrinsic neuronal unreliability can be accounted for by a noisy threshold mechanism. Our results suggest that, under random current injection into the soma, (i) neuronal behavior in the subthreshold regime can be well approximated by a simple linear filter; and (ii) most of the nonlinearities are captured by a simple threshold process.

Spine Ca2+ signaling in spike-timing-dependent plasticity

Calcium is a second messenger, which can trigger the modification of synaptic efficacy. We investigated the question of whether a differential rise in postsynaptic Ca2+ ([Ca2+]i) alone is sufficient to account for the induction of long-term potentiation (LTP) and long-term depression (LTD) of EPSPs in the basal dendrites of layer 2/3 pyramidal neurons of the somatosensory cortex. Volume-averaged [Ca2+]i transients were measured in spines of the basal dendritic arbor for spike-timing-dependent plasticity induction protocols. The rise in [Ca2+]i was uncorrelated to the direction of the change in synaptic efficacy, because several pairing protocols evoked similar spine [Ca2+]i transients but resulted in either LTP or LTD. The sequence dependence of near-coincident presynaptic and postsynaptic activity on the direction of changes in synaptic strength suggested that LTP and LTD were induced by two processes, which were controlled separately by postsynaptic [Ca2+]i levels. Activation of voltage-dependent Ca2+ channels before metabotropic glutamate receptors (mGluRs) resulted in the phospholipase C-dependent (PLC-dependent) synthesis of endocannabinoids, which acted as a retrograde messenger to induce LTD. LTP required a large [Ca2+]i transient evoked by NMDA receptor activation. Blocking mGluRs abolished the induction of LTD and uncovered the Ca2+-dependent induction of LTP. We conclude that the volume-averaged peak elevation of [Ca2+]i in spines of layer 2/3 pyramids determines the magnitude of long-term changes in synaptic efficacy. The direction of the change is controlled, however, via a mGluR-coupled signaling cascade. mGluRs act in conjunction with PLC as sequence-sensitive coincidence detectors when postsynaptic precede presynaptic action potentials to induce LTD. Thus presumably two different Ca2+ sensors in spines control the induction of spike-timing-dependent synaptic plasticity.

Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).