Learning Spike-Based Population Codes by Reward and Population Feedback

We investigate a recently proposed model for decision learning in a population of spiking neurons where synaptic plasticity is modulated by a population signal in addition to reward feedback. For the basic model, binary population decision making based on spike/no-spike coding, a detailed computational analysis is given about how learning performance depends on population size and task complexity. Next, we extend the basic model to n-ary decision making and show that it can also be used in conjunction with other population codes such as rate or even latency coding.

Intermittent spike-wave dynamics in a heterogeneous, spatially extended neural mass model

Generalised epileptic seizures are frequently accompanied by sudden, reversible transitions from low amplitude, irregular background activity to high amplitude, regular spike-wave discharges (SWD) in the EEG. The underlying mechanisms responsible for SWD generation and for the apparently spontaneous transitions to SWD and back again are still not fully understood. Specifically, the role of spatial cortico-cortical interactions in ictogenesis is not well studied. We present a macroscopic, neural mass model of a cortical column which includes two distinct time scales of inhibition. This model can produce both an oscillatory background and a pathological SWD rhythm. We demonstrate that coupling two of these cortical columns can lead to a bistability between out-of-phase, low amplitude background dynamics and in-phase, high amplitude SWD activity. Stimuli can cause state-dependent transitions from background into SWD. In an extended local area of cortex, spatial heterogeneities in a model parameter can lead to spontaneous reversible transitions from a desynchronised background to synchronous SWD due to intermittency. The deterministic model is therefore capable of producing absence seizure-like events without any time dependent adjustment of model parameters. The emergence of such mechanisms due to spatial coupling demonstrates the importance of spatial interactions in modelling ictal dynamics, and in the study of ictogenesis.

A triplet spike-timing–dependent plasticity model generalizes the Bienenstock–Cooper–Munro rule to higher-order spatiotemporal correlations

Synaptic strength depresses for low and potentiates for high activation of the postsynaptic neuron. This feature is a key property of the Bienenstock–Cooper–Munro (BCM) synaptic learning rule, which has been shown to maximize the selectivity of the postsynaptic neuron, and thereby offers a possible explanation for experience-dependent cortical plasticity such as orientation selectivity. However, the BCM framework is rate-based and a significant amount of recent work has shown that synaptic plasticity also depends on the precise timing of presynaptic and postsynaptic spikes. Here we consider a triplet model of spike-timing–dependent plasticity (STDP) that depends on the interactions of three precisely timed spikes. Triplet STDP has been shown to describe plasticity experiments that the classical STDP rule, based on pairs of spikes, has failed to capture. In the case of rate-based patterns, we show a tight correspondence between the triplet STDP rule and the BCM rule. We analytically demonstrate the selectivity property of the triplet STDP rule for orthogonal inputs and perform numerical simulations for nonorthogonal inputs. Moreover, in contrast to BCM, we show that triplet STDP can also induce selectivity for input patterns consisting of higher-order spatiotemporal correlations, which exist in natural stimuli and have been measured in the brain. We show that this sensitivity to higher-order correlations can be used to develop direction and speed selectivity.

Synaptic regulation of spike firing in interneurons of the striatum in vivo

The striatum, the major input nucleus of the basal ganglia, is numerically dominated by a single class of principal neurons, the GABAergic spiny projection neuron (SPN) that has been extensively studied both in vitro and in vivo. Much less is known about the sparsely distributed interneurons, principally the cholinergic interneuron (CIN) and the GABAergic fast-spiking interneuron (FSI). Here, we summarize results from two recent studies on these interneurons where we used in vivo intracellular recording techniques in urethane-anaesthetized rats (Schulz et al., J Neurosci 31[31], 2011; J Physiol, in press). Interneurons were identified by their characteristic responses to intracellular current steps and spike waveforms. Spontaneous spiking contained a high proportion (~45%) of short inter-spike intervals (ISI) of <30 ms in FSIs, but virtually none in CINs. Spiking patterns in CINs covered a broad spectrum ranging from regular tonic spiking to phasic activity despite very similar unimodal membrane potential distributions across neurons. In general, phasic spiking activity occurred in phase with the slow ECoG waves, whereas CINs exhibiting tonic regular spiking were little affected by afferent network activity. In contrast, FSIs exhibited transitions between Down and Up states very similar to SPNs. Compared to SPNs, the FSI Up state membrane potential was noisier and power spectra exhibited significantly larger power at frequencies in the gamma range (55-95 Hz). Cortical-evoked inputs had faster dynamics in FSIs than SPNs and the membrane potential preceding spontaneous spike discharge exhibited short and steep trajectories, suggesting that fast input components controlled spike output in FSIs. Intrinsic resonance mechanisms may have further enhanced the sensitivity of FSIs to fast oscillatory inputs. Induction of an activated ECoG state by local ejection of bicuculline into the superior colliculus, resulted in increased spike frequency in both interneuron classes without changing the overall distribution of ISIs. This manipulation also made CINs responsive to a light flashed into the contralateral eye. Typically, the response consisted of an excitation at short latency followed by a pause in spike firing, via an underlying depolarization-hyperpolarization membrane sequence. These results highlight the differential sensitivity of striatal interneurons to afferent synaptic signals and support a model where CINs modulate the striatal network in response to salient sensory bottom-up signals, while FSIs serve gating of top-down signals from the cortex during action selection and reward-related learning.

Spike-triggered reaction-time EEG as a possible assessment tool for driving ability

The impact of interictal epileptic activity (IEA) on driving is a rarely investigated issue. We analyzed the impact of IEA on reaction time in a pilot study. Reactions to simple visual stimuli (light flash) in the Flash test or complex visual stimuli (obstacle on a road) in a modified car driving computer game, the Steer Clear, were measured during IEA bursts and unremarkable electroencephalography (EEG) periods. Individual epilepsy patients showed slower reaction times (RTs) during generalized IEA compared to RTs during unremarkable EEG periods. RT differences were approximately 300 ms (p < 0.001) in the Flash test and approximately 200 ms (p < 0.001) in the Steer Clear. Prior work suggested that RT differences >100 ms may become clinically relevant. This occurred in 40% of patients in the Flash test and in up to 50% in the Steer Clear. When RT were pooled, mean RT differences were 157 ms in the Flash test (p < 0.0001) and 116 ms in the Steer Clear (p < 0.0001). Generalized IEA of short duration seems to impair brain function, that is, the ability to react. The reaction-time EEG could be used routinely to assess driving ability.

Spike-based decision learning of nash equilibria in two-player games

Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.