A triplet spike-timing–dependent plasticity model generalizes the Bienenstock–Cooper–Munro rule to higher-order spatiotemporal correlations

Synaptic strength depresses for low and potentiates for high activation of the postsynaptic neuron. This feature is a key property of the Bienenstock–Cooper–Munro (BCM) synaptic learning rule, which has been shown to maximize the selectivity of the postsynaptic neuron, and thereby offers a possible explanation for experience-dependent cortical plasticity such as orientation selectivity. However, the BCM framework is rate-based and a significant amount of recent work has shown that synaptic plasticity also depends on the precise timing of presynaptic and postsynaptic spikes. Here we consider a triplet model of spike-timing–dependent plasticity (STDP) that depends on the interactions of three precisely timed spikes. Triplet STDP has been shown to describe plasticity experiments that the classical STDP rule, based on pairs of spikes, has failed to capture. In the case of rate-based patterns, we show a tight correspondence between the triplet STDP rule and the BCM rule. We analytically demonstrate the selectivity property of the triplet STDP rule for orthogonal inputs and perform numerical simulations for nonorthogonal inputs. Moreover, in contrast to BCM, we show that triplet STDP can also induce selectivity for input patterns consisting of higher-order spatiotemporal correlations, which exist in natural stimuli and have been measured in the brain. We show that this sensitivity to higher-order correlations can be used to develop direction and speed selectivity.

Interactions between short-term and long-term plasticity: shooting for a moving target

Far from being static transmission units, synapses are highly dynamical elements that change over multiple time scales depending on the history of the neural activity of both the pre- and postsynaptic neuron. Moreover, synaptic changes on different time scales interact: long-term plasticity (LTP) can modify the properties of short-term plasticity (STP) in the same synapse. Most existing theories of synaptic plasticity focus on only one of these time scales (either STP or LTP or late-LTP) and the theoretical principles underlying their interactions are thus largely unknown. Here we develop a normative model of synaptic plasticity that combines both STP and LTP and predicts specific patterns for their interactions. Recently, it has been proposed that STP arranges for the local postsynaptic membrane potential at a synapse to behave as an optimal estimator of the presynaptic membrane potential based on the incoming spikes. Here we generalize this approach by considering an optimal estimator of a non-linear function of the membrane potential and the long-term synaptic efficacy—which itself may be subject to change on a slower time scale. We find that an increase in the long-term synaptic efficacy necessitates changes in the dynamics of STP. More precisely, for a realistic non-linear function to be estimated, our model predicts that after the induction of LTP, causing long-term synaptic efficacy to increase, a depressing synapse should become even more depressing. That is, in a protocol using trains of presynaptic stimuli, as the initial EPSP becomes stronger due to LTP, subsequent EPSPs should become weakened and this weakening should be more pronounced with LTP. This form of redistribution of synaptic efficacies agrees well with electrophysiological data on synapses connecting layer 5 pyramidal neurons.

Structural plasticity of hippocampal mossy fiber synapses as revealed by high-pressure freezing

Despite recent progress in fluorescence microscopy techniques, electron microscopy (EM) is still superior in the simultaneous analysis of all tissue components at high resolution. However, it is unclear to what extent conventional fixation for EM using aldehydes results in tissue alteration. Here we made an attempt to minimize tissue alteration by using rapid high-pressure freezing (HPF) of hippocampal slice cultures. We used this approach to monitor fine-structural changes at hippocampal mossy fiber synapses associated with chemically induced long-term potentiation (LTP). Synaptic plasticity in LTP has been known to involve structural changes at synapses including reorganization of the actin cytoskeleton and de novo formation of spines. While LTP-induced formation and growth of postsynaptic spines have been reported, little is known about associated structural changes in presynaptic boutons. Mossy fiber synapses are assumed to exhibit presynaptic LTP expression and are easily identified by EM. In slice cultures from wildtype mice, we found that chemical LTP increased the length of the presynaptic membrane of mossy fiber boutons, associated with a de novo formation of small spines and an increase in the number of active zones. Of note, these changes were not observed in slice cultures from Munc13-1 knockout mutants exhibiting defective vesicle priming. These findings show that activation of hippocampal mossy fibers induces pre- and postsynaptic structural changes at mossy fiber synapses that can be monitored by EM.

Spine Ca2+ signaling in spike-timing-dependent plasticity

Calcium is a second messenger, which can trigger the modification of synaptic efficacy. We investigated the question of whether a differential rise in postsynaptic Ca2+ ([Ca2+]i) alone is sufficient to account for the induction of long-term potentiation (LTP) and long-term depression (LTD) of EPSPs in the basal dendrites of layer 2/3 pyramidal neurons of the somatosensory cortex. Volume-averaged [Ca2+]i transients were measured in spines of the basal dendritic arbor for spike-timing-dependent plasticity induction protocols. The rise in [Ca2+]i was uncorrelated to the direction of the change in synaptic efficacy, because several pairing protocols evoked similar spine [Ca2+]i transients but resulted in either LTP or LTD. The sequence dependence of near-coincident presynaptic and postsynaptic activity on the direction of changes in synaptic strength suggested that LTP and LTD were induced by two processes, which were controlled separately by postsynaptic [Ca2+]i levels. Activation of voltage-dependent Ca2+ channels before metabotropic glutamate receptors (mGluRs) resulted in the phospholipase C-dependent (PLC-dependent) synthesis of endocannabinoids, which acted as a retrograde messenger to induce LTD. LTP required a large [Ca2+]i transient evoked by NMDA receptor activation. Blocking mGluRs abolished the induction of LTD and uncovered the Ca2+-dependent induction of LTP. We conclude that the volume-averaged peak elevation of [Ca2+]i in spines of layer 2/3 pyramids determines the magnitude of long-term changes in synaptic efficacy. The direction of the change is controlled, however, via a mGluR-coupled signaling cascade. mGluRs act in conjunction with PLC as sequence-sensitive coincidence detectors when postsynaptic precede presynaptic action potentials to induce LTD. Thus presumably two different Ca2+ sensors in spines control the induction of spike-timing-dependent synaptic plasticity.

Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).

Development of dendritic tonic GABAergic inhibition regulates excitability and plasticity in CA1 pyramidal neurons

Synaptic plasticity rules change during development: while hippocampal synapses can be potentiated by a single action potential pairing protocol in young neurons, mature neurons require burst firing to induce synaptic potentiation. An essential component for spike timing-dependent plasticity is the backpropagating action potential (BAP). BAP along the dendrites can be modulated by morphology and ion channel composition, both of which change during late postnatal development. However it is unclear whether these dendritic changes can explain the developmental changes in synaptic plasticity induction rules. Here, we show that tonic GABAergic inhibition regulates dendritic action potential backpropagation in adolescent but not pre-adolescent CA1 pyramidal neurons. These developmental changes in tonic inhibition also altered the induction threshold for spike timing-dependent plasticity in adolescent neurons. This GABAergic regulatory effect upon backpropagation is restricted to distal regions of apical dendrites (>200 μm) and mediated by α5-containing GABA(A) receptors. Direct dendritic recordings demonstrate α5-mediated tonic GABA(A) currents in adolescent neurons which can modulate backpropagating action potentials. These developmental modulations in dendritic excitability could not be explained by concurrent changes in dendritic morphology. To explain our data, model simulations propose a distally-increasing or localized distal expression of dendritic α5 tonic inhibition in mature neurons. Overall, our results demonstrate that dendritic integration and plasticity in more mature dendrites are significantly altered by tonic α5 inhibition in a dendritic region-specific and developmentally-regulated manner.

Learning Spike-Based Population Codes by Reward and Population Feedback

We investigate a recently proposed model for decision learning in a population of spiking neurons where synaptic plasticity is modulated by a population signal in addition to reward feedback. For the basic model, binary population decision making based on spike/no-spike coding, a detailed computational analysis is given about how learning performance depends on population size and task complexity. Next, we extend the basic model to n-ary decision making and show that it can also be used in conjunction with other population codes such as rate or even latency coding.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

Fine structure of hippocampal mossy fiber synapses following rapid high-pressure freezing

Synapses of hippocampal neurons play important roles in learning and memory processes and are involved in aberrant hippocampal function in temporal lobe epilepsy. Major neuronal types in the hippocampus as well as their input and output synapses are well known, but it has remained an open question to what extent conventional electron microscopy (EM) has provided us with the real appearance of synaptic fine structure under in vivo conditions. There is reason to assume that conventional aldehyde fixation and dehydration lead to protein denaturation and tissue shrinkage, likely associated with the occurrence of artifacts. However, realistic fine-structural data of synapses are required for our understanding of the transmission process and for its simulation. Here, we used high-pressure freezing and cryosubstitution of hippocampal tissue that was not subjected to aldehyde fixation and dehydration in ethanol to monitor the fine structure of an identified synapse in the hippocampal CA3 region, that is, the synapse between granule cell axons, the mossy fibers, and the proximal dendrites of CA3 pyramidal neurons. Our results showed that high-pressure freezing nicely preserved ultrastructural detail of this particular synapse and allowed us to study rapid structural changes associated with synaptic plasticity.

The time window for generation of dendritic spikes by coincidence of action potentials and EPSPs is layer specific in somatosensory cortex

The precise timing of events in the brain has consequences for intracellular processes, synaptic plasticity, integration and network behaviour. Pyramidal neurons, the most widespread excitatory neuron of the neocortex have multiple spike initiation zones, which interact via dendritic and somatic spikes actively propagating in all directions within the dendritic tree. For these neurons, therefore, both the location and timing of synaptic inputs are critical. The time window for which the backpropagating action potential can influence dendritic spike generation has been extensively studied in layer 5 neocortical pyramidal neurons of rat somatosensory cortex. Here, we re-examine this coincidence detection window for pyramidal cell types across the rat somatosensory cortex in layers 2/3, 5 and 6. We find that the time-window for optimal interaction is widest and shifted in layer 5 pyramidal neurons relative to cells in layers 6 and 2/3. Inputs arriving at the same time and locations will therefore differentially affect spike-timing dependent processes in the different classes of pyramidal neurons.

Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.