Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

Interactions between short-term and long-term plasticity: shooting for a moving target

Far from being static transmission units, synapses are highly dynamical elements that change over multiple time scales depending on the history of the neural activity of both the pre- and postsynaptic neuron. Moreover, synaptic changes on different time scales interact: long-term plasticity (LTP) can modify the properties of short-term plasticity (STP) in the same synapse. Most existing theories of synaptic plasticity focus on only one of these time scales (either STP or LTP or late-LTP) and the theoretical principles underlying their interactions are thus largely unknown. Here we develop a normative model of synaptic plasticity that combines both STP and LTP and predicts specific patterns for their interactions. Recently, it has been proposed that STP arranges for the local postsynaptic membrane potential at a synapse to behave as an optimal estimator of the presynaptic membrane potential based on the incoming spikes. Here we generalize this approach by considering an optimal estimator of a non-linear function of the membrane potential and the long-term synaptic efficacy—which itself may be subject to change on a slower time scale. We find that an increase in the long-term synaptic efficacy necessitates changes in the dynamics of STP. More precisely, for a realistic non-linear function to be estimated, our model predicts that after the induction of LTP, causing long-term synaptic efficacy to increase, a depressing synapse should become even more depressing. That is, in a protocol using trains of presynaptic stimuli, as the initial EPSP becomes stronger due to LTP, subsequent EPSPs should become weakened and this weakening should be more pronounced with LTP. This form of redistribution of synaptic efficacies agrees well with electrophysiological data on synapses connecting layer 5 pyramidal neurons.

LGI2 truncation causes a remitting focal epilepsy in dogs

One quadrillion synapses are laid in the first two years of postnatal construction of the human brain, which are then pruned until age 10 to 500 trillion synapses composing the final network. Genetic epilepsies are the most common neurological diseases with onset during pruning, affecting 0.5% of 2-10-year-old children, and these epilepsies are often characterized by spontaneous remission. We previously described a remitting epilepsy in the Lagotto romagnolo canine breed. Here, we identify the gene defect and affected neurochemical pathway. We reconstructed a large Lagotto pedigree of around 34 affected animals. Using genome-wide association in 11 discordant sib-pairs from this pedigree, we mapped the disease locus to a 1.7 Mb region of homozygosity in chromosome 3 where we identified a protein-truncating mutation in the Lgi2 gene, a homologue of the human epilepsy gene LGI1. We show that LGI2, like LGI1, is neuronally secreted and acts on metalloproteinase-lacking members of the ADAM family of neuronal receptors, which function in synapse remodeling, and that LGI2 truncation, like LGI1 truncations, prevents secretion and ADAM interaction. The resulting epilepsy onsets at around seven weeks (equivalent to human two years), and remits by four months (human eight years), versus onset after age eight in the majority of human patients with LGI1 mutations. Finally, we show that Lgi2 is expressed highly in the immediate post-natal period until halfway through pruning, unlike Lgi1, which is expressed in the latter part of pruning and beyond. LGI2 acts at least in part through the same ADAM receptors as LGI1, but earlier, ensuring electrical stability (absence of epilepsy) during pruning years, preceding this same function performed by LGI1 in later years. LGI2 should be considered a candidate gene for common remitting childhood epilepsies, and LGI2-to-LGI1 transition for mechanisms of childhood epilepsy remission.

Eluding oblivion with smart stochastic selection of synaptic updates

The variables involved in the equations that describe realistic synaptic dynamics always vary in a limited range. Their boundedness makes the synapses forgetful, not for the mere passage of time, but because new experiences overwrite old memories. The forgetting rate depends on how many synapses are modified by each new experience: many changes means fast learning and fast forgetting, whereas few changes means slow learning and long memory retention. Reducing the average number of modified synapses can extend the memory span at the price of a reduced amount of information stored when a new experience is memorized. Every trick which allows to slow down the learning process in a smart way can improve the memory performance. We review some of the tricks that allow to elude fast forgetting (oblivion). They are based on the stochastic selection of the synapses whose modifications are actually consolidated following each new experience. In practice only a randomly selected, small fraction of the synapses eligible for an update are actually modified. This allows to acquire the amount of information necessary to retrieve the memory without compromising the retention of old experiences. The fraction of modified synapses can be further reduced in a smart way by changing synapses only when it is really necessary, i.e. when the post-synaptic neuron does not respond as desired. Finally we show that such a stochastic selection emerges naturally from spike driven synaptic dynamics which read noisy pre and post-synaptic neural activities. These activities can actually be generated by a chaotic system.

Vitamin E reduces antidepressant-related beta-adrenoceptor down-regulation in cultured cells. Comparable effects on St. John's wort and tricyclic antidepressant treatment

The mode of action of antidepressants is still a matter of debate. Acute inhibition of neurotransmitter reuptake in central neuronal synapses, followed by a down-regulation of central postsynaptic beta-adrenoceptor (beta-AR) numbers were consistently observed in vivo, while a reduction in surface beta-AR density was found in cell cultures. Effects of the tricyclic antidepressant desipramine (DMI) were abolished by vitamin E (alpha-TOC) in vitro as well as in vivo. Alpha-TOC interfered with antidepressant-induced changes of cellular plasma membrane properties and with recycling of beta-AR. St. John's wort (SJW) extract reduced beta-AR numbers in cultured cells to a similar extent as DMI or the selective serotonin re-uptake inhibitor fluoxetine. We chronically co-exposed cell cultures to SJW extract and to alpha-TOC. Receptor down-regulation following exposure to the plant extract was inhibited in the presence of alpha-TOC suggesting a mode of action of SJW extract comparable to that of synthetic antidepressants. Inhibition of cell proliferation by the plant extract was also significantly reduced by alpha-TOC.

Expression patterns of neurexin-1 and neuroligins in brain and retina of the chick embryo: Neuroligin-3 is absent in retina

Neuroligins (NLs) constitute a family of cell-surface proteins that interact with neurexins (beta-Nxs), another class of neuronal cell-surface proteins, one of each class functioning together in synapse formation. The localization of the various neurexins and neuroligins, however, has not yet been clarified in chicken. Therefore, we studied the expression patterns of neurexin-1 (Nx-1) and neuroligin-1 and -3 during embryonic development of the chick retina and brain by reverse-transcriptase polymerase chain reaction (RT-PCR) and in situ hybridization (ISH). While neurexin-1 increased continuously in both brain and retina, the expression of both neuroligins was more variable. As shown by ISH, Nx-1 is expressed in the inner half retina along with differentiation of ganglion and amacrine cells. Transcripts of NL-1 were detected as early as day 4 and increased with the maturation of the different brain regions. In different brain regions, NL-1 showed a different time regulation. Remarkably, neuroligin-3 was entirely absent in retina. This study indicates that synaptogenetic processes in brain and retina use different molecular machineries, whereby the neuroligins might represent the more distinctly regulated part of the neurexin-neuroligin complexes. Noticeably, NL-3 does not seem to be involved in the making of retinal synapses.

Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).

Properties of basal dendrites of layer 5 pyramidal neurons: a direct patch-clamp recording study

Basal dendrites receive the majority of synapses that contact neocortical pyramidal neurons, yet our knowledge of synaptic processing in these dendrites has been hampered by their inaccessibility for electrical recordings. A new approach to patch-clamp recordings enabled us to characterize the integrative properties of these cells. Despite the short physical length of rat basal dendrites, synaptic inputs were electrotonically remote from the soma (>30-fold excitatory postsynaptic potential (EPSP) attenuation) and back-propagating action potentials were significantly attenuated. Unitary EPSPs were location dependent, reaching large amplitudes distally (>8 mV), yet their somatic contribution was relatively location independent. Basal dendrites support sodium and NMDA spikes, but not calcium spikes, for 75% of their length. This suggests that basal dendrites, despite their proximity to the site of action potential initiation, do not form a single basal-somatic region but rather should be considered as a separate integrative compartment favoring two integration modes: subthreshold, location-independent summation versus local amplification of incoming spatiotemporally clustered information.

Angiotensinergic neurons in sympathetic coeliac ganglia innervating rat and human mesenteric resistance blood vessels

In contrast to the current belief that angiotensin II (Ang II) interacts with the sympathetic nervous system only as a circulating hormone, we document here the existence of endogenous Ang II in the neurons of rat and human sympathetic coeliac ganglia and their angiotensinergic innervation with mesenteric resistance blood vessels. Angiotensinogen - and angiotensin converting enzyme-mRNA were detected by using quantitative real time polymerase chain reaction in total RNA extracts of rat coeliac ganglia, while renin mRNA was untraceable. Cathepsin D, a protease responsible for cleavage beneath other substrates also angiotensinogen to angiotensin I, was successfully detected in rat coeliac ganglia indicating the possibility of existence of alternative pathways. Angiotensinogen mRNA was also detected by in situ hybridization in the cytoplasm of neurons of rat coeliac ganglia. Immunoreactivity for Ang II was demonstrated in rat and human coeliac ganglia as well as with mesenteric resistance blood vessels. By using confocal laser scanning microscopy we were able to demonstrate the presence of angiotensinergic synapses en passant along side of vascular smooth muscle cells. Our findings indicate that Ang II is synthesized inside the neurons of sympathetic coeliac ganglia and may act as an endogenous neurotransmitter locally with the mesenteric resistance blood vessels.

Learning by the Dendritic Prediction of Somatic Spiking

Recent modeling of spike-timing-dependent plasticity indicates that plasticity involves as a third factor a local dendritic potential, besides pre- and postsynaptic firing times. We present a simple compartmental neuron model together with a non-Hebbian, biologically plausible learning rule for dendritic synapses where plasticity is modulated by these three factors. In functional terms, the rule seeks to minimize discrepancies between somatic firings and a local dendritic potential. Such prediction errors can arise in our model from stochastic fluctuations as well as from synaptic input, which directly targets the soma. Depending on the nature of this direct input, our plasticity rule subserves supervised or unsupervised learning. When a reward signal modulates the learning rate, reinforcement learning results. Hence a single plasticity rule supports diverse learning paradigms.

Matching Recall and Storage in Sequence Learning with Spiking Neural Networks

Storing and recalling spiking sequences is a general problem the brain needs to solve. It is, however, unclear what type of biologically plausible learning rule is suited to learn a wide class of spatiotemporal activity patterns in a robust way. Here we consider a recurrent network of stochastic spiking neurons composed of both visible and hidden neurons. We derive a generic learning rule that is matched to the neural dynamics by minimizing an upper bound on the Kullback–Leibler divergence from the target distribution to the model distribution. The derived learning rule is consistent with spike-timing dependent plasticity in that a presynaptic spike preceding a postsynaptic spike elicits potentiation while otherwise depression emerges. Furthermore, the learning rule for synapses that target visible neurons can be matched to the recently proposed voltage-triplet rule. The learning rule for synapses that target hidden neurons is modulated by a global factor, which shares properties with astrocytes and gives rise to testable predictions.

Development of dendritic tonic GABAergic inhibition regulates excitability and plasticity in CA1 pyramidal neurons

Synaptic plasticity rules change during development: while hippocampal synapses can be potentiated by a single action potential pairing protocol in young neurons, mature neurons require burst firing to induce synaptic potentiation. An essential component for spike timing-dependent plasticity is the backpropagating action potential (BAP). BAP along the dendrites can be modulated by morphology and ion channel composition, both of which change during late postnatal development. However it is unclear whether these dendritic changes can explain the developmental changes in synaptic plasticity induction rules. Here, we show that tonic GABAergic inhibition regulates dendritic action potential backpropagation in adolescent but not pre-adolescent CA1 pyramidal neurons. These developmental changes in tonic inhibition also altered the induction threshold for spike timing-dependent plasticity in adolescent neurons. This GABAergic regulatory effect upon backpropagation is restricted to distal regions of apical dendrites (>200 μm) and mediated by α5-containing GABA(A) receptors. Direct dendritic recordings demonstrate α5-mediated tonic GABA(A) currents in adolescent neurons which can modulate backpropagating action potentials. These developmental modulations in dendritic excitability could not be explained by concurrent changes in dendritic morphology. To explain our data, model simulations propose a distally-increasing or localized distal expression of dendritic α5 tonic inhibition in mature neurons. Overall, our results demonstrate that dendritic integration and plasticity in more mature dendrites are significantly altered by tonic α5 inhibition in a dendritic region-specific and developmentally-regulated manner.

Dendritic spine morphology determines membrane-associated protein exchange between dendritic shafts and spine heads.

The purpose of this study was to examine whether variability in the shape of dendritic spines affects protein movement within the plasma membrane. Using a combination of confocal microscopy and the fluorescence loss in photobleaching technique in living hippocampal CA1 pyramidal neurons expressing membrane-linked GFP, we observed a clear correlation between spine shape parameters and the diffusion and compartmentalization of membrane-associated proteins. The kinetics of membrane-linked GFP exchange between the dendritic shaft and the spine head compartment were slower in dendritic spines with long necks and/or large heads than in those with short necks and/or small heads. Furthermore, when the spine area was reduced by eliciting epileptiform activity, the kinetics of protein exchange between the spine compartments exhibited a concomitant decrease. As synaptic plasticity is considered to involve the dynamic flux by lateral diffusion of membrane-bound proteins into and out of the synapse, our data suggest that spine shape represents an important parameter in the susceptibility of synapses to undergo plastic change.

Autonomic angiotensinergic fibres in the human heart with an efferent sympathetic cophenotype

AIM The autonomic innervation of the heart consists of sympathetic and parasympathetic nerve fibres, and fibres of the intrinsic ganglionated plexus with noradrenaline and acytylcholine as principal neurotransmitters. The fibres co-release neuropeptides to modulate intracardiac neurotransmission by specific presynaptic and postsynaptic receptors. The coexpression of angiotensin II in sympathetic fibres of the human heart and its role are not known so far. METHODS Autopsy specimens of human hearts were studied (n=3; ventricles). Using immunocytological methods, cryostat sections were stained by a murine monoclonal antibody (4B3) directed against angiotensin II and co-stained by polyclonal antibodies against tyrosine hydroxylase, a catecholaminergic marker. Visualisation of the antibodies was by confocal light microscopy or laser scanning microscopy. RESULTS Angiotensin II-positive autonomic fibres with and without a catecholaminergic cophenotype (hydroxylase-positive) were found in all parts of the human ventricles. In the epicardium, the fibres were grouped in larger bundles of up to 100 and more fibres. They followed the preformed anatomic septa and epicardial vessels towards the myocardium and endocardium where the bundles dissolved and the individual fibres spread between myocytes and within the endocardium. Generally, angiotensinergic fibres showed no synaptic enlargements or only a few if they were also catecholaminergic. The exclusively catechalominergic fibres were characterised by multiple beaded synapses. CONCLUSION The autonomic innervation of the human heart contains angiotensinergic fibres with a sympathetic efferent phenotype and exclusively angiotensinergic fibers representing probably afferents. Angiotensinergic neurotransmission may modulate intracardiac sympathetic and parasympathetic activity and thereby influence cardiac and circulatory function.