Intraneuronal angiotensinergic system in rat and human dorsal root ganglia

To elucidate the local formation of angiotensin II (Ang II) in the neurons of sensory dorsal root ganglia (DRG), we studied the expression of angiotensinogen (Ang-N)-, renin-, angiotensin converting enzyme (ACE)- and cathepsin D-mRNA, and the presence of protein renin, Ang II, Substance P and calcitonin gene-related peptide (CGRP) in the rat and human thoracic DRG. Quantitative real time PCR (qRT-PCR) studies revealed that rat DRG expressed substantial amounts of Ang-N- and ACE mRNA, while renin mRNA as well as the protein renin were untraceable. Cathepsin D-mRNA and cathepsin D-protein were detected in the rat DRG indicating the possibility of existence of pathways alternative to renin for Ang I formation. Angiotensin peptides were successfully detected with high performance liquid chromatography and radioimmunoassay in human DRG extracts. In situ hybridization in rat DRG confirmed additionally expression of Ang-N mRNA in the cytoplasm of numerous neurons. Intracellular Ang II staining could be shown in number of neurons and their processes in both the rat and human DRG. Interestingly we observed neuronal processes with angiotensinergic synapses en passant, colocalized with synaptophysin, within the DRG. In the DRG, we also identified by qRT-PCR, expression of Ang II receptor AT(1A) and AT(2)-mRNA while AT(1B)-mRNA was not traceable. In some neurons Substance P and CGRP were found colocalized with Ang II. The intracellular localization and colocalization of Ang II with Substance P and CGRP in the DRG neurons may indicate a participation and function of Ang II in the regulation of nociception. In conclusion, these results suggest that Ang II may be produced locally in the neurons of rat and human DRG and act as a neurotransmitter.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

Interactions between short-term and long-term plasticity: shooting for a moving target

Far from being static transmission units, synapses are highly dynamical elements that change over multiple time scales depending on the history of the neural activity of both the pre- and postsynaptic neuron. Moreover, synaptic changes on different time scales interact: long-term plasticity (LTP) can modify the properties of short-term plasticity (STP) in the same synapse. Most existing theories of synaptic plasticity focus on only one of these time scales (either STP or LTP or late-LTP) and the theoretical principles underlying their interactions are thus largely unknown. Here we develop a normative model of synaptic plasticity that combines both STP and LTP and predicts specific patterns for their interactions. Recently, it has been proposed that STP arranges for the local postsynaptic membrane potential at a synapse to behave as an optimal estimator of the presynaptic membrane potential based on the incoming spikes. Here we generalize this approach by considering an optimal estimator of a non-linear function of the membrane potential and the long-term synaptic efficacy—which itself may be subject to change on a slower time scale. We find that an increase in the long-term synaptic efficacy necessitates changes in the dynamics of STP. More precisely, for a realistic non-linear function to be estimated, our model predicts that after the induction of LTP, causing long-term synaptic efficacy to increase, a depressing synapse should become even more depressing. That is, in a protocol using trains of presynaptic stimuli, as the initial EPSP becomes stronger due to LTP, subsequent EPSPs should become weakened and this weakening should be more pronounced with LTP. This form of redistribution of synaptic efficacies agrees well with electrophysiological data on synapses connecting layer 5 pyramidal neurons.

Responses to Central Oxotremorine and Scopolamine Support the Cholinergic Control of Male Mating Behavior in Hamsters

The responses of hamsters to intracranial injections of the cholinergic agonist oxotremorine (OXO) implicate cholinergic mechanisms in the medial preoptic area (MPOA) in the control of male mating behavior. To extend these observations, we ran three studies of responses to cholinergic drugs delivered singly or in combination to the vicinity of the MPOA. The first tested responses to OXO, confirming its ability to reduce the postejaculatory interval. The second complemented the first by examining responses to MPOA microinjections of the cholinergic antagonist scopolamine (SCO). These caused several changes revolving around intromission. These included increases in intromission frequency and ejaculation latency. They also included a change in the patterning of intromissions, marked by continuous strings without the usual separation by dismounts. The final study resembled the others in examining the effects of MPOA injections of OXO and SCO but focused on the ability of each drug to antagonize responses to the other. Most of the responses to OXO and SCO individually replicated earlier findings, though the measures examined here also permitted the description of effects on some noncopulatory sexual behaviors, specifically the male's inspection of the female. However, the most interesting results may be those suggesting asymmetry in the responses to the addition of the second drug: Whereas responses to OXO tended to be antagonized by SCO, OXO was less effective at counteracting responses to SCO. Though the explanation of this asymmetry is not completely clear, it is consistent with previous suggestions of differences in the affinities of these drugs for subtypes of muscarinic receptors. Therefore, it suggests that the cholinergic synapses and circuits controlling distinct elements of male behavior could differ in their dependence on these receptors. Copyright 2013 Elsevier Inc. All rights reserved.

LGI2 truncation causes a remitting focal epilepsy in dogs

One quadrillion synapses are laid in the first two years of postnatal construction of the human brain, which are then pruned until age 10 to 500 trillion synapses composing the final network. Genetic epilepsies are the most common neurological diseases with onset during pruning, affecting 0.5% of 2-10-year-old children, and these epilepsies are often characterized by spontaneous remission. We previously described a remitting epilepsy in the Lagotto romagnolo canine breed. Here, we identify the gene defect and affected neurochemical pathway. We reconstructed a large Lagotto pedigree of around 34 affected animals. Using genome-wide association in 11 discordant sib-pairs from this pedigree, we mapped the disease locus to a 1.7 Mb region of homozygosity in chromosome 3 where we identified a protein-truncating mutation in the Lgi2 gene, a homologue of the human epilepsy gene LGI1. We show that LGI2, like LGI1, is neuronally secreted and acts on metalloproteinase-lacking members of the ADAM family of neuronal receptors, which function in synapse remodeling, and that LGI2 truncation, like LGI1 truncations, prevents secretion and ADAM interaction. The resulting epilepsy onsets at around seven weeks (equivalent to human two years), and remits by four months (human eight years), versus onset after age eight in the majority of human patients with LGI1 mutations. Finally, we show that Lgi2 is expressed highly in the immediate post-natal period until halfway through pruning, unlike Lgi1, which is expressed in the latter part of pruning and beyond. LGI2 acts at least in part through the same ADAM receptors as LGI1, but earlier, ensuring electrical stability (absence of epilepsy) during pruning years, preceding this same function performed by LGI1 in later years. LGI2 should be considered a candidate gene for common remitting childhood epilepsies, and LGI2-to-LGI1 transition for mechanisms of childhood epilepsy remission.

Role of Acetylcholine in Control of Sexual Behavior of Male and Female Mammals

The results of studies using systemic or central applications of cholinergic drugs suggest that acetylcholine makes important contributions to the neurochemical control of male- and female-typical reproductive behaviors. In males, cholinergic control seems largely specific to some elements or aspects of copulatory behavior that can vary significantly across species. Synapses in or near the medial preoptic area represent part of this mechanism, but the entire system appears to extend more widely, perhaps especially to one or more structures flanking some part of the lateral ventricle. In females, the lordosis response that essentially defines sexual receptivity is clearly responsive to cholinergic drugs. The same seems likely to be true of other elements of female sexual behavior, but additional studies will be needed to confirm this. Changes in cholinergic activity may help to mediate estrogenic effects on female sexual behavior. However, estrogen exposure can increase or decrease cholinergic effects, suggesting a relationship that is complex and requires further analysis. Also presently unclear is the localization of the cholinergic effects on female sexual responses. Though periventricular sites again have been implicated, their identity is presently unknown. This review discusses these and other aspects of the central cholinergic systems affecting male and female sexual behaviors. Copyright 2014 Elsevier Inc. All rights reserved.

Eluding oblivion with smart stochastic selection of synaptic updates

The variables involved in the equations that describe realistic synaptic dynamics always vary in a limited range. Their boundedness makes the synapses forgetful, not for the mere passage of time, but because new experiences overwrite old memories. The forgetting rate depends on how many synapses are modified by each new experience: many changes means fast learning and fast forgetting, whereas few changes means slow learning and long memory retention. Reducing the average number of modified synapses can extend the memory span at the price of a reduced amount of information stored when a new experience is memorized. Every trick which allows to slow down the learning process in a smart way can improve the memory performance. We review some of the tricks that allow to elude fast forgetting (oblivion). They are based on the stochastic selection of the synapses whose modifications are actually consolidated following each new experience. In practice only a randomly selected, small fraction of the synapses eligible for an update are actually modified. This allows to acquire the amount of information necessary to retrieve the memory without compromising the retention of old experiences. The fraction of modified synapses can be further reduced in a smart way by changing synapses only when it is really necessary, i.e. when the post-synaptic neuron does not respond as desired. Finally we show that such a stochastic selection emerges naturally from spike driven synaptic dynamics which read noisy pre and post-synaptic neural activities. These activities can actually be generated by a chaotic system.

Vitamin E reduces antidepressant-related beta-adrenoceptor down-regulation in cultured cells. Comparable effects on St. John's wort and tricyclic antidepressant treatment

The mode of action of antidepressants is still a matter of debate. Acute inhibition of neurotransmitter reuptake in central neuronal synapses, followed by a down-regulation of central postsynaptic beta-adrenoceptor (beta-AR) numbers were consistently observed in vivo, while a reduction in surface beta-AR density was found in cell cultures. Effects of the tricyclic antidepressant desipramine (DMI) were abolished by vitamin E (alpha-TOC) in vitro as well as in vivo. Alpha-TOC interfered with antidepressant-induced changes of cellular plasma membrane properties and with recycling of beta-AR. St. John's wort (SJW) extract reduced beta-AR numbers in cultured cells to a similar extent as DMI or the selective serotonin re-uptake inhibitor fluoxetine. We chronically co-exposed cell cultures to SJW extract and to alpha-TOC. Receptor down-regulation following exposure to the plant extract was inhibited in the presence of alpha-TOC suggesting a mode of action of SJW extract comparable to that of synthetic antidepressants. Inhibition of cell proliferation by the plant extract was also significantly reduced by alpha-TOC.

Expression patterns of neurexin-1 and neuroligins in brain and retina of the chick embryo: Neuroligin-3 is absent in retina

Neuroligins (NLs) constitute a family of cell-surface proteins that interact with neurexins (beta-Nxs), another class of neuronal cell-surface proteins, one of each class functioning together in synapse formation. The localization of the various neurexins and neuroligins, however, has not yet been clarified in chicken. Therefore, we studied the expression patterns of neurexin-1 (Nx-1) and neuroligin-1 and -3 during embryonic development of the chick retina and brain by reverse-transcriptase polymerase chain reaction (RT-PCR) and in situ hybridization (ISH). While neurexin-1 increased continuously in both brain and retina, the expression of both neuroligins was more variable. As shown by ISH, Nx-1 is expressed in the inner half retina along with differentiation of ganglion and amacrine cells. Transcripts of NL-1 were detected as early as day 4 and increased with the maturation of the different brain regions. In different brain regions, NL-1 showed a different time regulation. Remarkably, neuroligin-3 was entirely absent in retina. This study indicates that synaptogenetic processes in brain and retina use different molecular machineries, whereby the neuroligins might represent the more distinctly regulated part of the neurexin-neuroligin complexes. Noticeably, NL-3 does not seem to be involved in the making of retinal synapses.

Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).

Properties of basal dendrites of layer 5 pyramidal neurons: a direct patch-clamp recording study

Basal dendrites receive the majority of synapses that contact neocortical pyramidal neurons, yet our knowledge of synaptic processing in these dendrites has been hampered by their inaccessibility for electrical recordings. A new approach to patch-clamp recordings enabled us to characterize the integrative properties of these cells. Despite the short physical length of rat basal dendrites, synaptic inputs were electrotonically remote from the soma (>30-fold excitatory postsynaptic potential (EPSP) attenuation) and back-propagating action potentials were significantly attenuated. Unitary EPSPs were location dependent, reaching large amplitudes distally (>8 mV), yet their somatic contribution was relatively location independent. Basal dendrites support sodium and NMDA spikes, but not calcium spikes, for 75% of their length. This suggests that basal dendrites, despite their proximity to the site of action potential initiation, do not form a single basal-somatic region but rather should be considered as a separate integrative compartment favoring two integration modes: subthreshold, location-independent summation versus local amplification of incoming spatiotemporally clustered information.