Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

M-type channels selectively control bursting in rat dopaminergic neurons

Midbrain dopaminergic neurons in the substantia nigra, pars compacta and ventral tegmental area are critically important in many physiological functions. These neurons exhibit firing patterns that include tonic slow pacemaking, irregular firing and bursting, and the amount of dopamine that is present in the synaptic cleft is much increased during bursting. The mechanisms responsible for the switch between these spiking patterns remain unclear. Using both in-vivo recordings combined with microiontophoretic or intraperitoneal drug applications and in-vitro experiments, we have found that M-type channels, which are present in midbrain dopaminergic cells, modulate the firing during bursting without affecting the background low-frequency pacemaker firing. Thus, a selective blocker of these channels, 10,10-bis(4-pyridinylmethyl)-9(10H)- anthracenone dihydrochloride, specifically potentiated burst firing. Computer modeling of the dopamine neuron confirmed the possibility of a differential influence of M-type channels on excitability during various firing patterns. Therefore, these channels may provide a novel target for the treatment of dopamine-related diseases, including Parkinson's disease and drug addiction. Moreover, our results demonstrate that the influence of M-type channels on the excitability of these slow pacemaker neurons is conditional upon their firing pattern. © 2010 Federation of European Neuroscience Societies and Blackwell Publishing Ltd.

Optimal recall from bounded metaplastic synapses: predicting functional adaptations in hippocampal area CA3.

A venerable history of classical work on autoassociative memory has significantly shaped our understanding of several features of the hippocampus, and most prominently of its CA3 area, in relation to memory storage and retrieval. However, existing theories of hippocampal memory processing ignore a key biological constraint affecting memory storage in neural circuits: the bounded dynamical range of synapses. Recent treatments based on the notion of metaplasticity provide a powerful model for individual bounded synapses; however, their implications for the ability of the hippocampus to retrieve memories well and the dynamics of neurons associated with that retrieval are both unknown. Here, we develop a theoretical framework for memory storage and recall with bounded synapses. We formulate the recall of a previously stored pattern from a noisy recall cue and limited-capacity (and therefore lossy) synapses as a probabilistic inference problem, and derive neural dynamics that implement approximate inference algorithms to solve this problem efficiently. In particular, for binary synapses with metaplastic states, we demonstrate for the first time that memories can be efficiently read out with biologically plausible network dynamics that are completely constrained by the synaptic plasticity rule, and the statistics of the stored patterns and of the recall cue. Our theory organises into a coherent framework a wide range of existing data about the regulation of excitability, feedback inhibition, and network oscillations in area CA3, and makes novel and directly testable predictions that can guide future experiments.

Activity in descending dopaminergic neurons represents but is not required for leg movements in the fruit fly Drosophila.

Modulatory descending neurons (DNs) that link the brain to body motor circuits, including dopaminergic DNs (DA-DNs), are thought to contribute to the flexible control of behavior. Dopamine elicits locomotor-like outputs and influences neuronal excitability in isolated body motor circuits over tens of seconds to minutes, but it remains unknown how and over what time scale DA-DN activity relates to movement in behaving animals. To address this question, we identified DA-DNs in the Drosophila brain and developed an electrophysiological preparation to record and manipulate the activity of these cells during behavior. We find that DA-DN spike rates are rapidly modulated during a subset of leg movements and scale with the total speed of ongoing leg movements, whether occurring spontaneously or in response to stimuli. However, activating DA-DNs does not elicit leg movements in intact flies, nor do acute bidirectional manipulations of DA-DN activity affect the probability or speed of leg movements over a time scale of seconds to minutes. Our findings indicate that in the context of intact descending control, changes in DA-DN activity are not sufficient to influence ongoing leg movements and open the door to studies investigating how these cells interact with other descending and local neuromodulatory inputs to influence body motor output.

LEARNING IMPULSE CONTROL IN A NOVEL ANIMAL MODEL: SYNAPTIC, CELLULAR, AND PHARMACOLOGICAL SUBSTRATES

Impulse control, an executive process that restrains inappropriate actions, is impaired in numerous psychiatric conditions. This thesis reports three experiments that utilized a novel animal model of impulse control, the response inhibition (RI) task, to examine the substrates that underlie learning this task. In the first experiment, rats were trained to withhold responding on the RI task, and then euthanized for electrophysiological testing. Training in the RI task increased the AMPA/NMDA ratio at the synapses of pyramidal neurons in the prelimbic, but not infralimbic, region of the medial prefrontal cortex. This enhancement paralleled performance as subjects underwent acquisition and extinction of the inhibitory response. AMPA/NMDA was elevated only in neurons that project to the ventral striatum. Thus, this experiment identified a synaptic correlate of impulse control. In the second experiment, a separate group of rats were trained in the RI task prior to electrophysiological testing. Training in the RI task produced a decrease in membrane excitability in prelimbic, but not infralimbic, neurons as measured by maximal spiking evoked in response to increasing current injection. Importantly, this decrease was strongly correlated with successful inhibition in the task. Fortuitously, subjects trained in an operant control condition showed elevated infralimbic, but not prelimbic, excitability, which was produced by learning an anticipatory signal that predicted imminent reward availability. These experiments revealed two cellular correlates of performance, corresponding to learning two different associations under distinct task conditions. In the final experiment, rats were trained on the RI task under three conditions: Short (4-s), long (60-s), or unpredictable (1-s to 60-s) premature phases. These conditions produced distinct errors on the RI task. Interestingly, amphetamine increased premature responding in the short and long conditions, but decreased premature responding in the unpredictable condition. This dissociation may arise from interactions between amphetamine and underlying cognitive processes, such as attention, timing, and conditioned avoidance. In summary, this thesis showed that learning to inhibit a response produces distinct synaptic, cellular, and pharmacological changes. It is hoped that these advances will provide a starting point for future therapeutic interventions of disorders of impulse control.

The Function of Intestinal Afferent Neurons in Regulating Satiety During Health and Obesity

Background and aim: Within the gastrointestinal tract, vagal afferents regulate satiety and food intake via chemical and mechanical mechanisms. Cysteinyl Leukotrienes (CysLTs) are lipid mediators that are believed to regulate food intake and body weight. However, the involvement of vagal afferents in this effect remains to be established. Conversely, Glucagon like peptide-1 (GLP-1) is a satiety and incretin peptide hormone. The effect of obesity on GLP-1 mediated gut-brain signaling has yet to be investigated. Since intestinal vagal afferents’ activity is reduced during obesity, it is intriguing to investigate their responses to GLP-1 in such conditions. Methods: Extracellular recordings were performed on intestinal afferents from normal C57Bl6, low fat fed (LFF), and high fat fed (HFF) mice. To examine the effect on neuronal calcium signaling, calcium-imaging experiments were performed on isolated nodose ganglion neurons. Food intake experiments were conducted using LFF and HFF mice. Oral glucose tolerance tests (OGTT) were carried out. Whole cell patch clamp recordings were performed on nodose ganglion neurons from A) normal C57Bl mice to test the effect of CysLTs on membrane excitability, B) LFF and HFF mice to examine GLP-1 effect on membrane excitability during obesity. c-Fos immunohistochemical techniques were performed to measure the level of neuronal activation in the brainstem of both LFF and HFF mice in response to Ex-4. Results: CysLTs increased intestinal afferent firing rate and mechanosensitivity. In single nodose neuron experiments, CysLTs increased excitability. The GLP-1 agonist Ex-4 significantly decreased food intake in LFF but not HFF mice. However, Ex-4 markedly attenuated the rise in blood glucose in both LFF and HFF mice. The observed increase in nerve firing and mechanosensitivity following the application of GLP-1 and Ex-4 was abolished in HFF mice. Cell membrane excitability was significantly increased by Ex-4 in nodose from LFF but not HFF mice. Ex-4 significantly increased the number of activated neurons in the NTS area of LFF mice but not in their HFF counterparts. Conclusion: The previous observations indicate that the role CysLTs play in regulating satiety is likely to be vagally mediated. Also that satiety, but not incretin, effects of GLP-1 are impaired during obesity.

Reliability of the input-output properties of the cortico-spinal pathway obtained from transcranial magnetic and electrical stimulation

The purpose of this experiment was to assess the test-retest reliability of input-output parameters of the cortico-spinal pathway derived from transcranial magnetic (TMS) and electrical (TES) stimulation at rest and during muscle contraction. Motor evoked potentials (MEPs) were recorded from the first dorsal interosseous muscle of eight individuals on three separate days. The intensity of TMS at rest was varied from 5% below threshold to the maximal output of the stimulator. During trials in which the muscle was active, TMS and TES intensities were selected that elicited MEPs of between 150 and 300 X at rest. MEPs were evoked while the participants exerted torques up to 50% of their maximum capacity. The relationship between MEP size and stimulus intensity at rest was sigmoidal (R-2 = 0.97). Intra-class correlation coefficients (ICC) ranged between 0.47 and 0.81 for the parameters of the sigmoid function. For the active trials, the slope and intercept of regression equations of MEP size on level of background contraction were obtained more reliably for TES (ICC = 0.63 and 0.78, respectively) than for TMS (ICC = 0.50 and 0.53, respectively), These results suggest that input-output parameters of the cortico-spinal pathway may be reliably obtained via transcranial stimulation during longitudinal investigations of cortico-spinal plasticity. (C) 2001 Elsevier Science B.V. All rights reserved.

Immunohistochemical localization of a Shaker-related voltage-gated potassium channel protein in Schistosoma mansoni (Trematoda: Digenea)

We have recently isolated a cDNA (SKV1.1) encoding a Shakei-related K+ channel from the human parasitic trematode Schistosoma mansoni. In order to better understand the functions of SKv1.1 protein, the distribution of SKv1.1 protein in adult S. mansoni was analyzed by immunohistochemistry using a region-specific antibody. SKV1.1 proteins were widely expressed in the nervous and muscular systems. The strongest immunoreactivity (IR) was observed in the nervous system of both male and female. In the nervous system, IR for SKv1.1 proteins was localized in cell bodies and nerve fibers of the anterior ganglia, the central commissure, and the main nerve cords. IR was also observed in the dorsal and the ventral peripheral nerve nets, fine nerve fibers entering into a variety of structures such as the dorsal tubercles, longitudinal and ventral muscle fibers, and oral and ventral suckers. In the muscular system, SKv1.1 proteins were localized to the longitudinal, circular, and ventral muscle fibers of male as well as in isolated muscle fibers where native A-type K+ currents were measured. Moderate IR was also seen in a large number of cell bodies in the parenchyma. These results indicate that SKv1.1 protein may play an important role in the regulation of the excitability of neurons and muscle cells of S. mansoni. (C) 1995 Academic Press, Inc.

The up-regulation of voltage-gated sodium channels subtypes coincides with an increased sodium current in hippocampal neuronal culture model

Voltage-gated sodium channels (VGSC) have been linked to inherited forms of epilepsy. The expression and biophysical properties of VGSC in the hippocampal neuronal culture model have not been clarified. In order to evaluate mechanisms of epileptogenesis that are related to VGSC, we examined the expression and function of VGSC in the hippocampal neuronal culture model in vitro and spontaneously epileptic rats (SER) in vivo. Our data showed that the peak amplitude of transient, rapidly–inactivating Na+ current (INa,T) in model neurons was significantly increased compared with control neurons, and the activation curve was shifted to the negative potentials in model neurons in whole cell recording by patch–clamp. In addition, channel activity of persistent, non-inactivating Na+ current (INa,P) was obviously increased in the hippocampal neuronal culture model as judged by single–channel patch–clamp recording. Furthermore, VGSC subtypes NaV1.1, NaV1.2 and NaV1.3 were up-regulated at the protein expression level in model neurons and SER as assessed by Western blotting. Four subtypes of VGSC proteins in SER were clearly present throughout the hippocampus, including CA1, CA3 and dentate gyrus regions, and neurons expressing VGSC immunoreactivity were also detected in hippocampal neuronal culture model by immunofluorescence. These findings suggested that the up-regulation of voltage-gated sodium channels subtypes in neurons coincided with an increased sodium current in the hippocampal neuronal culture model, providing a possible explanation for the observed seizure discharge and enhanced excitability in epilepsy.

Functional expression of KCNQ (Kv7) channels in guinea-pig bladder smooth muscle and their contribution to spontaneous activity

Background and Purpose: The aim of the study was to determine whether KCNQ channels are functionally expressed in bladder smooth muscle cells (SMC) and to investigate their physiological significance in bladder contractility. 

Experimental Approach: KCNQ channels were examined at the genetic, protein, cellular and tissue level in guinea pig bladder smooth muscle using RT-PCR, immunofluorescence, patch-clamp electrophysiology, calcium imaging, detrusor strip myography, and a panel of KCNQ activators and inhibitors. 

Key Results: KCNQ subtypes 1-5 are expressed in bladder detrusor smooth muscle. Detrusor strips typically displayed TTX-insensitive myogenic spontaneous contractions that were increased in amplitude by the KCNQ channel inhibitors XE991, linopirdine or chromanol 293B. Contractility was inhibited by the KCNQ channel activators flupirtine or meclofenamic acid (MFA). The frequency of Ca2+-oscillations in SMC contained within bladder tissue sheets was increased by XE991. Outward currents in dispersed bladder SMC, recorded under conditions where BK and KATP currents were minimal, were significantly reduced by XE991, linopirdine, or chromanol, and enhanced by flupirtine or MFA. XE991 depolarized the cell membrane and could evoke transient depolarizations in quiescent cells. Flupirtine (20M) hyperpolarized the cell membrane with a simultaneous cessation of any spontaneous electrical activity. 

Conclusions and Implications: These novel findings reveal the role of KCNQ currents in the regulation of the resting membrane potential of detrusor SMC and their important physiological function in the control of spontaneous contractility in the guinea pig bladder.

Speed of processing in the primary motor cortex: A continuous theta burst stimulation study

‘Temporally urgent’ reactions are extremely rapid, spatially precise movements that are evoked following discrete stimuli. The involvement of primary motor cortex (M1) and its relationship to stimulus intensity in such reactions is not well understood. Continuous theta burst stimulation (cTBS) suppresses focal regions of the cortex and can assess the involvement of motor cortex in speed of processing. The primary objective of this study was to explore the involvement of M1 in speed of processing with respect to stimulus intensity. Thirteen healthy young adults participated in this experiment. Behavioral testing consisted of a simple button press using the index finger following median nerve stimulation of the opposite limb, at either high or low stimulus intensity. Reaction time was measured by the onset of electromyographic activity from the first dorsal interosseous (FDI) muscle of each limb. Participants completed a 30 min bout of behavioral testing prior to, and 15 min following, the delivery of cTBS to the motor cortical representation of the right FDI. The effect of cTBS on motor cortex was measured by recording the average of 30 motor evoked potentials (MEPs) just prior to, and 5 min following, cTBS. Paired t-tests revealed that, of thirteen participants, five demonstrated a significant attenuation, three demonstrated a significant facilitation and five demonstrated no significant change in MEP amplitude following cTBS. Of the group that demonstrated attenuated MEPs, there was a biologically significant interaction between stimulus intensity and effect of cTBS on reaction time and amplitude of muscle activation. This study demonstrates the variability of potential outcomes associated with the use of cTBS and further study on the mechanisms that underscore the methodology is required. Importantly, changes in motor cortical excitability may be an important determinant of speed of processing following high intensity stimulation.

Investigation of physiological properties of nerves and muscles using electromyography.

The measurement and representation of the electrical activity of muscles [electromyography (EMG)] have a long history from the Victorian Era until today. Currently, EMG has uses both as a research tool, in noninvasively recording muscle activation, and clinically in the diagnosis and assessment of nerve and muscle disease and injury as well as in assessing the recovery of neuromuscular function after nerve damage. In the present report, we describe the use of a basic EMG setup in our teaching laboratories to demonstrate some of these current applications. Our practical also illustrates some fundamental physiological and structural properties of nerves and muscles. Learning activities include 1) displaying the recruitment of muscle fibers with increasing force development; 2) the measurement of conduction velocity of motor nerves; 3) the assessment of reflex delay and demonstration of Jendrassik's maneuver; and 4) a Hoffman reflex experiment that illustrates the composition of mixed nerves and the differential excitability thresholds of fibers within the same nerve, thus aiding an understanding of the reflex nature of muscle control. We can set up the classes at various levels of inquiry depending on the needs/professional requirements of the class. The results can then provide an ideal platform for a discovery learning session/tutorial on how the central nervous system controls muscles, giving insights on how supraspinal control interacts with reflexes to give smooth, precise muscular activation.

Timing of response differentiation in human motor cortex during a speeded Go/No-go task

We explored the brain's ability to quickly prevent a pre-potent but unwanted motor response. To address this, transcranial magnetic stimulation was delivered over the motor cortex (hand representation) to probe excitability changes immediately after somatosensory cues prompted subjects to either move as fast as possible or withhold movement. Our results showed a difference in motor cortical excitability 90 ms post-stimulus contingent on cues to either promote or prevent movement. We suggest that our study design emphasizing response speed coupled with well-defined early probes allowed us to extend upon similar past investigations into the timing of response inhibition.

Estudo exploratório da excitabilidade axonal com a "técnica threshold tracking" : controlos saudáveis e doentes com polineuropatia amilóide familiar

Tese de mestrado, Neurociências, Faculdade de Medicina, Universidade de Lisboa, 2015

Etude des propriétés dynamiques de la sécrétion régulée des vésicules glutamatergiques des astrocytes

RESUME GRAND PUBLICLe cerveau est composé de différents types cellulaires, dont les neurones et les astrocytes. Faute de moyens pour les observer, les astrocytes sont très longtemps restés dans l'ombre alors que les neurones, bénéficiant des outils ad hoc pour être stimulés et étudiés, ont fait l'objet de toutes les attentions. Le développement de l'imagerie cellulaire et des outils fluorescents ont permis d'observer ces cellules non électriquement excitables et d'obtenir des informations qui laissent penser que ces cellules sont loin d'être passives et participent activement au fonctionnement cérébral. Cette participation au fonctionnement cérébral se fait en partie par le biais de la libération de substances neuro-actives (appellées gliotransmetteurs) que les astrocytes libèrent à proximité des synapses permettant ainsi de moduler le fonctionnement neuronal. Cette libération de gliotransmetteurs est principalement causée par l'activité neuronale que les astrocytes sont capables de sentir. Néanmoins, nous savons encore peu de chose sur les propriétés précises de la libération des gliotransmetteurs. Comprendre les propriétés spatio-temporelles de cette libération est essentiel pour comprendre le mode de communication de ces cellules et leur implication dans la transmission de l'information cérébrale. En utilisant des outils fluorescents récemment développés et en combinant différentes techniques d'imagerie cellulaire, nous avons pu obtenir des informations très précises sur la libération de ces gliotransmetteurs par les astrocytes. Nous avons ainsi confirmé que cette libération était un processus très rapide et qu'elle était contrôlée par des augmentations de calcium locales et rapides. Nous avons également décrit une organisation complexe de la machinerie supportant la libération des gliotransmetteurs. Cette organisation complexe semble être à la base de la libération extrêmement rapide des gliotransmetteurs. Cette rapidité de libération et cette complexité structurelle semblent indiquer que les astrocytes sont des cellules particulièrement adaptées à une communication rapide et qu'elles peuvent, au même titre que les neurones dont elles seraient les partenaires légitimes, participer à la transmission et à l'intégration de l'information cérébrale.RESUMEDe petites vésicules, les « SLMVs » ou « Synaptic Like MicroVesicles », exprimant des transporteurs vésiculaires du glutamate (VGluTs) et libérant du glutamate par exocytose régulée, ont récemment été décrites dans les astrocytes en culture et in situ. Néanmoins, nous savons peu de chose sur les propriétés précises de la sécrétion de ces SLMVs. Contrairement aux neurones, le couplage stimulussécrétion des astrocytes n'est pas basé sur l'ouverture des canaux calciques membranaires mais nécessite l'intervention de seconds messagers et la libération du calcium par le reticulum endoplasmique (RE). Comprendre les propriétés spatio-temporelles de la sécrétion astrocytaire est essentiel pour comprendre le mode de communication de ces cellules et leur implication dans la transmission de l'information cérébrale. Nous avons utilisé des outils fluorescents récemment développés pour étudier le recyclage des vésicules synaptiques glutamatergiques comme les colorants styryles et la pHluorin afin de pouvoir suivre la sécrétion des SLMVs à l'échelle de la cellule mais également à l'échelle des évènements. L'utilisation combinée de l'épifluorescence et de la fluorescence à onde évanescente nous a permis d'obtenir une résolution temporelle et spatiale sans précédent. Ainsi avons-nous confirmé que la sécrétion régulée des astrocytes était un processus très rapide (de l'ordre de quelques centaines de millisecondes). Nous avons découvert que cette sécrétion est contrôlée par des augmentations de calcium locales et rapides. Nous avons également décrit des compartiments cytosoliques délimités par le RE à proximité de la membrane plasmique et contenant les SLMVs. Cette organisation semble être à la base du couplage rapide entre l'activation des GPCRs et la sécrétion. L'existence de compartiments subcellulaires indépendants permettant de contenir les messagers intracellulaires et de limiter leur diffusion semble compenser de manière efficace la nonexcitabilité électrique des astrocytes. Par ailleurs, l'existence des différents pools de vésicules recrutés séquentiellement et fusionnant selon des modalités distinctes ainsi que l'existence de mécanismes permettant le renouvellement de ces pools lors de la stimulation suggèrent que les astrocytes peuvent faire face à une stimulation soutenue de leur sécrétion. Ces données suggèrent que la libération de gliotransmetteurs par exocytose régulée n'est pas seulement une propriété des astrocytes en culture mais bien le résultat d'une forte spécialisation de ces cellules pour la sécrétion. La rapidité de cette sécrétion donne aux astrocytes toutes les compétences pour pouvoir intervenir de manière active dans la transmission et l'intégration de l'information.ABSTRACTRecently, astrocytic synaptic like microvesicles (SLMVs), that express vesicular glutamate transporters (VGluTs) and are able to release glutamate by Ca2+-dependent regulated exocytosis, have been described both in tissue and in cultured astrocytes. Nevertheless, little is known about the specific properties of regulated secretion in astrocytes. Important differences may exist between astrocytic and neuronal exocytosis, starting from the fact that stimulus-secretion coupling in astrocytes is voltage independent, mediated by G-protein-coupled receptors and the release of Ca2+ from internal stores. Elucidating the spatiotemporal properties of astrocytic exo-endocytosis is, therefore, of primary importance for understanding the mode of communication of these cells and their role in brain signaling. We took advantage of fluorescent tools recently developed for studying recycling of glutamatergic vesicles at synapses like styryl dyes and pHluorin in order to follow exocytosis and endocytosis of SLMVs at the level of the entire cell or at the level of single event. We combined epifluorescence and total internal reflection fluorescence imaging to investigate, with unprecedented temporal and spatial resolution, the events underlying the stimulus-secretion in astrocytes. We confirmed that exo-endocytosis process in astrocytes proceeds with a time course on the millisecond time scale. We discovered that SLMVs exocytosis is controlled by local and fast Ca2+ elevations; indeed submicrometer cytosolic compartments delimited by endoplasmic reticulum (ER) tubuli reaching beneath the plasma membrane and containing SLMVs. Such complex organization seems to support the fast stimulus-secretion coupling reported here. Independent subcellular compartments formed by ER, SLMVs and plasma membrane containing intracellular messengers and limiting their diffusion seem to compensate efficiently the non-electrical excitability of astrocytes. Moreover, the existence of two pools of SLMVs which are sequentially recruited suggests a compensatory mechanisms allowing the refill of SLMVs and supporting exocytosis process over a wide range of multiple stimuli. These data suggest that regulated secretion is not only a feature of cultured astrocytes but results from a strong specialization of these cells. The rapidity of secretion demonstrates that astrocytes are able to actively participate in brain information transmission and processing.