Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

M-type channels selectively control bursting in rat dopaminergic neurons

Midbrain dopaminergic neurons in the substantia nigra, pars compacta and ventral tegmental area are critically important in many physiological functions. These neurons exhibit firing patterns that include tonic slow pacemaking, irregular firing and bursting, and the amount of dopamine that is present in the synaptic cleft is much increased during bursting. The mechanisms responsible for the switch between these spiking patterns remain unclear. Using both in-vivo recordings combined with microiontophoretic or intraperitoneal drug applications and in-vitro experiments, we have found that M-type channels, which are present in midbrain dopaminergic cells, modulate the firing during bursting without affecting the background low-frequency pacemaker firing. Thus, a selective blocker of these channels, 10,10-bis(4-pyridinylmethyl)-9(10H)- anthracenone dihydrochloride, specifically potentiated burst firing. Computer modeling of the dopamine neuron confirmed the possibility of a differential influence of M-type channels on excitability during various firing patterns. Therefore, these channels may provide a novel target for the treatment of dopamine-related diseases, including Parkinson's disease and drug addiction. Moreover, our results demonstrate that the influence of M-type channels on the excitability of these slow pacemaker neurons is conditional upon their firing pattern. © 2010 Federation of European Neuroscience Societies and Blackwell Publishing Ltd.

Optimal recall from bounded metaplastic synapses: predicting functional adaptations in hippocampal area CA3.

A venerable history of classical work on autoassociative memory has significantly shaped our understanding of several features of the hippocampus, and most prominently of its CA3 area, in relation to memory storage and retrieval. However, existing theories of hippocampal memory processing ignore a key biological constraint affecting memory storage in neural circuits: the bounded dynamical range of synapses. Recent treatments based on the notion of metaplasticity provide a powerful model for individual bounded synapses; however, their implications for the ability of the hippocampus to retrieve memories well and the dynamics of neurons associated with that retrieval are both unknown. Here, we develop a theoretical framework for memory storage and recall with bounded synapses. We formulate the recall of a previously stored pattern from a noisy recall cue and limited-capacity (and therefore lossy) synapses as a probabilistic inference problem, and derive neural dynamics that implement approximate inference algorithms to solve this problem efficiently. In particular, for binary synapses with metaplastic states, we demonstrate for the first time that memories can be efficiently read out with biologically plausible network dynamics that are completely constrained by the synaptic plasticity rule, and the statistics of the stored patterns and of the recall cue. Our theory organises into a coherent framework a wide range of existing data about the regulation of excitability, feedback inhibition, and network oscillations in area CA3, and makes novel and directly testable predictions that can guide future experiments.

Activity in descending dopaminergic neurons represents but is not required for leg movements in the fruit fly Drosophila.

Modulatory descending neurons (DNs) that link the brain to body motor circuits, including dopaminergic DNs (DA-DNs), are thought to contribute to the flexible control of behavior. Dopamine elicits locomotor-like outputs and influences neuronal excitability in isolated body motor circuits over tens of seconds to minutes, but it remains unknown how and over what time scale DA-DN activity relates to movement in behaving animals. To address this question, we identified DA-DNs in the Drosophila brain and developed an electrophysiological preparation to record and manipulate the activity of these cells during behavior. We find that DA-DN spike rates are rapidly modulated during a subset of leg movements and scale with the total speed of ongoing leg movements, whether occurring spontaneously or in response to stimuli. However, activating DA-DNs does not elicit leg movements in intact flies, nor do acute bidirectional manipulations of DA-DN activity affect the probability or speed of leg movements over a time scale of seconds to minutes. Our findings indicate that in the context of intact descending control, changes in DA-DN activity are not sufficient to influence ongoing leg movements and open the door to studies investigating how these cells interact with other descending and local neuromodulatory inputs to influence body motor output.

LEARNING IMPULSE CONTROL IN A NOVEL ANIMAL MODEL: SYNAPTIC, CELLULAR, AND PHARMACOLOGICAL SUBSTRATES

Impulse control, an executive process that restrains inappropriate actions, is impaired in numerous psychiatric conditions. This thesis reports three experiments that utilized a novel animal model of impulse control, the response inhibition (RI) task, to examine the substrates that underlie learning this task. In the first experiment, rats were trained to withhold responding on the RI task, and then euthanized for electrophysiological testing. Training in the RI task increased the AMPA/NMDA ratio at the synapses of pyramidal neurons in the prelimbic, but not infralimbic, region of the medial prefrontal cortex. This enhancement paralleled performance as subjects underwent acquisition and extinction of the inhibitory response. AMPA/NMDA was elevated only in neurons that project to the ventral striatum. Thus, this experiment identified a synaptic correlate of impulse control. In the second experiment, a separate group of rats were trained in the RI task prior to electrophysiological testing. Training in the RI task produced a decrease in membrane excitability in prelimbic, but not infralimbic, neurons as measured by maximal spiking evoked in response to increasing current injection. Importantly, this decrease was strongly correlated with successful inhibition in the task. Fortuitously, subjects trained in an operant control condition showed elevated infralimbic, but not prelimbic, excitability, which was produced by learning an anticipatory signal that predicted imminent reward availability. These experiments revealed two cellular correlates of performance, corresponding to learning two different associations under distinct task conditions. In the final experiment, rats were trained on the RI task under three conditions: Short (4-s), long (60-s), or unpredictable (1-s to 60-s) premature phases. These conditions produced distinct errors on the RI task. Interestingly, amphetamine increased premature responding in the short and long conditions, but decreased premature responding in the unpredictable condition. This dissociation may arise from interactions between amphetamine and underlying cognitive processes, such as attention, timing, and conditioned avoidance. In summary, this thesis showed that learning to inhibit a response produces distinct synaptic, cellular, and pharmacological changes. It is hoped that these advances will provide a starting point for future therapeutic interventions of disorders of impulse control.

The Function of Intestinal Afferent Neurons in Regulating Satiety During Health and Obesity

Background and aim: Within the gastrointestinal tract, vagal afferents regulate satiety and food intake via chemical and mechanical mechanisms. Cysteinyl Leukotrienes (CysLTs) are lipid mediators that are believed to regulate food intake and body weight. However, the involvement of vagal afferents in this effect remains to be established. Conversely, Glucagon like peptide-1 (GLP-1) is a satiety and incretin peptide hormone. The effect of obesity on GLP-1 mediated gut-brain signaling has yet to be investigated. Since intestinal vagal afferents’ activity is reduced during obesity, it is intriguing to investigate their responses to GLP-1 in such conditions. Methods: Extracellular recordings were performed on intestinal afferents from normal C57Bl6, low fat fed (LFF), and high fat fed (HFF) mice. To examine the effect on neuronal calcium signaling, calcium-imaging experiments were performed on isolated nodose ganglion neurons. Food intake experiments were conducted using LFF and HFF mice. Oral glucose tolerance tests (OGTT) were carried out. Whole cell patch clamp recordings were performed on nodose ganglion neurons from A) normal C57Bl mice to test the effect of CysLTs on membrane excitability, B) LFF and HFF mice to examine GLP-1 effect on membrane excitability during obesity. c-Fos immunohistochemical techniques were performed to measure the level of neuronal activation in the brainstem of both LFF and HFF mice in response to Ex-4. Results: CysLTs increased intestinal afferent firing rate and mechanosensitivity. In single nodose neuron experiments, CysLTs increased excitability. The GLP-1 agonist Ex-4 significantly decreased food intake in LFF but not HFF mice. However, Ex-4 markedly attenuated the rise in blood glucose in both LFF and HFF mice. The observed increase in nerve firing and mechanosensitivity following the application of GLP-1 and Ex-4 was abolished in HFF mice. Cell membrane excitability was significantly increased by Ex-4 in nodose from LFF but not HFF mice. Ex-4 significantly increased the number of activated neurons in the NTS area of LFF mice but not in their HFF counterparts. Conclusion: The previous observations indicate that the role CysLTs play in regulating satiety is likely to be vagally mediated. Also that satiety, but not incretin, effects of GLP-1 are impaired during obesity.

Muscle-specific variations in use-dependent crossed-facilitation of corticospinal pathways mediated by transcranial direct current (DC) stimulation

The tendency for contractions of muscles in the upper limb to give rise to increases in the excitability of corticospinal projections to the homologous muscles of the opposite limb is well known. Although the suppression of this tendency is integral to tasks of daily living, its exploitation may prove to be critical in the rehabilitation of acquired hemiplegias. Transcranial direct current (DC) stimulation induces changes in cortical excitability that outlast the period of application. We present evidence that changes in the reactivity of the corticospinal pathway induced by DC stimulation of the motor cortex interact systematically with those brought about by contraction of the muscles of the ipsilateral limb. During the application of flexion torques (up to 50% of maximum) applied at the left wrist, motor evoked potentials (MEPs) were evoked in the quiescent muscles of the right arm by magnetic stimulation of the left motor cortex (M1). The MEPs were obtained prior to and following 10 min of anodal, cathodal or sham DC stimulation of left M1. Cathodal stimulation counteracted increases in the crossed-facilitation of projections to the (right) wrist flexors that otherwise occurred as a result of repeated flexion contractions at the left wrist. In addition, cathodal stimulation markedly decreased the excitability of corticospinal projections to the wrist extensors of the right limb. Thus changes in corticospinal excitability induced by DC stimulation can be shaped (i.e. differentiated by muscle group) by focal contractions of muscles in the limb ipsilateral to the site of stimulation. (C) 2008 Elsevier Ireland Ltd. All rights reserved.

The role of the primary motor cortex during skill acquisition on a two-degrees-of-freedom movement task

One can partially eliminate motor skills acquired through practice in the hours immediately following practice by applying repetitive transcranial stimulation (rTMS) over the primary motor cortex. The disruption of acquired levels of performance has been demonstrated on tasks that are ballistic in nature. The authors investigated whether motor recall on a discrete aiming task is degraded following a disruption of the primary motor cortex induced via rTMS. Participants (N = 16) maintained acquired performance levels and patterns of muscle activity following the application of rTMS. despite a reduction in corticospinal excitability. Disruption of the primary motor cortex during a consolidation period did not influence the retention of acquired skill in this type of discrete visuomotor task.

Immunohistochemical localization of a Shaker-related voltage-gated potassium channel protein in Schistosoma mansoni (Trematoda: Digenea)

We have recently isolated a cDNA (SKV1.1) encoding a Shakei-related K+ channel from the human parasitic trematode Schistosoma mansoni. In order to better understand the functions of SKv1.1 protein, the distribution of SKv1.1 protein in adult S. mansoni was analyzed by immunohistochemistry using a region-specific antibody. SKV1.1 proteins were widely expressed in the nervous and muscular systems. The strongest immunoreactivity (IR) was observed in the nervous system of both male and female. In the nervous system, IR for SKv1.1 proteins was localized in cell bodies and nerve fibers of the anterior ganglia, the central commissure, and the main nerve cords. IR was also observed in the dorsal and the ventral peripheral nerve nets, fine nerve fibers entering into a variety of structures such as the dorsal tubercles, longitudinal and ventral muscle fibers, and oral and ventral suckers. In the muscular system, SKv1.1 proteins were localized to the longitudinal, circular, and ventral muscle fibers of male as well as in isolated muscle fibers where native A-type K+ currents were measured. Moderate IR was also seen in a large number of cell bodies in the parenchyma. These results indicate that SKv1.1 protein may play an important role in the regulation of the excitability of neurons and muscle cells of S. mansoni. (C) 1995 Academic Press, Inc.

Primary motor cortex involvement in initial learning during visuomotor adaptation

Human motor behaviour is continually modified on the basis of errors between desired and actual movement outcomes. It is emerging that the role played by the primary motor cortex (M1) in this process is contingent upon a variety of factors, including the nature of the task being performed, and the stage of learning. Here we used repetitive TMS to test the hypothesis that M1 is intimately involved in the initial phase of sensorimotor adaptation. Inhibitory theta burst stimulation was applied to M1 prior to a task requiring modification of torques generated about the elbow/forearm complex in response to rotations of a visual feedback display. Participants were first exposed to a 30° clockwise (CW) rotation (Block A), then a 60° counterclockwise rotation (Block B), followed immediately by a second block of 30° CW rotation (A2). In the STIM condition, participants received 20s of continuous theta burst stimulation (cTBS) prior to the initial A Block. In the conventional (CON) condition, no stimulation was applied. The overt characteristics of performance in the two conditions were essentially equivalent with respect to the errors exhibited upon exposure to a new variant of the task. There were however, profound differences between the conditions in the latency of response preparation, and the excitability of corticospinal projections from M1, which accompanied phases of de-adaptation and re-adaptation (during Blocks B and A2). Upon subsequent exposure to the A rotation 24h later, the rate of re-adaptation was lower in the stimulation condition than that present in the conventional condition. These results support the assertion that primary motor cortex assumes a key role in a network that mediates adaptation to visuomotor perturbation, and emphasise that it is engaged functionally during the early phase of learning.

Vision Modulates Corticospinal Suppression in a Functionally Specific Manner during Movement of the Opposite Limb

The effect of vision on the excitability of corticospinal projections to the flexor carpi radialis (FCR) and extensor carpi radialis (ECR) muscles of right human forearm was investigated before and during discrete movement of the opposite limb. An external force opposed the initial phase of the movement (wrist flexion) and assisted the reverse phase, so that recruitment of the wrist extensors was minimized. Three conditions were used as follows: viewing the inactive right limb (Vision), viewing the mirror image of the moving left limb (Mirror), and with vision of the right limb occluded (No Vision). Transcranial magnetic stimulation was delivered to the left motor cortex: before, at the onset of, or during the left limb movement to obtain motor evoked potentials (MEPs) in the muscles of the right forearm. At and following movement onset, MEPs obtained in the right FCR were smaller in the Vision condition than in the Mirror and No Vision conditions. A distinct pattern of variation was obtained for the ECR. In all conditions, MEPs in this muscle were elevated upon or following movement of the opposite limb. An additional analysis of ipsilateral silent periods indicated that interhemispheric inhibition plays a role in mediating these effects. Activity-dependent changes in corticospinal output to a resting limb during discrete actions of the opposite limb are thus directly contingent upon where one looks. Furthermore, the extent to which vision exerts an influence upon projections to specific muscles varies in accordance with the functional contribution of their homologs to the intended action.

The up-regulation of voltage-gated sodium channels subtypes coincides with an increased sodium current in hippocampal neuronal culture model

Voltage-gated sodium channels (VGSC) have been linked to inherited forms of epilepsy. The expression and biophysical properties of VGSC in the hippocampal neuronal culture model have not been clarified. In order to evaluate mechanisms of epileptogenesis that are related to VGSC, we examined the expression and function of VGSC in the hippocampal neuronal culture model in vitro and spontaneously epileptic rats (SER) in vivo. Our data showed that the peak amplitude of transient, rapidly–inactivating Na+ current (INa,T) in model neurons was significantly increased compared with control neurons, and the activation curve was shifted to the negative potentials in model neurons in whole cell recording by patch–clamp. In addition, channel activity of persistent, non-inactivating Na+ current (INa,P) was obviously increased in the hippocampal neuronal culture model as judged by single–channel patch–clamp recording. Furthermore, VGSC subtypes NaV1.1, NaV1.2 and NaV1.3 were up-regulated at the protein expression level in model neurons and SER as assessed by Western blotting. Four subtypes of VGSC proteins in SER were clearly present throughout the hippocampus, including CA1, CA3 and dentate gyrus regions, and neurons expressing VGSC immunoreactivity were also detected in hippocampal neuronal culture model by immunofluorescence. These findings suggested that the up-regulation of voltage-gated sodium channels subtypes in neurons coincided with an increased sodium current in the hippocampal neuronal culture model, providing a possible explanation for the observed seizure discharge and enhanced excitability in epilepsy.

Functional expression of KCNQ (Kv7) channels in guinea-pig bladder smooth muscle and their contribution to spontaneous activity

Background and Purpose: The aim of the study was to determine whether KCNQ channels are functionally expressed in bladder smooth muscle cells (SMC) and to investigate their physiological significance in bladder contractility. 

Experimental Approach: KCNQ channels were examined at the genetic, protein, cellular and tissue level in guinea pig bladder smooth muscle using RT-PCR, immunofluorescence, patch-clamp electrophysiology, calcium imaging, detrusor strip myography, and a panel of KCNQ activators and inhibitors. 

Key Results: KCNQ subtypes 1-5 are expressed in bladder detrusor smooth muscle. Detrusor strips typically displayed TTX-insensitive myogenic spontaneous contractions that were increased in amplitude by the KCNQ channel inhibitors XE991, linopirdine or chromanol 293B. Contractility was inhibited by the KCNQ channel activators flupirtine or meclofenamic acid (MFA). The frequency of Ca2+-oscillations in SMC contained within bladder tissue sheets was increased by XE991. Outward currents in dispersed bladder SMC, recorded under conditions where BK and KATP currents were minimal, were significantly reduced by XE991, linopirdine, or chromanol, and enhanced by flupirtine or MFA. XE991 depolarized the cell membrane and could evoke transient depolarizations in quiescent cells. Flupirtine (20M) hyperpolarized the cell membrane with a simultaneous cessation of any spontaneous electrical activity. 

Conclusions and Implications: These novel findings reveal the role of KCNQ currents in the regulation of the resting membrane potential of detrusor SMC and their important physiological function in the control of spontaneous contractility in the guinea pig bladder.

Neural pathways mediating cross education of motor function

Cross education is the process whereby training of one limb gives rise to enhancements in the performance of the opposite, untrained limb. Despite interest in this phenomenon having been sustained for more than a century, a comprehensive explanation of the mediating neural mechanisms remains elusive. With new evidence emerging that cross education may have therapeutic utility, the need to provide a principled evidential basis upon which to design interventions becomes ever more pressing. Generally, mechanistic accounts of cross education align with one of two explanatory frameworks. Models of the 'cross activation' variety encapsulate the observation that unilateral execution of a movement task gives rise to bilateral increases in corticospinal excitability. The related conjecture is that such distributed activity, when present during unilateral practice, leads to simultaneous adaptations in neural circuits that project to the muscles of the untrained limb, thus facilitating subsequent performance of the task. Alternatively, 'bilateral access' models entail that motor engrams formed during unilateral practise, may subsequently be utilised bilaterally - that is, by the neural circuitry that constitutes the control centres for movements of both limbs. At present there is a paucity of direct evidence that allows the corresponding neural processes to be delineated, or their relative contributions in different task contexts to be ascertained. In the current review we seek to synthesise and assimilate the fragmentary information that is available, including consideration of knowledge that has emerged as a result of technological advances in structural and functional brain imaging. An emphasis upon task dependency is maintained throughout, the conviction being that the neural mechanisms that mediate cross education may only be understood in this context. © 2013 Ruddy and Carson.

Anticipatory Planning Reveals Segmentation of Cortical Motor Output During Action Observation

It has been argued that the variation in brain activity that occurs when observing another person reflects a representation of actions that is indivisible, and which plays out in full once the intent of the actor can be discerned. We used transcranial magnetic stimulation to probe the excitability of corticospinal projections to 2 intrinsic hand muscles while motions to reach and grasp an object were observed. A symbolic cue either faithfully indicated the required final orientation of the object and thus the nature of the grasp that was required, or was in conflict with the movement subsequently displayed. When the cue was veridical, modulation of excitability was in accordance with the functional role of the muscles in the action observed. If however the cue had indicated that the alternative grasp would be required, modulation of output to first dorsal interosseus was consistent with the action specified, rather than the action observed-until the terminal phase of the motion sequence during which the object was seen lifted. Modulation of corticospinal output during observation is thus segmented-it progresses initially in accordance with the action anticipated, and if discrepancies are revealed by visual input, coincides thereafter with that of the action seen.