Learning and Production of Movement Sequences: Behavioral, Neurophysiological, and Modeling Perspectives

A growing wave of behavioral studies, using a wide variety of paradigms that were introduced or greatly refined in recent years, has generated a new wealth of parametric observations about serial order behavior. What was a mere trickle of neurophysiological studies has grown to a more steady stream of probes of neural sites and mechanisms underlying sequential behavior. Moreover, simulation models of serial behavior generation have begun to open a channel to link cellular dynamics with cognitive and behavioral dynamics. Here we summarize the major results from prominent sequence learning and performance tasks, namely immediate serial recall, typing, 2XN, discrete sequence production, and serial reaction time. These populate a continuum from higher to lower degrees of internal control of sequential organization. The main movement classes covered are speech and keypressing, both involving small amplitude movements that are very amenable to parametric study. A brief synopsis of classes of serial order models, vis-à-vis the detailing of major effects found in the behavioral data, leads to a focus on competitive queuing (CQ) models. Recently, the many behavioral predictive successes of CQ models have been joined by successful prediction of distinctively patterend electrophysiological recordings in prefrontal cortex, wherein parallel activation dynamics of multiple neural ensembles strikingly matches the parallel dynamics predicted by CQ theory. An extended CQ simulation model-the N-STREAMS neural network model-is then examined to highlight issues in ongoing attemptes to accomodate a broader range of behavioral and neurophysiological data within a CQ-consistent theory. Important contemporary issues such as the nature of working memory representations for sequential behavior, and the development and role of chunks in hierarchial control are prominent throughout.

Temporal Dynamics of Decision-Making during Motion Perception in the Visual Cortex

How does the brain make decisions? Speed and accuracy of perceptual decisions covary with certainty in the input, and correlate with the rate of evidence accumulation in parietal and frontal cortical "decision neurons." A biophysically realistic model of interactions within and between Retina/LGN and cortical areas V1, MT, MST, and LIP, gated by basal ganglia, simulates dynamic properties of decision-making in response to ambiguous visual motion stimuli used by Newsome, Shadlen, and colleagues in their neurophysiological experiments. The model clarifies how brain circuits that solve the aperture problem interact with a recurrent competitive network with self-normalizing choice properties to carry out probablistic decisions in real time. Some scientists claim that perception and decision-making can be described using Bayesian inference or related general statistical ideas, that estimate the optimal interpretation of the stimulus given priors and likelihoods. However, such concepts do not propose the neocortical mechanisms that enable perception, and make decisions. The present model explains behavioral and neurophysiological decision-making data without an appeal to Bayesian concepts and, unlike other existing models of these data, generates perceptual representations and choice dynamics in response to the experimental visual stimuli. Quantitative model simulations include the time course of LIP neuronal dynamics, as well as behavioral accuracy and reaction time properties, during both correct and error trials at different levels of input ambiguity in both fixed duration and reaction time tasks. Model MT/MST interactions compute the global direction of random dot motion stimuli, while model LIP computes the stochastic perceptual decision that leads to a saccadic eye movement.

Neuropeptide Co-Release with Gaba May Explain Functional Non-Monotonic Uncertainty Responses in Dopamine Neurons

Co-release of the inhibitory neurotransmitter GABA and the neuropeptide substance-P (SP) from single axons is a conspicuous feature of the basal ganglia, yet its computational role, if any, has not been resolved. In a new learning model, co-release of GABA and SP from axons of striatal projection neurons emerges as a highly efficient way to compute the uncertainty responses that are exhibited by dopamine (DA) neurons when animals adapt to probabilistic contingencies between rewards and the stimuli that predict their delivery. Such uncertainty-related dopamine release appears to be an adaptive phenotype, because it promotes behavioral switching at opportune times. Understanding the computational linkages between SP and DA in the basal ganglia is important, because Huntington's disease is characterized by massive SP depletion, whereas Parkinson's disease is characterized by massive DA depletion.

Adaptation of Dopamine Neurons' Mismatch Sensitivity Is not Compatible With Reinforcement Learning Theory

Recent electrophysical data inspired the claim that dopaminergic neurons adapt their mismatch sensitivities to reflect variances of expected rewards. This contradicts reward prediction error theory and most basal ganglia models. Application of learning principles points to a testable alternative interpretation-of the same data-that is compatible with existing theory.

A Local Circuit Model of Learned Straitial and Dopamine Cell Responses under Probabilistic Schedules of Reward

Before choosing, it helps to know both the expected value signaled by a predictive cue and the associated uncertainty that the reward will be forthcoming. Recently, Fiorillo et al. (2003) found the dopamine (DA) neurons of the SNc exhibit sustained responses related to the uncertainty that a cure will be followed by reward, in addition to phasic responses related to reward prediction errors (RPEs). This suggests that cue-dependent anticipations of the timing, magnitude, and uncertainty of rewards are learned and reflected in components of the DA signals broadcast by SNc neurons. What is the minimal local circuit model that can explain such multifaceted reward-related learning? A new computational model shows how learned uncertainty responses emerge robustly on single trial along with phasic RPE responses, such that both types of DA responses exhibit the empirically observed dependence on conditional probability, expected value of reward, and time since onset of the reward-predicting cue. The model includes three major pathways for computing: immediate expected values of cures, timed predictions of reward magnitudes (and RPEs), and the uncertainty associated with these predictions. The first two model pathways refine those previously modeled by Brown et al. (1999). A third, newly modeled, pathway is formed by medium spiny projection neurons (MSPNs) of the matrix compartment of the striatum, whose axons co-release GABA and a neuropeptide, substance P, both at synapses with GABAergic neurons in the SNr and with the dendrites (in SNr) of DA neurons whose somas are in ventral SNc. Co-release enables efficient computation of sustained DA uncertainty responses that are a non-monotonic function of the conditonal probability that a reward will follow the cue. The new model's incorporation of a striatal microcircuit allowed it to reveals that variability in striatal cholinergic transmission can explain observed difference, between monkeys, in the amplitutude of the non-monotonic uncertainty function. Involvement of matriceal MSPNs and striatal cholinergic transmission implpies a relation between uncertainty in the cue-reward contigency and action-selection functions of the basal ganglia. The model synthesizes anatomical, electrophysiological and behavioral data regarding the midbrain DA system in a novel way, by relating the ability to compute uncertainty, in parallel with other aspects of reward contingencies, to the unique distribution of SP inputs in ventral SN.

Dopaminergic and Non-Dopaminergic Value Systems in Conditioning and Outcome-Specific Revaluation

Animals are motivated to choose environmental options that can best satisfy current needs. To explain such choices, this paper introduces the MOTIVATOR (Matching Objects To Internal Values Triggers Option Revaluations) neural model. MOTIVATOR describes cognitiveemotional interactions between higher-order sensory cortices and an evaluative neuraxis composed of the hypothalamus, amygdala, and orbitofrontal cortex. Given a conditioned stimulus (CS), the model amygdala and lateral hypothalamus interact to calculate the expected current value of the subjective outcome that the CS predicts, constrained by the current state of deprivation or satiation. The amygdala relays the expected value information to orbitofrontal cells that receive inputs from anterior inferotemporal cells, and medial orbitofrontal cells that receive inputs from rhinal cortex. The activations of these orbitofrontal cells code the subjective values of objects. These values guide behavioral choices. The model basal ganglia detect errors in CS-specific predictions of the value and timing of rewards. Excitatory inputs from the pedunculopontine nucleus interact with timed inhibitory inputs from model striosomes in the ventral striatum to regulate dopamine burst and dip responses from cells in the substantia nigra pars compacta and ventral tegmental area. Learning in cortical and striatal regions is strongly modulated by dopamine. The model is used to address tasks that examine food-specific satiety, Pavlovian conditioning, reinforcer devaluation, and simultaneous visual discrimination. Model simulations successfully reproduce discharge dynamics of known cell types, including signals that predict saccadic reaction times and CS-dependent changes in systolic blood pressure.

Simulating Effects of Learning and Lesions with a Model of Intrinsic and Synaptically Gated Responses of Striatal Cholinergic Interneurons

The giant cholinergic interneurons of the striatum are tonically active neurons (TANs) that respond with characteristic pauses to novel events and to appetitive and aversive conditioned stimuli. Fluctuations in acetylcholine release by TANs modulate performance- and learning-related dynamics in the striatum. Whereas tonic activity emerges from intrinsic properties of these neurons, glutamatergic inputs from thalamic centromedian-parafascicular nuclei, and dopaminergic inputs from midbrain, are required for the generation of pause responses. No prior computational models encompass both intrinsic and synaptically-gated dynamics. We present a mathematical model that robustly accounts for behavior-related electrophysiological properties of TANs in terms of their intrinsic physiological properties and known afferents. In the model, balanced intrinsic hyperpolarizing and depolarizing currents engender tonic firing, and glutamatergic inputs from thalamus (and cortex) both directly excite and indirectly inhibit TANs. If the latter inhibition, presumably mediated by GABAergic interneurons, exceeds a threshold, its effect is amplified by a KIR current to generate a prolonged pause. In the model, the intrinsic mechanisms and external inputs are both modulated by learning-dependent dopamine (DA) signals and our simulations revealed that many learning-dependent behaviors of TANs are explicable without recourse to learning-dependent changes in synapses onto TANs. The "teaching signal" that modulates reinforcement learning at cortico-striatal synapses may be a sequence composed of an adaptively scaled DA burst, a brief ACh burst, and a scaled ACh pause. Such an interpretation is consistent with recent data on cholinergic control of LTD of cortical synapses onto striatal spiny projection neurons.

The role of glucocorticoid-induced tumour necrosis factor receptor in developing mouse sympathetic neurons

Hereditary sensory autonomic neuropathy IV (HSAN IV) is an autosomal recessive disorder characterised by inability to feel pain and anhidrosis and is a consequence of defective NGF/TrkA signalling and growth of sensory and sympathetic neurons. Glucocortiocoid-induced tumour necrosis factors receptor (GITR), a transmembrane protein, activated by its specific ligand, GITRL, is well known for its role in the regulation of innate and acquired immune system responses. Recently, GITR was found to be required for NGF-dependant and extracellular signal-related kinase 1/2 (ERK1/2)-induced neurite growth and target innervation in the developing sympathetic nervous system (SNS). Given this novel role of GITR, it is possible that strategies targeting GITR have potential therapeutic benefit in promoting neurite growth in autonomic neuropathies such as HSAN IV. Using P1 mouse SCG neurons as a model, in addition to various SCG cell treatments, knock down models and transfection methods, we investigated whether GITR increases the sensitivity of sympathetic neurons to NGF; the region of GITR required for the enhancement of NGF-promoted growth, the signalling pathways downstream of GITR and how extensively GITR is involved in regulating peripheral innervation of the SNS. Results indicate that the region responsible for the growth promoting effects of GITR lies in its juxtamembrane intracellular region (here termed the growth promoting domain (GPD)) of GITR. The GPD of GITR activates ERK1/2 and inhibits nuclear factor kappa B (NF-κB) in an inverse fashion to provide an optimal cellular growth environment for P1 SCG neurons. While deleting the GPD of GITR had no effect on TrkA expression, constitutive phosphorylation of specific sites in the GPD reduced TrkA expression indicating a possible role for GITR in increasing the sensitivity of SCG neurons to NGF by the regulation of these sites, TrkA expression and subsequent NGF/TrkA binding. GITR appears to be heterogeneously required for NGF-promoted target innervation of SCG neurons in some organs, implying additional factors are involved in extensive NGF-target innervation of the SNS. In conclusion, this study answers basic biological questions regarding the molecular mechanism behind the role of GITR in the development of the SNS, and provides a basis for future research if GITR modulation is to be developed as a strategy for promoting axonal growth.

Molecular genetics of the imprinted gene - SPROUTY3 (SPRY3)

Sprouty proteins are key regulators of cell growth and branching morphogenesis during development. Human SPRY3 which maps to the pseudoautosomal region 2, undergoes random X-inactivation in females and preferential Y-inactivation in males, behaving as though genetically X-linked. Spry3 is widely expressed in neuronal tissues, being found at high levels in the cerebellum and particularly in the Purkinje cells which, notably, are deficient in the autistic brain. Spry3 is also highly expressed in other ganglia in adults including retinal ganglion cells, dorsal root ganglion and superior cervical ganglion. SPRY3 enhancer can drive SPRY3 expression in the lung airway, which is consistent with a role in branching morphogenesis and the function of the original Drosophila Spry gene, which is critical for lung morphogenesis, providing a possible explanation for an observed anatomic abnormality in the autistic lung airway. In the human and mouse, the SPRY3 core promoter contains an AG-rich repeat and we found evidence of coexpression, promoter binding and regulation of SPRY3 expression by transcription factors EGR1, ZNF263 and PAX6. Spry3 over-expression in mouse superior cervical ganglion cells inhibits axon branching and Spry3 knockdown in those cells increases axon branching, consistent with known functions of other Sprouty proteins. Novel SPRY3 upstream transcripts that I characterised originate from three start sites in the X-linked F8A3 – TMLHE gene region, which is recently implicated in autism causation. Arising from these findings, I propose that the lung airway abnormality and low levels of blood carnitine found in autism suggest that deregulation of SPRY3 may underpin a subset of autism cases.

Phenotypic overlap between infantile systemic lupus erythematosus and Aicardi-Goutières syndrome.

Case Reports

Advances in understanding mechanisms of thalamic relays in cognition and behavior.

The main impetus for a mini-symposium on corticothalamic interrelationships was the recent number of studies highlighting the role of the thalamus in aspects of cognition beyond sensory processing. The thalamus contributes to a range of basic cognitive behaviors that include learning and memory, inhibitory control, decision-making, and the control of visual orienting responses. Its functions are deeply intertwined with those of the better studied cortex, although the principles governing its coordination with the cortex remain opaque, particularly in higher-level aspects of cognition. How should the thalamus be viewed in the context of the rest of the brain? Although its role extends well beyond relaying of sensory information from the periphery, the main function of many of its subdivisions does appear to be that of a relay station, transmitting neural signals primarily to the cerebral cortex from a number of brain areas. In cognition, its main contribution may thus be to coordinate signals between diverse regions of the telencephalon, including the neocortex, hippocampus, amygdala, and striatum. This central coordination is further subject to considerable extrinsic control, for example, inhibition from the basal ganglia, zona incerta, and pretectal regions, and chemical modulation from ascending neurotransmitter systems. What follows is a brief review on the role of the thalamus in aspects of cognition and behavior, focusing on a summary of the topics covered in a mini-symposium held at the Society for Neuroscience meeting, 2014.

Cognitive control of movement via the cerebellar-recipient thalamus.

The cognitive control of behavior was long considered to be centralized in cerebral cortex. More recently, subcortical structures such as cerebellum and basal ganglia have been implicated in cognitive functions as well. The fact that subcortico-cortical circuits for the control of movement involve the thalamus prompts the notion that activity in movement-related thalamus may also reflect elements of cognitive behavior. Yet this hypothesis has rarely been investigated. Using the pathways linking cerebellum to cerebral cortex via the thalamus as a template, we review evidence that the motor thalamus, together with movement-related central thalamus have the requisite connectivity and activity to mediate cognitive aspects of movement control.

Exploring the role of the substantia nigra pars reticulata in eye movements.

Experiments that demonstrated a role for the substantia nigra in eye movements have played an important role in our understanding of the function of the basal ganglia in behavior more broadly. In this review we explore more recent experiments that extend the role of the substantia nigra pars reticulata from a simple gate for eye movements to include a role in cognitive processes for eye movements. We review recent evidence suggesting that basal ganglia nuclei beyond the substantia nigra may also play a role in eye movements and the cognitive events leading up to the production of eye movements. We close by pointing out some unresolved questions in our understanding of the relationship of basal ganglia nuclei and eye movements.

The role of stimulus salience and attentional capture across the neural hierarchy in a stop-signal task.

Inhibitory motor control is a core function of cognitive control. Evidence from diverse experimental approaches has linked this function to a mostly right-lateralized network of cortical and subcortical areas, wherein a signal from the frontal cortex to the basal ganglia is believed to trigger motor-response cancellation. Recently, however, it has been recognized that in the context of typical motor-control paradigms those processes related to actual response inhibition and those related to the attentional processing of the relevant stimuli are highly interrelated and thus difficult to distinguish. Here, we used fMRI and a modified Stop-signal task to specifically examine the role of perceptual and attentional processes triggered by the different stimuli in such tasks, thus seeking to further distinguish other cognitive processes that may precede or otherwise accompany the implementation of response inhibition. In order to establish which brain areas respond to sensory stimulation differences by rare Stop-stimuli, as well as to the associated attentional capture that these may trigger irrespective of their task-relevance, we compared brain activity evoked by Stop-trials to that evoked by Go-trials in task blocks where Stop-stimuli were to be ignored. In addition, region-of-interest analyses comparing the responses to these task-irrelevant Stop-trials, with those to typical relevant Stop-trials, identified separable activity profiles as a function of the task-relevance of the Stop-signal. While occipital areas were mostly blind to the task-relevance of Stop-stimuli, activity in temporo-parietal areas dissociated between task-irrelevant and task-relevant ones. Activity profiles in frontal areas, in turn, were activated mainly by task-relevant Stop-trials, presumably reflecting a combination of triggered top-down attentional influences and inhibitory motor-control processes.

The evolutionarily conserved transcription factor PRDM12 controls sensory neuron development and pain perception

PR homology domain-containing member 12 (PRDM12) is a highly evolutionary conserved member of the Prdm family of transcription factors that play essential roles in many cell fate decisions. In human, PRDM12 coding mutations have been recently identified in several patients with hereditary sensory and autonomic neuropathy (HSAN) (submitted elsewhere). Here we show that PRDM12 is involved in sensory neurogenesis in Xenopus and that several of the human Prdm12 mutants show altered structure, subcellular localization and function. In Drosophila, we demonstrate that the sensory neuron specific RNAi knockdown of the Prdm12 ortholog Hamlet induces impaired nociception and that a similar phenotype is observed in hypomorph hamlet mutants. In human fibroblasts of patients with PRDM12 mutations, we identified additional possible downstream target genes including thyrotropin-releasing hormone degrading enzyme (TRHDE). Knock-down of fly TRHDE in sensory neurons resulted in altered nociceptive neurons and impaired nociception. Collectively, these findings provide the first evidence showing that Prdm12 plays an important role in sensory neuron development. They also suggest that it has a critical evolutionarily conserved role in pain perception via modulation of the TRH signaling pathway.