A Neural Network of Adaptively Timed Reinforcement Learning and Hippocampal Dynamics

A neural model is described of how adaptively timed reinforcement learning occurs. The adaptive timing circuit is suggested to exist in the hippocampus, and to involve convergence of dentate granule cells on CA3 pyramidal cells, and NMDA receptors. This circuit forms part of a model neural system for the coordinated control of recognition learning, reinforcement learning, and motor learning, whose properties clarify how an animal can learn to acquire a delayed reward. Behavioral and neural data are summarized in support of each processing stage of the system. The relevant anatomical sites are in thalamus, neocortex, hippocampus, hypothalamus, amygdala, and cerebellum. Cerebellar influences on motor learning are distinguished from hippocampal influences on adaptive timing of reinforcement learning. The model simulates how damage to the hippocampal formation disrupts adaptive timing, eliminates attentional blocking, and causes symptoms of medial temporal amnesia. It suggests how normal acquisition of subcortical emotional conditioning can occur after cortical ablation, even though extinction of emotional conditioning is retarded by cortical ablation. The model simulates how increasing the duration of an unconditioned stimulus increases the amplitude of emotional conditioning, but does not change adaptive timing; and how an increase in the intensity of a conditioned stimulus "speeds up the clock", but an increase in the intensity of an unconditioned stimulus does not. Computer simulations of the model fit parametric conditioning data, including a Weber law property and an inverted U property. Both primary and secondary adaptively timed conditioning are simulated, as are data concerning conditioning using multiple interstimulus intervals (ISIs), gradually or abruptly changing ISis, partial reinforcement, and multiple stimuli that lead to time-averaging of responses. Neurobiologically testable predictions are made to facilitate further tests of the model.

Investigations into the cellular and molecular changes in the amygdala in models of temporal lobe epilepsy and maternal immune activation

The amygdala is a limbic structure that is involved in many of our emotions and processing of these emotions such as fear, anger and pleasure. Conditions such as anxiety, autism, and also epilepsy, have been linked to abnormal functioning of the amygdala, owing to improper neurodevelopment or damage. This thesis investigated the cellular and molecular changes in the amygdala in models of temporal lobe epilepsy (TLE) and maternal immune activation (MIA). The kainic acid (KA) model of temporal lobe epilepsy (TLE) was used to induce Ammon’s-horn sclerosis (AHS) and to investigate behavioural and cytoarchitectural changes that occur in the amygdala related to Neuropeptide Y1 receptor expression. Results showed that KA-injected animals showed increased anxiety-like behaviours and displayed histopathological hallmarks of AHS including CA1 ablation, granule cell dispersion, volume reduction and astrogliosis. Amygdalar volume and neuronal loss was observed in the ipsilateral nuclei which was accompanied by astrogliosis. In addition, a decrease in Y1 receptor expressing cells in the ipsilateral CA1 and CA3 sectors of the hippocampus, ipsi- and contralateral granule cell layer of the dentate gyrus and ipsilateral central nucleus of the amygdala was found, consistent with a reduction in Y1 receptor protein levels. The results suggest that plastic changes in hippocampal and/or amygdalar Y1 receptor expression may negatively impact anxiety levels. Gamma-aminobutyric acid (GABA) is the main inhibitory neurotransmitter in the brain and tight regulation and appropriate control of GABA is vital for neurochemical homeostasis. GABA transporter-1 (GAT-1) is abundantly expressed by neurones and astrocytes and plays a key role in GABA reuptake and regulation. Imbalance in GABA homeostasis has been implicated in epilepsy with GAT-1 being an attractive pharmacological target. Electron microscopy was used to examine the distribution, expression and morphology of GAT-1 expressing structures in the amygdala of the TLE model. Results suggest that GAT-1 was preferentially expressed on putative axon terminals over astrocytic processes in this TLE model. Myelin integrity was examined and results suggested that in the TLE model myelinated fibres were damaged in comparison to controls. Synaptic morphology was studied and results suggested that asymmetric (excitatory) synapses occurred more frequently than symmetric (inhibitory) synapses in the TLE model in comparison to controls. This study illustrated that the amygdala undergoes ultrastructural alterations in this TLE model. Maternal immune activation (MIA) is a risk factor for neurodevelopmental disorders such as autism, schizophrenia and also epilepsy. MIA was induced at a critical window of amygdalar development at E12 using bacterial mimetic lipopolysaccharide (LPS). Results showed that MIA activates cytokine, toll-like receptor and chemokine expression in the fetal brain that is prolonged in the postnatal amygdala. Inflammation elicited by MIA may prime the fetal brain for alterations seen in the glial environment and this in turn have deleterious effects on neuronal populations as seen in the amygdala at P14. These findings may suggest that MIA induced during amygdalar development may predispose offspring to amygdalar related disorders such as heightened anxiety, fear impairment and also neurodevelopmental disorders.

Preschool anxiety disorders predict different patterns of amygdala-prefrontal connectivity at school-age.

OBJECTIVE: In this prospective, longitudinal study of young children, we examined whether a history of preschool generalized anxiety, separation anxiety, and/or social phobia is associated with amygdala-prefrontal dysregulation at school-age. As an exploratory analysis, we investigated whether distinct anxiety disorders differ in the patterns of this amygdala-prefrontal dysregulation. METHODS: Participants were children taking part in a 5-year study of early childhood brain development and anxiety disorders. Preschool symptoms of generalized anxiety, separation anxiety, and social phobia were assessed with the Preschool Age Psychiatric Assessment (PAPA) in the first wave of the study when the children were between 2 and 5 years old. The PAPA was repeated at age 6. We conducted functional MRIs when the children were 5.5 to 9.5 year old to assess neural responses to viewing of angry and fearful faces. RESULTS: A history of preschool social phobia predicted less school-age functional connectivity between the amygdala and the ventral prefrontal cortices to angry faces. Preschool generalized anxiety predicted less functional connectivity between the amygdala and dorsal prefrontal cortices in response to fearful faces. Finally, a history of preschool separation anxiety predicted less school-age functional connectivity between the amygdala and the ventral prefrontal cortices to angry faces and greater school-age functional connectivity between the amygdala and dorsal prefrontal cortices to angry faces. CONCLUSIONS: Our results suggest that there are enduring neurobiological effects associated with a history of preschool anxiety, which occur over-and-above the effect of subsequent emotional symptoms. Our results also provide preliminary evidence for the neurobiological differentiation of specific preschool anxiety disorders.

Neural networks supporting autobiographical memory retrieval in posttraumatic stress disorder.

Posttraumatic stress disorder (PTSD) affects the functional recruitment and connectivity between neural regions during autobiographical memory (AM) retrieval that overlap with default and control networks. Whether such univariate changes relate to potential differences in the contributions of the large-scale neural networks supporting cognition in PTSD is unknown. In the present functional MRI study, we employed independent-component analysis to examine the influence of the engagement of neural networks during the recall of personal memories in a PTSD group (15 participants) as compared to non-trauma-exposed healthy controls (14 participants). We found that the PTSD group recruited similar neural networks when compared to the controls during AM recall, including default-network subsystems and control networks, but group differences emerged in the spatial and temporal characteristics of these networks. First, we found spatial differences in the contributions of the anterior and posterior midline across the networks, and of the amygdala in particular, for the medial temporal subsystem of the default network. Second, we found temporal differences within the medial prefrontal subsystem of the default network, with less temporal coupling of this network during AM retrieval in PTSD relative to controls. These findings suggest that the spatial and temporal characteristics of the default and control networks potentially differ in a PTSD group versus healthy controls and contribute to altered recall of personal memory.

Functional neuroimaging of emotionally intense autobiographical memories in post-traumatic stress disorder.

Post-traumatic stress disorder (PTSD) affects regions that support autobiographical memory (AM) retrieval, such as the hippocampus, amygdala and ventral medial prefrontal cortex (PFC). However, it is not well understood how PTSD may impact the neural mechanisms of memory retrieval for the personal past. We used a generic cue method combined with parametric modulation analysis and functional MRI (fMRI) to investigate the neural mechanisms affected by PTSD symptoms during the retrieval of a large sample of emotionally intense AMs. There were three main results. First, the PTSD group showed greater recruitment of the amygdala/hippocampus during the construction of negative versus positive emotionally intense AMs, when compared to controls. Second, across both the construction and elaboration phases of retrieval the PTSD group showed greater recruitment of the ventral medial PFC for negatively intense memories, but less recruitment for positively intense memories. Third, the PTSD group showed greater functional coupling between the ventral medial PFC and the amygdala for negatively intense memories, but less coupling for positively intense memories. In sum, the fMRI data suggest that there was greater recruitment and coupling of emotional brain regions during the retrieval of negatively intense AMs in the PTSD group when compared to controls.

Mental hoop diaries: emotional memories of a college basketball game in rival fans.

The rivalry between the men's basketball teams of Duke University and the University of North Carolina-Chapel Hill (UNC) is one of the most storied traditions in college sports. A subculture of students at each university form social bonds with fellow fans, develop expertise in college basketball rules, team statistics, and individual players, and self-identify as a member of a fan group. The present study capitalized on the high personal investment of these fans and the strong affective tenor of a Duke-UNC basketball game to examine the neural correlates of emotional memory retrieval for a complex sporting event. Male fans watched a competitive, archived game in a social setting. During a subsequent functional magnetic resonance imaging session, participants viewed video clips depicting individual plays of the game that ended with the ball being released toward the basket. For each play, participants recalled whether or not the shot went into the basket. Hemodynamic signal changes time locked to correct memory decisions were analyzed as a function of emotional intensity and valence, according to the fan's perspective. Results showed intensity-modulated retrieval activity in midline cortical structures, sensorimotor cortex, the striatum, and the medial temporal lobe, including the amygdala. Positively valent memories specifically recruited processing in dorsal frontoparietal regions, and additional activity in the insula and medial temporal lobe for positively valent shots recalled with high confidence. This novel paradigm reveals how brain regions implicated in emotion, memory retrieval, visuomotor imagery, and social cognition contribute to the recollection of specific plays in the mind of a sports fan.

The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving.

We sought to map the time course of autobiographical memory retrieval, including brain regions that mediate phenomenological experiences of reliving and emotional intensity. Participants recalled personal memories to auditory word cues during event-related functional magnetic resonance imaging (fMRI). Participants pressed a button when a memory was accessed, maintained and elaborated the memory, and then gave subjective ratings of emotion and reliving. A novel fMRI approach based on timing differences capitalized on the protracted reconstructive process of autobiographical memory to segregate brain areas contributing to initial access and later elaboration and maintenance of episodic memories. The initial period engaged hippocampal, retrosplenial, and medial and right prefrontal activity, whereas the later period recruited visual, precuneus, and left prefrontal activity. Emotional intensity ratings were correlated with activity in several regions, including the amygdala and the hippocampus during the initial period. Reliving ratings were correlated with activity in visual cortex and ventromedial and inferior prefrontal regions during the later period. Frontopolar cortex was the only brain region sensitive to emotional intensity across both periods. Results were confirmed by time-locked averages of the fMRI signal. The findings indicate dynamic recruitment of emotion-, memory-, and sensory-related brain regions during remembering and their dissociable contributions to phenomenological features of the memories.

Activation in mesolimbic and visuospatial neural circuits elicited by smoking cues: evidence from functional magnetic resonance imaging.

OBJECTIVE: The authors sought to increase understanding of the brain mechanisms involved in cigarette addiction by identifying neural substrates modulated by visual smoking cues in nicotine-deprived smokers. METHOD: Event-related functional magnetic resonance imaging (fMRI) was used to detect brain activation after exposure to smoking-related images in a group of nicotine-deprived smokers and a nonsmoking comparison group. Subjects viewed a pseudo-random sequence of smoking images, neutral nonsmoking images, and rare targets (photographs of animals). Subjects pressed a button whenever a rare target appeared. RESULTS: In smokers, the fMRI signal was greater after exposure to smoking-related images than after exposure to neutral images in mesolimbic dopamine reward circuits known to be activated by addictive drugs (right posterior amygdala, posterior hippocampus, ventral tegmental area, and medial thalamus) as well as in areas related to visuospatial attention (bilateral prefrontal and parietal cortex and right fusiform gyrus). In nonsmokers, no significant differences in fMRI signal following exposure to smoking-related and neutral images were detected. In most regions studied, both subject groups showed greater activation following presentation of rare target images than after exposure to neutral images. CONCLUSIONS: In nicotine-deprived smokers, both reward and attention circuits were activated by exposure to smoking-related images. Smoking cues are processed like rare targets in that they activate attentional regions. These cues are also processed like addictive drugs in that they activate mesolimbic reward regions.

The Impact of Prenatal Nicotine Exposure on Impulsivity and Neural Firing in the Medial Prefrontal Cortex

Prenatal nicotine exposure (PNE) is linked to a large number of psychiatric disorders, including attention deficit hyperactivity disorder (ADHD). Current literature suggests that core deficits observed in ADHD reflect abnormal inhibitory control governed by the prefrontal cortex (PFC) of the brain. The PFC is structurally altered by PNE, but it is still unclear how neural firing is affected during tasks that test behavioral inhibition, such as the stop-signal task, or if neural correlates related to inhibitory control are affected after PNE in awake behaving animals. To address these questions, we recorded from single medial PFC (mPFC) neurons in control rats and PNE rats as they performed our stopsignal task. We found that PNE rats were faster for all trial types and were less likely to inhibit the behavioral response on STOP trials. Neurons in mPFC fired more strongly on STOP trials and were correlated with accuracy and reaction time. Although the number of neurons exhibiting significant modulation during task performance did not differ between groups, overall activity in PNE was reduced. We conclude that PNE makes rats impulsive and reduces firing in mPFC neurons that carry signals related to response inhibition.