Imagery and retrieval of auditory and visual information: neural correlates of successful and unsuccessful performance.

Remembering past events - or episodic retrieval - consists of several components. There is evidence that mental imagery plays an important role in retrieval and that the brain regions supporting imagery overlap with those supporting retrieval. An open issue is to what extent these regions support successful vs. unsuccessful imagery and retrieval processes. Previous studies that examined regional overlap between imagery and retrieval used uncontrolled memory conditions, such as autobiographical memory tasks, that cannot distinguish between successful and unsuccessful retrieval. A second issue is that fMRI studies that compared imagery and retrieval have used modality-aspecific cues that are likely to activate auditory and visual processing regions simultaneously. Thus, it is not clear to what extent identified brain regions support modality-specific or modality-independent imagery and retrieval processes. In the current fMRI study, we addressed this issue by comparing imagery to retrieval under controlled memory conditions in both auditory and visual modalities. We also obtained subjective measures of imagery quality allowing us to dissociate regions contributing to successful vs. unsuccessful imagery. Results indicated that auditory and visual regions contribute both to imagery and retrieval in a modality-specific fashion. In addition, we identified four sets of brain regions with distinct patterns of activity that contributed to imagery and retrieval in a modality-independent fashion. The first set of regions, including hippocampus, posterior cingulate cortex, medial prefrontal cortex and angular gyrus, showed a pattern common to imagery/retrieval and consistent with successful performance regardless of task. The second set of regions, including dorsal precuneus, anterior cingulate and dorsolateral prefrontal cortex, also showed a pattern common to imagery and retrieval, but consistent with unsuccessful performance during both tasks. Third, left ventrolateral prefrontal cortex showed an interaction between task and performance and was associated with successful imagery but unsuccessful retrieval. Finally, the fourth set of regions, including ventral precuneus, midcingulate cortex and supramarginal gyrus, showed the opposite interaction, supporting unsuccessful imagery, but successful retrieval performance. Results are discussed in relation to reconstructive, attentional, semantic memory, and working memory processes. This is the first study to separate the neural correlates of successful and unsuccessful performance for both imagery and retrieval and for both auditory and visual modalities.

Spatial and temporal scales of neuronal correlation in visual area V4.

The spiking activity of nearby cortical neurons is correlated on both short and long time scales. Understanding this shared variability in firing patterns is critical for appreciating the representation of sensory stimuli in ensembles of neurons, the coincident influences of neurons on common targets, and the functional implications of microcircuitry. Our knowledge about neuronal correlations, however, derives largely from experiments that used different recording methods, analysis techniques, and cortical regions. Here we studied the structure of neuronal correlation in area V4 of alert macaques using recording and analysis procedures designed to match those used previously in primary visual cortex (V1), the major input to V4. We found that the spatial and temporal properties of correlations in V4 were remarkably similar to those of V1, with two notable differences: correlated variability in V4 was approximately one-third the magnitude of that in V1 and synchrony in V4 was less temporally precise than in V1. In both areas, spontaneous activity (measured during fixation while viewing a blank screen) was approximately twice as correlated as visual-evoked activity. The results provide a foundation for understanding how the structure of neuronal correlation differs among brain regions and stages in cortical processing and suggest that it is likely governed by features of neuronal circuits that are shared across the visual cortex.

Division of labor in frontal eye field neurons during presaccadic remapping of visual receptive fields.

Our percept of visual stability across saccadic eye movements may be mediated by presaccadic remapping. Just before a saccade, neurons that remap become visually responsive at a future field (FF), which anticipates the saccade vector. Hence, the neurons use corollary discharge of saccades. Many of the neurons also decrease their response at the receptive field (RF). Presaccadic remapping occurs in several brain areas including the frontal eye field (FEF), which receives corollary discharge of saccades in its layer IV from a collicular-thalamic pathway. We studied, at two levels, the microcircuitry of remapping in the FEF. At the laminar level, we compared remapping between layers IV and V. At the cellular level, we compared remapping between different neuron types of layer IV. In the FEF in four monkeys (Macaca mulatta), we identified 27 layer IV neurons with orthodromic stimulation and 57 layer V neurons with antidromic stimulation from the superior colliculus. With the use of established criteria, we classified the layer IV neurons as putative excitatory (n = 11), putative inhibitory (n = 12), or ambiguous (n = 4). We found that just before a saccade, putative excitatory neurons increased their visual response at the RF, putative inhibitory neurons showed no change, and ambiguous neurons increased their visual response at the FF. None of the neurons showed presaccadic visual changes at both RF and FF. In contrast, neurons in layer V showed full remapping (at both the RF and FF). Our data suggest that elemental signals for remapping are distributed across neuron types in early cortical processing and combined in later stages of cortical microcircuitry.

Functional neuroimaging of autobiographical memory.

Functional neuroimaging studies of autobiographical memory have grown dramatically in recent years. These studies are important because they can investigate the neural correlates of processes that are difficult to study using laboratory stimuli, including: (i) complex constructive processes, (ii) recollective qualities of emotion and vividness, and (iii) remote memory retrieval. Constructing autobiographical memories involves search, monitoring and self-referential processes that are associated with activity in separable prefrontal regions. The contributions of emotion and vividness have been linked to the amygdala and visual cortex respectively. Finally, there is evidence that recent and remote autobiographical memories might activate the hippocampus equally, which has implications for memory-consolidation theories. The rapid development of innovative methods for eliciting personal memories in the scanner provides the opportunity to delve into the functional neuroanatomy of our personal past.

The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving.

We sought to map the time course of autobiographical memory retrieval, including brain regions that mediate phenomenological experiences of reliving and emotional intensity. Participants recalled personal memories to auditory word cues during event-related functional magnetic resonance imaging (fMRI). Participants pressed a button when a memory was accessed, maintained and elaborated the memory, and then gave subjective ratings of emotion and reliving. A novel fMRI approach based on timing differences capitalized on the protracted reconstructive process of autobiographical memory to segregate brain areas contributing to initial access and later elaboration and maintenance of episodic memories. The initial period engaged hippocampal, retrosplenial, and medial and right prefrontal activity, whereas the later period recruited visual, precuneus, and left prefrontal activity. Emotional intensity ratings were correlated with activity in several regions, including the amygdala and the hippocampus during the initial period. Reliving ratings were correlated with activity in visual cortex and ventromedial and inferior prefrontal regions during the later period. Frontopolar cortex was the only brain region sensitive to emotional intensity across both periods. Results were confirmed by time-locked averages of the fMRI signal. The findings indicate dynamic recruitment of emotion-, memory-, and sensory-related brain regions during remembering and their dissociable contributions to phenomenological features of the memories.

The neuropsychology of autobiographical memory.

This special issue of Cortex focuses on the relative contribution of different neural networks to memory and the interaction of 'core' memory processes with other cognitive processes. In this article, we examine both. Specifically, we identify cognitive processes other than encoding and retrieval that are thought to be involved in memory; we then examine the consequences of damage to brain regions that support these processes. This approach forces a consideration of the roles of brain regions outside of the frontal, medial-temporal, and diencephalic regions that form a central part of neurobiological theories of memory. Certain kinds of damage to visual cortex or lateral temporal cortex produced impairments of visual imagery or semantic memory; these patterns of impairment are associated with a unique pattern of amnesia that was distinctly different from the pattern associated with medial-temporal trauma. On the other hand, damage to language regions, auditory cortex, or parietal cortex produced impairments of language, auditory imagery, or spatial imagery; however, these impairments were not associated with amnesia. Therefore, a full model of autobiographical memory must consider cognitive processes that are not generally considered 'core processes,' as well as the brain regions upon which these processes depend.

Fear learning circuitry is biased toward generalization of fear associations in posttraumatic stress disorder.

Fear conditioning is an established model for investigating posttraumatic stress disorder (PTSD). However, symptom triggers may vaguely resemble the initial traumatic event, differing on a variety of sensory and affective dimensions. We extended the fear-conditioning model to assess generalization of conditioned fear on fear processing neurocircuitry in PTSD. Military veterans (n=67) consisting of PTSD (n=32) and trauma-exposed comparison (n=35) groups underwent functional magnetic resonance imaging during fear conditioning to a low fear-expressing face while a neutral face was explicitly unreinforced. Stimuli that varied along a neutral-to-fearful continuum were presented before conditioning to assess baseline responses, and after conditioning to assess experience-dependent changes in neural activity. Compared with trauma-exposed controls, PTSD patients exhibited greater post-study memory distortion of the fear-conditioned stimulus toward the stimulus expressing the highest fear intensity. PTSD patients exhibited biased neural activation toward high-intensity stimuli in fusiform gyrus (P<0.02), insula (P<0.001), primary visual cortex (P<0.05), locus coeruleus (P<0.04), thalamus (P<0.01), and at the trend level in inferior frontal gyrus (P=0.07). All regions except fusiform were moderated by childhood trauma. Amygdala-calcarine (P=0.01) and amygdala-thalamus (P=0.06) functional connectivity selectively increased in PTSD patients for high-intensity stimuli after conditioning. In contrast, amygdala-ventromedial prefrontal cortex (P=0.04) connectivity selectively increased in trauma-exposed controls compared with PTSD patients for low-intensity stimuli after conditioning, representing safety learning. In summary, fear generalization in PTSD is biased toward stimuli with higher emotional intensity than the original conditioned-fear stimulus. Functional brain differences provide a putative neurobiological model for fear generalization whereby PTSD symptoms are triggered by threat cues that merely resemble the index trauma.

Changes in Brain Resting-state Functional Connectivity Associated with Peripheral Nerve Block: A Pilot Study.

BACKGROUND: Limited information exists on the effects of temporary functional deafferentation (TFD) on brain activity after peripheral nerve block (PNB) in healthy humans. Increasingly, resting-state functional connectivity (RSFC) is being used to study brain activity and organization. The purpose of this study was to test the hypothesis that TFD through PNB will influence changes in RSFC plasticity in central sensorimotor functional brain networks in healthy human participants. METHODS: The authors achieved TFD using a supraclavicular PNB model with 10 healthy human participants undergoing functional connectivity magnetic resonance imaging before PNB, during active PNB, and during PNB recovery. RSFC differences among study conditions were determined by multiple-comparison-corrected (false discovery rate-corrected P value less than 0.05) random-effects, between-condition, and seed-to-voxel analyses using the left and right manual motor regions. RESULTS: The results of this pilot study demonstrated disruption of interhemispheric left-to-right manual motor region RSFC (e.g., mean Fisher-transformed z [effect size] at pre-PNB 1.05 vs. 0.55 during PNB) but preservation of intrahemispheric RSFC of these regions during PNB. Additionally, there was increased RSFC between the left motor region of interest (PNB-affected area) and bilateral higher order visual cortex regions after clinical PNB resolution (e.g., Fisher z between left motor region of interest and right and left lingual gyrus regions during PNB, -0.1 and -0.6 vs. 0.22 and 0.18 after PNB resolution, respectively). CONCLUSIONS: This pilot study provides evidence that PNB has features consistent with other models of deafferentation, making it a potentially useful approach to investigate brain plasticity. The findings provide insight into RSFC of sensorimotor functional brain networks during PNB and PNB recovery and support modulation of the sensory-motor integration feedback loop as a mechanism for explaining the behavioral correlates of peripherally induced TFD through PNB.

Influence of the thalamus on spatial visual processing in frontal cortex.

Each of our movements activates our own sensory receptors, and therefore keeping track of self-movement is a necessary part of analysing sensory input. One way in which the brain keeps track of self-movement is by monitoring an internal copy, or corollary discharge, of motor commands. This concept could explain why we perceive a stable visual world despite our frequent quick, or saccadic, eye movements: corollary discharge about each saccade would permit the visual system to ignore saccade-induced visual changes. The critical missing link has been the connection between corollary discharge and visual processing. Here we show that such a link is formed by a corollary discharge from the thalamus that targets the frontal cortex. In the thalamus, neurons in the mediodorsal nucleus relay a corollary discharge of saccades from the midbrain superior colliculus to the cortical frontal eye field. In the frontal eye field, neurons use corollary discharge to shift their visual receptive fields spatially before saccades. We tested the hypothesis that these two components-a pathway for corollary discharge and neurons with shifting receptive fields-form a circuit in which the corollary discharge drives the shift. First we showed that the known spatial and temporal properties of the corollary discharge predict the dynamic changes in spatial visual processing of cortical neurons when saccades are made. Then we moved from this correlation to causation by isolating single cortical neurons and showing that their spatial visual processing is impaired when corollary discharge from the thalamus is interrupted. Thus the visual processing of frontal neurons is spatiotemporally matched with, and functionally dependent on, corollary discharge input from the thalamus. These experiments establish the first link between corollary discharge and visual processing, delineate a brain circuit that is well suited for mediating visual stability, and provide a framework for studying corollary discharge in other sensory systems.

Visual memory loss and autobiographical amnesia: a case study.

Amnesia typically results from trauma to the medial temporal regions that coordinate activation among the disparate areas of cortex that represent the information that make up autobiographical memories. We proposed that amnesia should also result from damage to these regions, particularly regions that subserve long-term visual memory [Rubin, D. C., & Greenberg, D. L. (1998). Visual memory-deficit amnesia: A distinct amnesic presentation and etiology. Proceedings of the National Academy of Sciences of the USA, 95, 5413-5416]. We previously found 11 such cases in the literature, and all 11 had amnesia. We now present a detailed investigation of one of these patients. M.S. suffers from long-term visual memory loss along with some semantic deficits; he also manifests a severe retrograde amnesia and moderate anterograde amnesia. The presentation of his amnesia differs from that of the typical medial-temporal or lateral-temporal amnesic; we suggest that his visual deficits may be contributing to his autobiographical amnesia.

Neuronal correlates of visual time perception at brief timescales.

Successful interaction with the world depends on accurate perception of the timing of external events. Neurons at early stages of the primate visual system represent time-varying stimuli with high precision. However, it is unknown whether this temporal fidelity is maintained in the prefrontal cortex, where changes in neuronal activity generally correlate with changes in perception. One reason to suspect that it is not maintained is that humans experience surprisingly large fluctuations in the perception of time. To investigate the neuronal correlates of time perception, we recorded from neurons in the prefrontal cortex and midbrain of monkeys performing a temporal-discrimination task. Visual time intervals were presented at a timescale relevant to natural behavior (<500 ms). At this brief timescale, neuronal adaptation--time-dependent changes in the size of successive responses--occurs. We found that visual activity fluctuated with timing judgments in the prefrontal cortex but not in comparable midbrain areas. Surprisingly, only response strength, not timing, predicted task performance. Intervals perceived as longer were associated with larger visual responses and shorter intervals with smaller responses, matching the dynamics of adaptation. These results suggest that the magnitude of prefrontal activity may be read out to provide temporal information that contributes to judging the passage of time.

Neuronal correlates of metacognition in primate frontal cortex.

Humans are metacognitive: they monitor and control their cognition. Our hypothesis was that neuronal correlates of metacognition reside in the same brain areas responsible for cognition, including frontal cortex. Recent work demonstrated that nonhuman primates are capable of metacognition, so we recorded from single neurons in the frontal eye field, dorsolateral prefrontal cortex, and supplementary eye field of monkeys (Macaca mulatta) that performed a metacognitive visual-oculomotor task. The animals made a decision and reported it with a saccade, but received no immediate reward or feedback. Instead, they had to monitor their decision and bet whether it was correct. Activity was correlated with decisions and bets in all three brain areas, but putative metacognitive activity that linked decisions to appropriate bets occurred exclusively in the SEF. Our results offer a survey of neuronal correlates of metacognition and implicate the SEF in linking cognitive functions over short periods of time.

Dynamics of visual receptive fields in the macaque frontal eye field.

Neuronal receptive fields (RFs) provide the foundation for understanding systems-level sensory processing. In early visual areas, investigators have mapped RFs in detail using stochastic stimuli and sophisticated analytical approaches. Much less is known about RFs in prefrontal cortex. Visual stimuli used for mapping RFs in prefrontal cortex tend to cover a small range of spatial and temporal parameters, making it difficult to understand their role in visual processing. To address these shortcomings, we implemented a generalized linear model to measure the RFs of neurons in the macaque frontal eye field (FEF) in response to sparse, full-field stimuli. Our high-resolution, probabilistic approach tracked the evolution of RFs during passive fixation, and we validated our results against conventional measures. We found that FEF neurons exhibited a surprising level of sensitivity to stimuli presented as briefly as 10 ms or to multiple dots presented simultaneously, suggesting that FEF visual responses are more precise than previously appreciated. FEF RF spatial structures were largely maintained over time and between stimulus conditions. Our results demonstrate that the application of probabilistic RF mapping to FEF and similar association areas is an important tool for clarifying the neuronal mechanisms of cognition.

A subcortical source of visual input to the frontal eye field

Many neurons in the frontal eye field (FEF) exhibit visual responses and are thought to play important roles in visuosaccadic behavior. The FEF, however, is far removed from striate cortex. Where do the FEF's visual signals come from? Usually they are reasonably assumed to enter the FEF through afferents from extrastriate cortex. Here we show that, surprisingly, visual signals also enter the FEF through a subcortical route: a disynaptic, ascending pathway originating in the intermediate layers of the superior colliculus (SC). We recorded from identified neurons at all three stages of this pathway (n=30-40 in each sample): FEF recipient neurons, orthodromically activated from the SC; mediodorsal thalamus (MD) relay neurons, antidromically activated from FEF and orthodromically activated from SC; and SC source neurons, antidromically activated from MD. We studied the neurons while monkeys performed delayed saccade tasks designed to temporally resolve visual responses from presaccadic discharges. We found, first, that most neurons at every stage in the pathway had visual responses, presaccadic bursts, or both. Second, we found marked similarities between the SC source neurons and MD relay neurons: in both samples, about 15% of the neurons had only a visual response, 10% had only a presaccadic burst, and 75% had both. In contrast, FEF recipient neurons tended to be more visual in nature: 50% had only a visual response, none had only a presaccadic burst, and 50% had both a visual response and a presaccadic burst. This suggests that in addition to their subcortical inputs, these FEF neurons also receive other visual inputs, e.g. from extrastriate cortex. We conclude that visual activity in the FEF results not only from cortical afferents but also from subcortical inputs. Intriguingly, this implies that some of the visual signals in FEF are pre-processed by the SC.

What the brain stem tells the frontal cortex. II. Role of the SC-MD-FEF pathway in corollary discharge.

One way we keep track of our movements is by monitoring corollary discharges or internal copies of movement commands. This study tested a hypothesis that the pathway from superior colliculus (SC) to mediodorsal thalamus (MD) to frontal eye field (FEF) carries a corollary discharge about saccades made into the contralateral visual field. We inactivated the MD relay node with muscimol in monkeys and measured corollary discharge deficits using a double-step task: two sequential saccades were made to the locations of briefly flashed targets. To make second saccades correctly, monkeys had to internally monitor their first saccades; therefore deficits in the corollary discharge representation of first saccades should disrupt second saccades. We found, first, that monkeys seemed to misjudge the amplitudes of their first saccades; this was revealed by systematic shifts in second saccade end points. Thus corollary discharge accuracy was impaired. Second, monkeys were less able to detect trial-by-trial variations in their first saccades; this was revealed by reduced compensatory changes in second saccade angles. Thus corollary discharge precision also was impaired. Both deficits occurred only when first saccades went into the contralateral visual field. Single-saccade generation was unaffected. Additional deficits occurred in reaction time and overall performance, but these were bilateral. We conclude that the SC-MD-FEF pathway conveys a corollary discharge used for coordinating sequential saccades and possibly for stabilizing vision across saccades. This pathway is the first elucidated in what may be a multilevel chain of corollary discharge circuits extending from the extraocular motoneurons up into cerebral cortex.