Imagery and retrieval of auditory and visual information: neural correlates of successful and unsuccessful performance.

Remembering past events - or episodic retrieval - consists of several components. There is evidence that mental imagery plays an important role in retrieval and that the brain regions supporting imagery overlap with those supporting retrieval. An open issue is to what extent these regions support successful vs. unsuccessful imagery and retrieval processes. Previous studies that examined regional overlap between imagery and retrieval used uncontrolled memory conditions, such as autobiographical memory tasks, that cannot distinguish between successful and unsuccessful retrieval. A second issue is that fMRI studies that compared imagery and retrieval have used modality-aspecific cues that are likely to activate auditory and visual processing regions simultaneously. Thus, it is not clear to what extent identified brain regions support modality-specific or modality-independent imagery and retrieval processes. In the current fMRI study, we addressed this issue by comparing imagery to retrieval under controlled memory conditions in both auditory and visual modalities. We also obtained subjective measures of imagery quality allowing us to dissociate regions contributing to successful vs. unsuccessful imagery. Results indicated that auditory and visual regions contribute both to imagery and retrieval in a modality-specific fashion. In addition, we identified four sets of brain regions with distinct patterns of activity that contributed to imagery and retrieval in a modality-independent fashion. The first set of regions, including hippocampus, posterior cingulate cortex, medial prefrontal cortex and angular gyrus, showed a pattern common to imagery/retrieval and consistent with successful performance regardless of task. The second set of regions, including dorsal precuneus, anterior cingulate and dorsolateral prefrontal cortex, also showed a pattern common to imagery and retrieval, but consistent with unsuccessful performance during both tasks. Third, left ventrolateral prefrontal cortex showed an interaction between task and performance and was associated with successful imagery but unsuccessful retrieval. Finally, the fourth set of regions, including ventral precuneus, midcingulate cortex and supramarginal gyrus, showed the opposite interaction, supporting unsuccessful imagery, but successful retrieval performance. Results are discussed in relation to reconstructive, attentional, semantic memory, and working memory processes. This is the first study to separate the neural correlates of successful and unsuccessful performance for both imagery and retrieval and for both auditory and visual modalities.

The short and long of it: neural correlates of temporal-order memory for autobiographical events.

Previous functional neuroimaging studies of temporal-order memory have investigated memory for laboratory stimuli that are causally unrelated and poor in sensory detail. In contrast, the present functional magnetic resonance imaging (fMRI) study investigated temporal-order memory for autobiographical events that were causally interconnected and rich in sensory detail. Participants took photographs at many campus locations over a period of several hours, and the following day they were scanned while making temporal-order judgments to pairs of photographs from different locations. By manipulating the temporal lag between the two locations in each trial, we compared the neural correlates associated with reconstruction processes, which we hypothesized depended on recollection and contribute mainly to short lags, and distance processes, which we hypothesized to depend on familiarity and contribute mainly to longer lags. Consistent with our hypotheses, parametric fMRI analyses linked shorter lags to activations in regions previously associated with recollection (left prefrontal, parahippocampal, precuneus, and visual cortices), and longer lags with regions previously associated with familiarity (right prefrontal cortex). The hemispheric asymmetry in prefrontal cortex activity fits very well with evidence and theories regarding the contributions of the left versus right prefrontal cortex to memory (recollection vs. familiarity processes) and cognition (systematic vs. heuristic processes). In sum, using a novel photo-paradigm, this study provided the first evidence regarding the neural correlates of temporal-order for autobiographical events.

Core and Shell Song Systems Unique to the Parrot Brain.

The ability to imitate complex sounds is rare, and among birds has been found only in parrots, songbirds, and hummingbirds. Parrots exhibit the most advanced vocal mimicry among non-human animals. A few studies have noted differences in connectivity, brain position and shape in the vocal learning systems of parrots relative to songbirds and hummingbirds. However, only one parrot species, the budgerigar, has been examined and no differences in the presence of song system structures were found with other avian vocal learners. Motivated by questions of whether there are important differences in the vocal systems of parrots relative to other vocal learners, we used specialized constitutive gene expression, singing-driven gene expression, and neural connectivity tracing experiments to further characterize the song system of budgerigars and/or other parrots. We found that the parrot brain uniquely contains a song system within a song system. The parrot "core" song system is similar to the song systems of songbirds and hummingbirds, whereas the "shell" song system is unique to parrots. The core with only rudimentary shell regions were found in the New Zealand kea, representing one of the only living species at a basal divergence with all other parrots, implying that parrots evolved vocal learning systems at least 29 million years ago. Relative size differences in the core and shell regions occur among species, which we suggest could be related to species differences in vocal and cognitive abilities.

Cross-hemispheric collaboration and segregation associated with task difficulty as revealed by structural and functional connectivity.

Although it is known that brain regions in one hemisphere may interact very closely with their corresponding contralateral regions (collaboration) or operate relatively independent of them (segregation), the specific brain regions (where) and conditions (how) associated with collaboration or segregation are largely unknown. We investigated these issues using a split field-matching task in which participants matched the meaning of words or the visual features of faces presented to the same (unilateral) or to different (bilateral) visual fields. Matching difficulty was manipulated by varying the semantic similarity of words or the visual similarity of faces. We assessed the white matter using the fractional anisotropy (FA) measure provided by diffusion tensor imaging (DTI) and cross-hemispheric communication in terms of fMRI-based connectivity between homotopic pairs of cortical regions. For both perceptual and semantic matching, bilateral trials became faster than unilateral trials as difficulty increased (bilateral processing advantage, BPA). The study yielded three novel findings. First, whereas FA in anterior corpus callosum (genu) correlated with word-matching BPA, FA in posterior corpus callosum (splenium-occipital) correlated with face-matching BPA. Second, as matching difficulty intensified, cross-hemispheric functional connectivity (CFC) increased in domain-general frontopolar cortex (for both word and face matching) but decreased in domain-specific ventral temporal lobe regions (temporal pole for word matching and fusiform gyrus for face matching). Last, a mediation analysis linking DTI and fMRI data showed that CFC mediated the effect of callosal FA on BPA. These findings clarify the mechanisms by which the hemispheres interact to perform complex cognitive tasks.

Spatial and temporal scales of neuronal correlation in visual area V4.

The spiking activity of nearby cortical neurons is correlated on both short and long time scales. Understanding this shared variability in firing patterns is critical for appreciating the representation of sensory stimuli in ensembles of neurons, the coincident influences of neurons on common targets, and the functional implications of microcircuitry. Our knowledge about neuronal correlations, however, derives largely from experiments that used different recording methods, analysis techniques, and cortical regions. Here we studied the structure of neuronal correlation in area V4 of alert macaques using recording and analysis procedures designed to match those used previously in primary visual cortex (V1), the major input to V4. We found that the spatial and temporal properties of correlations in V4 were remarkably similar to those of V1, with two notable differences: correlated variability in V4 was approximately one-third the magnitude of that in V1 and synchrony in V4 was less temporally precise than in V1. In both areas, spontaneous activity (measured during fixation while viewing a blank screen) was approximately twice as correlated as visual-evoked activity. The results provide a foundation for understanding how the structure of neuronal correlation differs among brain regions and stages in cortical processing and suggest that it is likely governed by features of neuronal circuits that are shared across the visual cortex.

Division of labor in frontal eye field neurons during presaccadic remapping of visual receptive fields.

Our percept of visual stability across saccadic eye movements may be mediated by presaccadic remapping. Just before a saccade, neurons that remap become visually responsive at a future field (FF), which anticipates the saccade vector. Hence, the neurons use corollary discharge of saccades. Many of the neurons also decrease their response at the receptive field (RF). Presaccadic remapping occurs in several brain areas including the frontal eye field (FEF), which receives corollary discharge of saccades in its layer IV from a collicular-thalamic pathway. We studied, at two levels, the microcircuitry of remapping in the FEF. At the laminar level, we compared remapping between layers IV and V. At the cellular level, we compared remapping between different neuron types of layer IV. In the FEF in four monkeys (Macaca mulatta), we identified 27 layer IV neurons with orthodromic stimulation and 57 layer V neurons with antidromic stimulation from the superior colliculus. With the use of established criteria, we classified the layer IV neurons as putative excitatory (n = 11), putative inhibitory (n = 12), or ambiguous (n = 4). We found that just before a saccade, putative excitatory neurons increased their visual response at the RF, putative inhibitory neurons showed no change, and ambiguous neurons increased their visual response at the FF. None of the neurons showed presaccadic visual changes at both RF and FF. In contrast, neurons in layer V showed full remapping (at both the RF and FF). Our data suggest that elemental signals for remapping are distributed across neuron types in early cortical processing and combined in later stages of cortical microcircuitry.

Frontal eye field neurons assess visual stability across saccades.

The image on the retina may move because the eyes move, or because something in the visual scene moves. The brain is not fooled by this ambiguity. Even as we make saccades, we are able to detect whether visual objects remain stable or move. Here we test whether this ability to assess visual stability across saccades is present at the single-neuron level in the frontal eye field (FEF), an area that receives both visual input and information about imminent saccades. Our hypothesis was that neurons in the FEF report whether a visual stimulus remains stable or moves as a saccade is made. Monkeys made saccades in the presence of a visual stimulus outside of the receptive field. In some trials, the stimulus remained stable, but in other trials, it moved during the saccade. In every trial, the stimulus occupied the center of the receptive field after the saccade, thus evoking a reafferent visual response. We found that many FEF neurons signaled, in the strength and timing of their reafferent response, whether the stimulus had remained stable or moved. Reafferent responses were tuned for the amount of stimulus translation, and, in accordance with human psychophysics, tuning was better (more prevalent, stronger, and quicker) for stimuli that moved perpendicular, rather than parallel, to the saccade. Tuning was sometimes present as well for nonspatial transaccadic changes (in color, size, or both). Our results indicate that FEF neurons evaluate visual stability during saccades and may be general purpose detectors of transaccadic visual change.

Metacognition in monkeys during an oculomotor task.

This study investigated whether rhesus monkeys show evidence of metacognition in a reduced, visual oculomotor task that is particularly suitable for use in fMRI and electrophysiology. The 2-stage task involved punctate visual stimulation and saccadic eye movement responses. In each trial, monkeys made a decision and then made a bet. To earn maximum reward, they had to monitor their decision and use that information to bet advantageously. Two monkeys learned to base their bets on their decisions within a few weeks. We implemented an operational definition of metacognitive behavior that relied on trial-by-trial analyses and signal detection theory. Both monkeys exhibited metacognition according to these quantitative criteria. Neither external visual cues nor potential reaction time cues explained the betting behavior; the animals seemed to rely exclusively on internal traces of their decisions. We documented the learning process of one monkey. During a 10-session transition phase, betting switched from random to a decision-based strategy. The results reinforce previous findings of metacognitive ability in monkeys and may facilitate the neurophysiological investigation of metacognitive functions.

A Dynamic Directional Model for Effective Brain Connectivity using Electrocorticographic (ECoG) Time Series.

We introduce a dynamic directional model (DDM) for studying brain effective connectivity based on intracranial electrocorticographic (ECoG) time series. The DDM consists of two parts: a set of differential equations describing neuronal activity of brain components (state equations), and observation equations linking the underlying neuronal states to observed data. When applied to functional MRI or EEG data, DDMs usually have complex formulations and thus can accommodate only a few regions, due to limitations in spatial resolution and/or temporal resolution of these imaging modalities. In contrast, we formulate our model in the context of ECoG data. The combined high temporal and spatial resolution of ECoG data result in a much simpler DDM, allowing investigation of complex connections between many regions. To identify functionally segregated sub-networks, a form of biologically economical brain networks, we propose the Potts model for the DDM parameters. The neuronal states of brain components are represented by cubic spline bases and the parameters are estimated by minimizing a log-likelihood criterion that combines the state and observation equations. The Potts model is converted to the Potts penalty in the penalized regression approach to achieve sparsity in parameter estimation, for which a fast iterative algorithm is developed. The methods are applied to an auditory ECoG dataset.

A Diffusion MRI Tractography Connectome of the Mouse Brain and Comparison with Neuronal Tracer Data.

Interest in structural brain connectivity has grown with the understanding that abnormal neural connections may play a role in neurologic and psychiatric diseases. Small animal connectivity mapping techniques are particularly important for identifying aberrant connectivity in disease models. Diffusion magnetic resonance imaging tractography can provide nondestructive, 3D, brain-wide connectivity maps, but has historically been limited by low spatial resolution, low signal-to-noise ratio, and the difficulty in estimating multiple fiber orientations within a single image voxel. Small animal diffusion tractography can be substantially improved through the combination of ex vivo MRI with exogenous contrast agents, advanced diffusion acquisition and reconstruction techniques, and probabilistic fiber tracking. Here, we present a comprehensive, probabilistic tractography connectome of the mouse brain at microscopic resolution, and a comparison of these data with a neuronal tracer-based connectivity data from the Allen Brain Atlas. This work serves as a reference database for future tractography studies in the mouse brain, and demonstrates the fundamental differences between tractography and neuronal tracer data.

Localization of Metal Electrodes in the Intact Rat Brain Using Registration of 3D Microcomputed Tomography Images to a Magnetic Resonance Histology Atlas.

Simultaneous neural recordings taken from multiple areas of the rodent brain are garnering growing interest due to the insight they can provide about spatially distributed neural circuitry. The promise of such recordings has inspired great progress in methods for surgically implanting large numbers of metal electrodes into intact rodent brains. However, methods for localizing the precise location of these electrodes have remained severely lacking. Traditional histological techniques that require slicing and staining of physical brain tissue are cumbersome, and become increasingly impractical as the number of implanted electrodes increases. Here we solve these problems by describing a method that registers 3-D computerized tomography (CT) images of intact rat brains implanted with metal electrode bundles to a Magnetic Resonance Imaging Histology (MRH) Atlas. Our method allows accurate visualization of each electrode bundle's trajectory and location without removing the electrodes from the brain or surgically implanting external markers. In addition, unlike physical brain slices, once the 3D images of the electrode bundles and the MRH atlas are registered, it is possible to verify electrode placements from many angles by "re-slicing" the images along different planes of view. Further, our method can be fully automated and easily scaled to applications with large numbers of specimens. Our digital imaging approach to efficiently localizing metal electrodes offers a substantial addition to currently available methods, which, in turn, may help accelerate the rate at which insights are gleaned from rodent network neuroscience.

Quantitative mapping of trimethyltin injury in the rat brain using magnetic resonance histology.

The growing exposure to chemicals in our environment and the increasing concern over their impact on health have elevated the need for new methods for surveying the detrimental effects of these compounds. Today's gold standard for assessing the effects of toxicants on the brain is based on hematoxylin and eosin (H&E)-stained histology, sometimes accompanied by special stains or immunohistochemistry for neural processes and myelin. This approach is time-consuming and is usually limited to a fraction of the total brain volume. We demonstrate that magnetic resonance histology (MRH) can be used for quantitatively assessing the effects of central nervous system toxicants in rat models. We show that subtle and sparse changes to brain structure can be detected using magnetic resonance histology, and correspond to some of the locations in which lesions are found by traditional pathological examination. We report for the first time diffusion tensor image-based detection of changes in white matter regions, including fimbria and corpus callosum, in the brains of rats exposed to 8 mg/kg and 12 mg/kg trimethyltin. Besides detecting brain-wide changes, magnetic resonance histology provides a quantitative assessment of dose-dependent effects. These effects can be found in different magnetic resonance contrast mechanisms, providing multivariate biomarkers for the same spatial location. In this study, deformation-based morphometry detected areas where previous studies have detected cell loss, while voxel-wise analyses of diffusion tensor parameters revealed microstructural changes due to such things as cellular swelling, apoptosis, and inflammation. Magnetic resonance histology brings a valuable addition to pathology with the ability to generate brain-wide quantitative parametric maps for markers of toxic insults in the rodent brain.

Inhibition-Induced Forgetting Results from Resource Competition between Response Inhibition and Memory Encoding Processes.

UNLABELLED: Response inhibition is a key component of executive control, but its relation to other cognitive processes is not well understood. We recently documented the "inhibition-induced forgetting effect": no-go cues are remembered more poorly than go cues. We attributed this effect to central-resource competition, whereby response inhibition saps attention away from memory encoding. However, this proposal is difficult to test with behavioral means alone. We therefore used fMRI in humans to test two neural predictions of the "common resource hypothesis": (1) brain regions associated with response inhibition should exhibit greater resource demands during encoding of subsequently forgotten than remembered no-go cues; and (2) this higher inhibitory resource demand should lead to memory encoding regions having less resources available during encoding of subsequently forgotten no-go cues. Participants categorized face stimuli by gender in a go/no-go task and, following a delay, performed a surprise recognition memory test for those faces. Replicating previous findings, memory was worse for no-go than for go stimuli. Crucially, forgetting of no-go cues was predicted by high inhibitory resource demand, as quantified by the trial-by-trial ratio of activity in neural "no-go" versus "go" networks. Moreover, this index of inhibitory demand exhibited an inverse trial-by-trial relationship with activity in brain regions responsible for the encoding of no-go cues into memory, notably the ventrolateral prefrontal cortex. This seesaw pattern between the neural resource demand of response inhibition and activity related to memory encoding directly supports the hypothesis that response inhibition temporarily saps attentional resources away from stimulus processing. SIGNIFICANCE STATEMENT: Recent behavioral experiments showed that inhibiting a motor response to a stimulus (a "no-go cue") impairs subsequent memory for that cue. Here, we used fMRI to test whether this "inhibition-induced forgetting effect" is caused by competition for neural resources between the processes of response inhibition and memory encoding. We found that trial-by-trial variations in neural inhibitory resource demand predicted subsequent forgetting of no-go cues and that higher inhibitory demand was furthermore associated with lower concurrent activation in brain regions responsible for successful memory encoding of no-go cues. Thus, motor inhibition and stimulus encoding appear to compete with each other: when more resources have to be devoted to inhibiting action, less are available for encoding sensory stimuli.

When Working Memory and Attention Compete: Characterizing the Dynamic Interdependence between our Mental Workspace and External Environment

All of us are taxed with juggling our inner mental lives with immediate external task demands. For many years, the temporary maintenance of internal information was considered to be handled by a dedicated working memory (WM) system. It has recently become increasingly clear, however, that such short-term internal activation interacts with attention focused on external stimuli. It is unclear, however, exactly why these two interact, at what level of processing, and to what degree. Because our internal maintenance and external attention processes co-occur with one another, the manner of their interaction has vast implications for functioning in daily life. The work described here has employed original experimental paradigms combining WM and attention task elements, functional magnetic resonance imaging (fMRI) to illuminate the associated neural processes, and transcranial magnetic stimulation (TMS) to clarify the causal substrates of attentional brain function. These studies have examined a mechanism that might explain why (and when) the content of WM can involuntarily capture visual attention. They have, furthermore, tested whether fundamental attentional selection processes operate within WM, and whether they are reciprocal with attention. Finally, they have illuminated the neural consequences of competing attentional demands. The findings indicate that WM shares representations, operating principles, and cognitive resources with externally-oriented attention.

The role of stimulus salience and attentional capture across the neural hierarchy in a stop-signal task.

Inhibitory motor control is a core function of cognitive control. Evidence from diverse experimental approaches has linked this function to a mostly right-lateralized network of cortical and subcortical areas, wherein a signal from the frontal cortex to the basal ganglia is believed to trigger motor-response cancellation. Recently, however, it has been recognized that in the context of typical motor-control paradigms those processes related to actual response inhibition and those related to the attentional processing of the relevant stimuli are highly interrelated and thus difficult to distinguish. Here, we used fMRI and a modified Stop-signal task to specifically examine the role of perceptual and attentional processes triggered by the different stimuli in such tasks, thus seeking to further distinguish other cognitive processes that may precede or otherwise accompany the implementation of response inhibition. In order to establish which brain areas respond to sensory stimulation differences by rare Stop-stimuli, as well as to the associated attentional capture that these may trigger irrespective of their task-relevance, we compared brain activity evoked by Stop-trials to that evoked by Go-trials in task blocks where Stop-stimuli were to be ignored. In addition, region-of-interest analyses comparing the responses to these task-irrelevant Stop-trials, with those to typical relevant Stop-trials, identified separable activity profiles as a function of the task-relevance of the Stop-signal. While occipital areas were mostly blind to the task-relevance of Stop-stimuli, activity in temporo-parietal areas dissociated between task-irrelevant and task-relevant ones. Activity profiles in frontal areas, in turn, were activated mainly by task-relevant Stop-trials, presumably reflecting a combination of triggered top-down attentional influences and inhibitory motor-control processes.