When Working Memory and Attention Compete: Characterizing the Dynamic Interdependence between our Mental Workspace and External Environment

All of us are taxed with juggling our inner mental lives with immediate external task demands. For many years, the temporary maintenance of internal information was considered to be handled by a dedicated working memory (WM) system. It has recently become increasingly clear, however, that such short-term internal activation interacts with attention focused on external stimuli. It is unclear, however, exactly why these two interact, at what level of processing, and to what degree. Because our internal maintenance and external attention processes co-occur with one another, the manner of their interaction has vast implications for functioning in daily life. The work described here has employed original experimental paradigms combining WM and attention task elements, functional magnetic resonance imaging (fMRI) to illuminate the associated neural processes, and transcranial magnetic stimulation (TMS) to clarify the causal substrates of attentional brain function. These studies have examined a mechanism that might explain why (and when) the content of WM can involuntarily capture visual attention. They have, furthermore, tested whether fundamental attentional selection processes operate within WM, and whether they are reciprocal with attention. Finally, they have illuminated the neural consequences of competing attentional demands. The findings indicate that WM shares representations, operating principles, and cognitive resources with externally-oriented attention.

Memory-Based Attentional Guidance: A Window to the Relationship between Working Memory and Attention

Attention, the cognitive means by which we prioritize the processing of a subset of information, is necessary for operating efficiently and effectively in the world. Thus, a critical theoretical question is how information is selected. In the visual domain, working memory (WM)—which refers to the short-term maintenance and manipulation of information that is no longer accessible by the senses—has been highlighted as an important determinant of what is selected by visual attention. Furthermore, although WM and attention have traditionally been conceived as separate cognitive constructs, an abundance of behavioral and neural evidence indicates that these two domains are in fact intertwined and overlapping. The aim of this dissertation is to better understand the nature of WM and attention, primarily through the phenomenon of memory-based attentional guidance, whereby the active maintenance of items in visual WM reliably biases the deployment of attention to memory-matching items in the visual environment. The research presented here employs a combination of behavioral, functional imaging, and computational modeling techniques that address: (1) WM guidance effects with respect to the traditional dichotomy of top-down versus bottom-up attentional control; (2) under what circumstances the contents of WM impact visual attention; and (3) the broader hypothesis of a predictive and competitive interaction between WM and attention. Collectively, these empirical findings reveal the importance of WM as a distinct factor in attentional control and support current models of multiple-state WM, which may have broader implications for how we select and maintain information.

Memory and coping with stress: the relationship between cognitive-emotional distinctiveness, memory valence, and distress.

Cognitive-emotional distinctiveness (CED), the extent to which an individual separates emotions from an event in the cognitive representation of the event, was explored in four studies. CED was measured using a modified multidimensional scaling procedure. The first study found that lower levels of CED in memories of the September 11 terrorist attacks predicted greater frequency of intrusive thoughts about the attacks. The second study revealed that CED levels are higher in negative events, in comparison to positive events and that low CED levels in emotionally intense negative events are associated with a pattern of greater event-related distress. The third study replicated the findings from the previous study when examining CED levels in participants' memories of the 2004 Presidential election. The fourth study revealed that low CED in emotionally intense negative events is associated with worse mental health. We argue that CED is an adaptive and healthy coping feature of stressful memories.

H.M.'s personal crossword puzzles: understanding memory and language.

The amnesic patient H.M. has been solving crossword puzzles nearly all his life. Here, we analysed the linguistic content of 277 of H.M.'s crossword-puzzle solutions. H.M. did not have any unusual difficulties with the orthographic and grammatical components inherent to the puzzles. He exhibited few spelling errors, responded with appropriate parts of speech, and provided answers that were, at times, more convincing to observers than those supplied by the answer keys. These results suggest that H.M.'s lexical word-retrieval skills remain fluid despite his profound anterograde amnesia. Once acquired, the maintenance of written language comprehension and production does not seem to require intact medial temporal lobe structures.

Memory and learning for a novel written style.

Subjects read and recalled a series of five short stories in one of four plot and style combinations. The stories were written in one of two styles that consisted of opposing clause orders (i.e., independent-dependent vs. dependent-independent), tense forms (i.e., past vs. present), and descriptor forms (modifier modifier vs. modifier as a noun). The subjects incorporated both plot and style characteristics into their recalls. Other subjects, who, after five recalls, either generated a new story or listed the rules that had been followed by the stories read, included the marked forms of the characteristics they learned more often, except for tense. The subjects read and recalled four stories of the same plot and style and then read and recalled a fifth story of the same plot and style or of one of the other three plot/style combinations. Ability to switch style depended on both the characteristic and the markedness.

Effects of task instruction on autobiographical memory specificity in young and older adults

Older adults tend to retrieve autobiographical information that is overly general (i.e., not restricted to a single event, termed the overgenerality effect) relative to young adults' specific memories. A vast majority of studies that have reported overgenerality effects explicitly instruct participants to retrieve specific memories, thereby requiring participants to maintain task goals, inhibit inappropriate responses, and control their memory search. Since these processes are impaired in healthy ageing, it is important to determine whether such task instructions influence the magnitude of the overgenerality effect in older adults. In the current study participants retrieved autobiographical memories during presentation of musical clips. Task instructions were manipulated to separate age-related differences in the specificity of underlying memory representations from age-related differences in following task instructions. Whereas young adults modulated memory specificity based on task demands, older adults did not. These findings suggest that reported rates of overgenerality in older adults' memories might include age-related differences in memory representation, as well as differences in task compliance. Such findings provide a better understanding of the underlying cognitive mechanisms involved in age-related changes in autobiographical memory and may also be valuable for future research examining effects of overgeneral memory on general well-being. © 2013 Taylor & Francis.

Effects of task instruction on autobiographical memory specificity in young and older adults.

Older adults tend to retrieve autobiographical information that is overly general (i.e., not restricted to a single event, termed the overgenerality effect) relative to young adults' specific memories. A vast majority of studies that have reported overgenerality effects explicitly instruct participants to retrieve specific memories, thereby requiring participants to maintain task goals, inhibit inappropriate responses, and control their memory search. Since these processes are impaired in healthy ageing, it is important to determine whether such task instructions influence the magnitude of the overgenerality effect in older adults. In the current study participants retrieved autobiographical memories during presentation of musical clips. Task instructions were manipulated to separate age-related differences in the specificity of underlying memory representations from age-related differences in following task instructions. Whereas young adults modulated memory specificity based on task demands, older adults did not. These findings suggest that reported rates of overgenerality in older adults' memories might include age-related differences in memory representation, as well as differences in task compliance. Such findings provide a better understanding of the underlying cognitive mechanisms involved in age-related changes in autobiographical memory and may also be valuable for future research examining effects of overgeneral memory on general well-being.

Imagery and retrieval of auditory and visual information: neural correlates of successful and unsuccessful performance.

Remembering past events - or episodic retrieval - consists of several components. There is evidence that mental imagery plays an important role in retrieval and that the brain regions supporting imagery overlap with those supporting retrieval. An open issue is to what extent these regions support successful vs. unsuccessful imagery and retrieval processes. Previous studies that examined regional overlap between imagery and retrieval used uncontrolled memory conditions, such as autobiographical memory tasks, that cannot distinguish between successful and unsuccessful retrieval. A second issue is that fMRI studies that compared imagery and retrieval have used modality-aspecific cues that are likely to activate auditory and visual processing regions simultaneously. Thus, it is not clear to what extent identified brain regions support modality-specific or modality-independent imagery and retrieval processes. In the current fMRI study, we addressed this issue by comparing imagery to retrieval under controlled memory conditions in both auditory and visual modalities. We also obtained subjective measures of imagery quality allowing us to dissociate regions contributing to successful vs. unsuccessful imagery. Results indicated that auditory and visual regions contribute both to imagery and retrieval in a modality-specific fashion. In addition, we identified four sets of brain regions with distinct patterns of activity that contributed to imagery and retrieval in a modality-independent fashion. The first set of regions, including hippocampus, posterior cingulate cortex, medial prefrontal cortex and angular gyrus, showed a pattern common to imagery/retrieval and consistent with successful performance regardless of task. The second set of regions, including dorsal precuneus, anterior cingulate and dorsolateral prefrontal cortex, also showed a pattern common to imagery and retrieval, but consistent with unsuccessful performance during both tasks. Third, left ventrolateral prefrontal cortex showed an interaction between task and performance and was associated with successful imagery but unsuccessful retrieval. Finally, the fourth set of regions, including ventral precuneus, midcingulate cortex and supramarginal gyrus, showed the opposite interaction, supporting unsuccessful imagery, but successful retrieval performance. Results are discussed in relation to reconstructive, attentional, semantic memory, and working memory processes. This is the first study to separate the neural correlates of successful and unsuccessful performance for both imagery and retrieval and for both auditory and visual modalities.

Event memory: A theory of memory for laboratory, autobiographical, and fictional events.

An event memory is a mental construction of a scene recalled as a single occurrence. It therefore requires the hippocampus and ventral visual stream needed for all scene construction. The construction need not come with a sense of reliving or be made by a participant in the event, and it can be a summary of occurrences from more than one encoding. The mental construction, or physical rendering, of any scene must be done from a specific location and time; this introduces a "self" located in space and time, which is a necessary, but need not be a sufficient, condition for a sense of reliving. We base our theory on scene construction rather than reliving because this allows the integration of many literatures and because there is more accumulated knowledge about scene construction's phenomenology, behavior, and neural basis. Event memory differs from episodic memory in that it does not conflate the independent dimensions of whether or not a memory is relived, is about the self, is recalled voluntarily, or is based on a single encoding with whether it is recalled as a single occurrence of a scene. Thus, we argue that event memory provides a clearer contrast to semantic memory, which also can be about the self, be recalled voluntarily, and be from a unique encoding; allows for a more comprehensive dimensional account of the structure of explicit memory; and better accounts for laboratory and real-world behavioral and neural results, including those from neuropsychology and neuroimaging, than does episodic memory.

The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving.

We sought to map the time course of autobiographical memory retrieval, including brain regions that mediate phenomenological experiences of reliving and emotional intensity. Participants recalled personal memories to auditory word cues during event-related functional magnetic resonance imaging (fMRI). Participants pressed a button when a memory was accessed, maintained and elaborated the memory, and then gave subjective ratings of emotion and reliving. A novel fMRI approach based on timing differences capitalized on the protracted reconstructive process of autobiographical memory to segregate brain areas contributing to initial access and later elaboration and maintenance of episodic memories. The initial period engaged hippocampal, retrosplenial, and medial and right prefrontal activity, whereas the later period recruited visual, precuneus, and left prefrontal activity. Emotional intensity ratings were correlated with activity in several regions, including the amygdala and the hippocampus during the initial period. Reliving ratings were correlated with activity in visual cortex and ventromedial and inferior prefrontal regions during the later period. Frontopolar cortex was the only brain region sensitive to emotional intensity across both periods. Results were confirmed by time-locked averages of the fMRI signal. The findings indicate dynamic recruitment of emotion-, memory-, and sensory-related brain regions during remembering and their dissociable contributions to phenomenological features of the memories.

The neuropsychology of autobiographical memory.

This special issue of Cortex focuses on the relative contribution of different neural networks to memory and the interaction of 'core' memory processes with other cognitive processes. In this article, we examine both. Specifically, we identify cognitive processes other than encoding and retrieval that are thought to be involved in memory; we then examine the consequences of damage to brain regions that support these processes. This approach forces a consideration of the roles of brain regions outside of the frontal, medial-temporal, and diencephalic regions that form a central part of neurobiological theories of memory. Certain kinds of damage to visual cortex or lateral temporal cortex produced impairments of visual imagery or semantic memory; these patterns of impairment are associated with a unique pattern of amnesia that was distinctly different from the pattern associated with medial-temporal trauma. On the other hand, damage to language regions, auditory cortex, or parietal cortex produced impairments of language, auditory imagery, or spatial imagery; however, these impairments were not associated with amnesia. Therefore, a full model of autobiographical memory must consider cognitive processes that are not generally considered 'core processes,' as well as the brain regions upon which these processes depend.

Activation in mesolimbic and visuospatial neural circuits elicited by smoking cues: evidence from functional magnetic resonance imaging.

OBJECTIVE: The authors sought to increase understanding of the brain mechanisms involved in cigarette addiction by identifying neural substrates modulated by visual smoking cues in nicotine-deprived smokers. METHOD: Event-related functional magnetic resonance imaging (fMRI) was used to detect brain activation after exposure to smoking-related images in a group of nicotine-deprived smokers and a nonsmoking comparison group. Subjects viewed a pseudo-random sequence of smoking images, neutral nonsmoking images, and rare targets (photographs of animals). Subjects pressed a button whenever a rare target appeared. RESULTS: In smokers, the fMRI signal was greater after exposure to smoking-related images than after exposure to neutral images in mesolimbic dopamine reward circuits known to be activated by addictive drugs (right posterior amygdala, posterior hippocampus, ventral tegmental area, and medial thalamus) as well as in areas related to visuospatial attention (bilateral prefrontal and parietal cortex and right fusiform gyrus). In nonsmokers, no significant differences in fMRI signal following exposure to smoking-related and neutral images were detected. In most regions studied, both subject groups showed greater activation following presentation of rare target images than after exposure to neutral images. CONCLUSIONS: In nicotine-deprived smokers, both reward and attention circuits were activated by exposure to smoking-related images. Smoking cues are processed like rare targets in that they activate attentional regions. These cues are also processed like addictive drugs in that they activate mesolimbic reward regions.

Olfactory cuing of autobiographical memory.

In Experiment 1, subjects were presented with either the odors or the names of 15 common objects. In Experiment 2, subjects were presented with either the odors, photographs, or names of 16 common objects. All subjects were asked to describe an autobiographical memory evoked by each cue, to date each memory, and to rate each memory on vividness, pleasantness, and the number of times that the memory had been thought of and talked about prior to the experiment. Compared with memories evoked by photographs or names, memories evoked by odors were reported to be thought of and talked about less often prior to the experiment and were more likely to be reported as never having been thought of or talked about prior to the experiment. No other effects were consistently found, though there was a suggestion that odors might evoke more pleasant and emotional memories than other types of cues. The relation of these results to the folklore concerning olfactory cuing is discussed.

Contributions of Dorsal/Ventral Hippocampus and Dorsolateral/Dorsomedial Striatum to Interval Timing

Humans and animals have remarkable capabilities in keeping time and using time as a guide to orient their learning and decision making. Psychophysical models of timing and time perception have been proposed for decades and have received behavioral, anatomical and pharmacological data support. However, despite numerous studies that aimed at delineating the neural underpinnings of interval timing, a complete picture of the neurobiological network of timing in the seconds-to-minutes range remains elusive. Based on classical interval timing protocols and proposing a Timing, Immersive Memory and Emotional Regulation (TIMER) test battery, the author investigates the contributions of the dorsal and ventral hippocampus as well as the dorsolateral and the dorsomedial striatum to interval timing by comparing timing performances in mice after they received cytotoxic lesions in the corresponding brain regions. On the other hand, a timing-based theoretical framework for the emergence of conscious experience that is closely related to the function of the claustrum is proposed so as to serve both biological guidance and the research and evolution of “strong” artificial intelligence. Finally, a new “Double Saturation Model of Interval Timing” that integrates the direct- and indirect- pathways of striatum is proposed to explain the set of empirical findings.

Learning and Exploiting Camera Geometry for Computer Vision

This work explores the use of statistical methods in describing and estimating camera poses, as well as the information feedback loop between camera pose and object detection. Surging development in robotics and computer vision has pushed the need for algorithms that infer, understand, and utilize information about the position and orientation of the sensor platforms when observing and/or interacting with their environment.

The first contribution of this thesis is the development of a set of statistical tools for representing and estimating the uncertainty in object poses. A distribution for representing the joint uncertainty over multiple object positions and orientations is described, called the mirrored normal-Bingham distribution. This distribution generalizes both the normal distribution in Euclidean space, and the Bingham distribution on the unit hypersphere. It is shown to inherit many of the convenient properties of these special cases: it is the maximum-entropy distribution with fixed second moment, and there is a generalized Laplace approximation whose result is the mirrored normal-Bingham distribution. This distribution and approximation method are demonstrated by deriving the analytical approximation to the wrapped-normal distribution. Further, it is shown how these tools can be used to represent the uncertainty in the result of a bundle adjustment problem.

Another application of these methods is illustrated as part of a novel camera pose estimation algorithm based on object detections. The autocalibration task is formulated as a bundle adjustment problem using prior distributions over the 3D points to enforce the objects' structure and their relationship with the scene geometry. This framework is very flexible and enables the use of off-the-shelf computational tools to solve specialized autocalibration problems. Its performance is evaluated using a pedestrian detector to provide head and foot location observations, and it proves much faster and potentially more accurate than existing methods.

Finally, the information feedback loop between object detection and camera pose estimation is closed by utilizing camera pose information to improve object detection in scenarios with significant perspective warping. Methods are presented that allow the inverse perspective mapping traditionally applied to images to be applied instead to features computed from those images. For the special case of HOG-like features, which are used by many modern object detection systems, these methods are shown to provide substantial performance benefits over unadapted detectors while achieving real-time frame rates, orders of magnitude faster than comparable image warping methods.

The statistical tools and algorithms presented here are especially promising for mobile cameras, providing the ability to autocalibrate and adapt to the camera pose in real time. In addition, these methods have wide-ranging potential applications in diverse areas of computer vision, robotics, and imaging.