Studies of working memory in interrupted human-computer interaction

Distraction in the workplace is increasingly more common in the information age. Several tasks and sources of information compete for a worker's limited cognitive capacities in human-computer interaction (HCI). In some situations even very brief interruptions can have detrimental effects on memory. Nevertheless, in other situations where persons are continuously interrupted, virtually no interruption costs emerge. This dissertation attempts to reveal the mental conditions and causalities differentiating the two outcomes. The explanation, building on the theory of long-term working memory (LTWM; Ericsson and Kintsch, 1995), focuses on the active, skillful aspects of human cognition that enable the storage of task information beyond the temporary and unstable storage provided by short-term working memory (STWM). Its key postulate is called a retrieval structure an abstract, hierarchical knowledge representation built into long-term memory that can be utilized to encode, update, and retrieve products of cognitive processes carried out during skilled task performance. If certain criteria of practice and task processing are met, LTWM allows for the storage of large representations for long time periods, yet these representations can be accessed with the accuracy, reliability, and speed typical of STWM. The main thesis of the dissertation is that the ability to endure interruptions depends on the efficiency in which LTWM can be recruited for maintaing information. An observational study and a field experiment provide ecological evidence for this thesis. Mobile users were found to be able to carry out heavy interleaving and sequencing of tasks while interacting, and they exhibited several intricate time-sharing strategies to orchestrate interruptions in a way sensitive to both external and internal demands. Interruptions are inevitable, because they arise as natural consequences of the top-down and bottom-up control of multitasking. In this process the function of LTWM is to keep some representations ready for reactivation and others in a more passive state to prevent interference. The psychological reality of the main thesis received confirmatory evidence in a series of laboratory experiments. They indicate that after encoding into LTWM, task representations are safeguarded from interruptions, regardless of their intensity, complexity, or pacing. However, when LTWM cannot be deployed, the problems posed by interference in long-term memory and the limited capacity of the STWM surface. A major contribution of the dissertation is the analysis of when users must resort to poorer maintenance strategies, like temporal cues and STWM-based rehearsal. First, one experiment showed that task orientations can be associated with radically different patterns of retrieval cue encodings. Thus the nature of the processing of the interface determines which features will be available as retrieval cues and which must be maintained by other means. In another study it was demonstrated that if the speed of encoding into LTWM, a skill-dependent parameter, is slower than the processing speed allowed for by the task, interruption costs emerge. Contrary to the predictions of competing theories, these costs turned out to involve intrusions in addition to omissions. Finally, it was learned that in rapid visually oriented interaction, perceptual-procedural expectations guide task resumption, and neither STWM nor LTWM are utilized due to the fact that access is too slow. These findings imply a change in thinking about the design of interfaces. Several novel principles of design are presented, basing on the idea of supporting the deployment of LTWM in the main task.

Human brain networks of auditory attention and working memory

This thesis examines brain networks involved in auditory attention and auditory working memory using measures of task performance, brain activity, and neuroanatomical connectivity. Auditory orienting and maintenance of attention were compared with visual orienting and maintenance of attention, and top-down controlled attention was compared to bottom-up triggered attention in audition. Moreover, the effects of cognitive load on performance and brain activity were studied using an auditory working memory task. Corbetta and Shulman s (2002) model of visual attention suggests that what is known as the dorsal attention system (intraparietal sulcus/superior parietal lobule, IPS/SPL and frontal eye field, FEF) is involved in the control of top-down controlled attention, whereas what is known as the ventral attention system (temporo-parietal junction, TPJ and areas of the inferior/middle frontal gyrus, IFG/MFG) is involved in bottom-up triggered attention. The present results show that top-down controlled auditory attention also activates IPS/SPL and FEF. Furthermore, in audition, TPJ and IFG/MFG were activated not only by bottom-up triggered attention, but also by top-down controlled attention. In addition, the posterior cerebellum and thalamus were activated by top-down controlled attention shifts and the ventromedial prefrontal cortex (VMPFC) was activated by to-be-ignored, but attention-catching salient changes in auditory input streams. VMPFC may be involved in the evaluation of environmental events causing the bottom-up triggered engagement of attention. Auditory working memory activated a brain network that largely overlapped with the one activated by top-down controlled attention. The present results also provide further evidence of the role of the cerebellum in cognitive processing: During auditory working memory tasks, both activity in the posterior cerebellum (the crus I/II) and reaction speed increased when the cognitive load increased. Based on the present results and earlier theories on the role of the cerebellum in cognitive processing, the function of the posterior cerebellum in cognitive tasks may be related to the optimization of response speed.

Task-dependent activations of human auditory cortex during pitch discrimination and pitch memory tasks

"The functional organization of auditory cortex (AC) is still poorly understood. Previous studies suggest segregation of auditory processing streams for spatial and nonspatial information located in the posterior and anterior AC, respectively (Rauschecker and Tian, 2000; Arnott et al., 2004; Lomber and Malhotra, 2008). Furthermore, previous studies have shown that active listening tasks strongly modulate AC activations (Petkov et al., 2004; Fritz et al., 2005; Polley et al., 2006). However, the task dependence of AC activations has not been systematically investigated. In the present study, we applied high-resolution functional magnetic resonance imaging of the AC and adjacent areas to compare activations during pitch discrimination and n-back pitch memory tasks that were varied parametrically in difficulty. We found that anterior AC activations were increased during discrimination but not during memory tasks, while activations in the inferior parietal lobule posterior to the AC were enhanced during memory tasks but not during discrimination. We also found that wide areas of the anterior AC and anterior insula were strongly deactivated during the pitch memory tasks. While these results are consistent with the proposition that the anterior and posterior AC belong to functionally separate auditory processing streams, our results show that this division is present also between tasks using spatially invariant sounds. Together, our results indicate that activations of human AC are strongly dependent on the characteristics of the behavioral task."

Developmental, functional brain imaging and electrophysiological evidence of visual and auditory working memory

Intact function of working memory (WM) is essential for children and adults to cope with every day life. Children with deficits in WM mechanisms have learning difficulties that are often accompanied by behavioral problems. The neural processes subserving WM, and brain structures underlying this system, continue to develop during childhood till adolescence and young adulthood. With functional magnetic resonance imaging (fMRI) it is possible to investigate the organization and development of WM. The present thesis aimed to investigate, using behavioral and neuroimaging methods, whether mnemonic processing of spatial and nonspatial visual information is segregated in the developing and mature human brain. A further aim in this research was to investigate the organization and development of audiospatial and visuospatial information processing in WM. The behavioral results showed that spatial and nonspatial visual WM processing is segregated in the adult brain. The fMRI result in children suggested that memory load related processing of spatial and nonspatial visual information engages common cortical networks, whereas selective attention to either type of stimuli recruits partially segregated areas in the frontal, parietal and occipital cortices. Deactivation mechanisms that are important in the performance of WM tasks in adults are already operational in healthy school-aged children. Electrophysiological evidence suggested segregated mnemonic processing of visual and auditory location information. The results of the development of audiospatial and visuospatial WM demonstrate that WM performance improves with age, suggesting functional maturation of underlying cognitive processes and brain areas. The development of the performance of spatial WM tasks follows a different time course in boys and girls indicating a larger degree of immaturity in the male than female WM systems. Furthermore, the differences in mastering auditory and visual WM tasks may indicate that visual WM reaches functional maturity earlier than the corresponding auditory system. Spatial WM deficits may underlie some learning difficulties and behavioral problems related to impulsivity, difficulties in concentration, and hyperactivity. Alternatively, anxiety or depressive symptoms may affect WM function and the ability to concentrate, being thus the primary cause of poor academic achievement in children.

Microbe-induced activation of inflammatory cytokine response in human cells

Human body is in continuous contact with microbes. Although many microbes are harmless or beneficial for humans, pathogenic microbes possess a threat to wellbeing. Antimicrobial protection is provided by the immune system, which can be functionally divided into two parts, namely innate and adaptive immunity. The key players of the innate immunity are phagocytic white blood cells such as neutrophils, monocytes, macrophages and dendritic cells (DCs), which constantly monitor the blood and peripheral tissues. These cells are armed for rapid activation upon microbial contact since they express a variety of microbe-recognizing receptors. Macrophages and DCs also act as antigen presenting cells (APCs) and play an important role in the development of adaptive immunity. The development of adaptive immunity requires intimate cooperation between APCs and T lymphocytes and results in microbe-specific immune responses. Moreover, adaptive immunity generates immunological memory, which rapidly and efficiently protects the host from reinfection. Properly functioning immune system requires efficient communication between cells. Cytokines are proteins, which mediate intercellular communication together with direct cell-cell contacts. Immune cells produce inflammatory cytokines rapidly following microbial contact. Inflammatory cytokines modulate the development of local immune response by binding to cell surface receptors, which results in the activation of intracellular signalling and modulates target cell gene expression. One class of inflammatory cytokines chemokines has a major role in regulating cellular traffic. Locally produced inflammatory chemokines guide the recruitment of effector cells to the site of inflammation during microbial infection. In this study two key questions were addressed. First, the ability of pathogenic and non-pathogenic Gram-positive bacteria to activate inflammatory cytokine and chemokine production in different human APCs was compared. In these studies macrophages and DCs were stimulated with pathogenic Steptococcus pyogenes or non-pathogenic Lactobacillus rhamnosus. The second aim of this thesis work was to analyze the role of pro-inflammatory cytokines in the regulation of microbe-induced chemokine production. In these studies bacteria-stimulated macrophages and influenza A virus-infected lung epithelial cells were used as model systems. The results of this study show that although macrophages and DCs share several common antimicrobial functions, these cells have significantly distinct responses against pathogenic and non-pathogenic Gram-positive bacteria. Macrophages were activated in a nearly similar fashion by pathogenic S. pyogenes and non-pathogenic L. rhamnosus. Both bacteria induced the production of similar core set of inflammatory chemokines consisting of several CC-class chemokines and CXCL8. These chemokines attract monocytes, neutrophils, dendritic cells and T cells. Thus, the results suggest that bacteria-activated macrophages efficiently recruit other effector cells to the site of inflammation. Moreover, macrophages seem to be activated by all bacteria irrespective of their pathogenicity. DCs, in contrast, were efficiently activated only by pathogenic S. pyogenes, which induced DC maturation and production of several inflammatory cytokines and chemokines. In contrast, L. rhamnosus-stimulated DCs matured only partially and, most importantly, these cells did not produce inflammatory cytokines or chemokines. L. rhamnosus-stimulated DCs had a phenotype of "semi-mature" DCs and this type of DCs have been suggested to enhance tolerogenic adaptive immune responses. Since DCs have an essential role in the development of adaptive immune response the results suggest that, in contrast to macrophages, DCs may be able to discriminate between pathogenic and non-pathogenic bacteria and thus mount appropriate inflammatory or tolerogenic adaptive immune response depending on the microbe in question. The results of this study also show that pro-inflammatory cytokines can contribute to microbe-induced chemokine production at multiple levels. S. pyogenes-induced type I interferon (IFN) was found to enhance the production of certain inflammatory chemokines in macrophages during bacterial stimulation. Thus, bacteria-induced chemokine production is regulated by direct (microbe-induced) and indirect (pro-inflammatory cytokine-induced) mechanisms during inflammation. In epithelial cells IFN- and tumor necrosis factor- (TNF-) were found to enhance the expression of PRRs and components of cellular signal transduction machinery. Pre-treatment of epithelial cells with these cytokines prior to virus infection resulted in markedly enhanced chemokine response compared to untreated cells. In conclusion, the results obtained from this study show that pro-inflammatory cytokines can enhance microbe-induced chemokine production during microbial infection by providing a positive feedback loop. In addition, pro-inflammatory cytokines can render normally low-responding cells to high chemokine producers via enhancement of microbial detection and signal transduction.

Systems between information and knowledge : In a memory management model of an extended enterprise

The research question of this thesis was how knowledge can be managed with information systems. Information systems can support but not replace knowledge management. Systems can mainly store epistemic organisational knowledge included in content, and process data and information. Certain value can be achieved by adding communication technology to systems. All communication, however, can not be managed. A new layer between communication and manageable information was named as knowformation. Knowledge management literature was surveyed, together with information species from philosophy, physics, communication theory, and information system science. Positivism, post-positivism, and critical theory were studied, but knowformation in extended organisational memory seemed to be socially constructed. A memory management model of an extended enterprise (M3.exe) and knowformation concept were findings from iterative case studies, covering data, information and knowledge management systems. The cases varied from groups towards extended organisation. Systems were investigated, and administrators, users (knowledge workers) and managers interviewed. The model building required alternative sets of data, information and knowledge, instead of using the traditional pyramid. Also the explicit-tacit dichotomy was reconsidered. As human knowledge is the final aim of all data and information in the systems, the distinction between management of information vs. management of people was harmonised. Information systems were classified as the core of organisational memory. The content of the systems is in practice between communication and presentation. Firstly, the epistemic criterion of knowledge is not required neither in the knowledge management literature, nor from the content of the systems. Secondly, systems deal mostly with containers, and the knowledge management literature with applied knowledge. Also the construction of reality based on the system content and communication supports the knowformation concept. Knowformation belongs to memory management model of an extended enterprise (M3.exe) that is divided into horizontal and vertical key dimensions. Vertically, processes deal with content that can be managed, whereas communication can be supported, mainly by infrastructure. Horizontally, the right hand side of the model contains systems, and the left hand side content, which should be independent from each other. A strategy based on the model was defined.