Studies of working memory in interrupted human-computer interaction

Distraction in the workplace is increasingly more common in the information age. Several tasks and sources of information compete for a worker's limited cognitive capacities in human-computer interaction (HCI). In some situations even very brief interruptions can have detrimental effects on memory. Nevertheless, in other situations where persons are continuously interrupted, virtually no interruption costs emerge. This dissertation attempts to reveal the mental conditions and causalities differentiating the two outcomes. The explanation, building on the theory of long-term working memory (LTWM; Ericsson and Kintsch, 1995), focuses on the active, skillful aspects of human cognition that enable the storage of task information beyond the temporary and unstable storage provided by short-term working memory (STWM). Its key postulate is called a retrieval structure an abstract, hierarchical knowledge representation built into long-term memory that can be utilized to encode, update, and retrieve products of cognitive processes carried out during skilled task performance. If certain criteria of practice and task processing are met, LTWM allows for the storage of large representations for long time periods, yet these representations can be accessed with the accuracy, reliability, and speed typical of STWM. The main thesis of the dissertation is that the ability to endure interruptions depends on the efficiency in which LTWM can be recruited for maintaing information. An observational study and a field experiment provide ecological evidence for this thesis. Mobile users were found to be able to carry out heavy interleaving and sequencing of tasks while interacting, and they exhibited several intricate time-sharing strategies to orchestrate interruptions in a way sensitive to both external and internal demands. Interruptions are inevitable, because they arise as natural consequences of the top-down and bottom-up control of multitasking. In this process the function of LTWM is to keep some representations ready for reactivation and others in a more passive state to prevent interference. The psychological reality of the main thesis received confirmatory evidence in a series of laboratory experiments. They indicate that after encoding into LTWM, task representations are safeguarded from interruptions, regardless of their intensity, complexity, or pacing. However, when LTWM cannot be deployed, the problems posed by interference in long-term memory and the limited capacity of the STWM surface. A major contribution of the dissertation is the analysis of when users must resort to poorer maintenance strategies, like temporal cues and STWM-based rehearsal. First, one experiment showed that task orientations can be associated with radically different patterns of retrieval cue encodings. Thus the nature of the processing of the interface determines which features will be available as retrieval cues and which must be maintained by other means. In another study it was demonstrated that if the speed of encoding into LTWM, a skill-dependent parameter, is slower than the processing speed allowed for by the task, interruption costs emerge. Contrary to the predictions of competing theories, these costs turned out to involve intrusions in addition to omissions. Finally, it was learned that in rapid visually oriented interaction, perceptual-procedural expectations guide task resumption, and neither STWM nor LTWM are utilized due to the fact that access is too slow. These findings imply a change in thinking about the design of interfaces. Several novel principles of design are presented, basing on the idea of supporting the deployment of LTWM in the main task.

ICT and the Shaping of Society: Exploring Human -ICT Relationships in Everyday Life (summary section only)

This thesis explores the relationship between humans and ICTs (information and communication technologies). As ICTs are increasingly penetrating all spheres of social life, their role as mediators – between people, between people and information, and even between people and the natural world – is expanding, and they are increasingly shaping social life. Yet, we still know little of how our life is affected by their growing role. Our understanding of the actors and forces driving the accelerating adoption of new ICTs in all areas of life is also fairly limited. This thesis addresses these problems by interpretively exploring the link between ICTs and the shaping of society at home, in the office, and in the community. The thesis builds on empirical material gathered in three research projects, presented in four separate essays. The first project explores computerized office work through a case study. The second is a regional development project aiming at increasing ICT knowledge and use in 50 small-town families. In the third, the second project is compared to three other longitudinal development projects funded by the European Union. Using theories that consider the human-ICT relationship as intertwined, the thesis provides a multifaceted description of life with ICTs in contemporary information society. By oscillating between empirical and theoretical investigations and balancing between determinist and constructivist conceptualisations of the human-ICT relationship, I construct a dialectical theoretical framework that can be used for studying socio-technical contexts in society. This framework helps us see how societal change stems from the complex social processes that surround routine everyday actions. For example, interacting with and through ICTs may change individuals’ perceptions of time and space, social roles, and the proper ways to communicate – changes which at some point in time result in societal change in terms of, for example, new ways of acting and knowing things.

Psychologically-Based and Content-Oriented Experience in Entertainment Virtual Environments

In daily life, rich experiences evolve in every environmental and social interaction. Because experience has a strong impact on how people behave, scholars in different fields are interested in understanding what constitutes an experience. Yet even if interest in conscious experience is on the increase, there is no consensus on how such experience should be studied. Whatever approach is taken, the subjective and psychologically multidimensional nature of experience should be respected. This study endeavours to understand and evaluate conscious experiences. First I intro-duce a theoretical approach to psychologically-based and content-oriented experience. In the experiential cycle presented here, classical psychology and orienting-environmental content are connected. This generic approach is applicable to any human-environment interaction. Here I apply the approach to entertainment virtual environments (VEs) such as digital games and develop a framework with the potential for studying experiences in VEs. The development of the methodological framework included subjective and objective data from experiences in the Cave Automatic Virtual Environment (CAVE) and with numerous digital games (N=2,414). The final framework consisted of fifteen factor-analytically formed subcomponents of the sense of presence, involvement and flow. Together, these show the multidimensional experiential profile of VEs. The results present general experiential laws of VEs and show that the interface of a VE is related to (physical) presence, which psychologically means attention, perception and the cognitively evaluated realness and spatiality of the VE. The narrative of the VE elicits (social) presence and involvement and affects emotional outcomes. Psychologically, these outcomes are related to social cognition, motivation and emotion. The mechanics of a VE affect the cognitive evaluations and emotional outcomes related to flow. In addition, at the very least, user background, prior experience and use context affect the experiential variation. VEs are part of many peoples lives and many different outcomes are related to them, such as enjoyment, learning and addiction, depending on who is making the evalua-tion. This makes VEs societally important and psychologically fruitful to study. The approach and framework presented here contribute to our understanding of experiences in general and VEs in particular. The research can provide VE developers with a state-of-the art method (www.eveqgp.fi) that can be utilized whenever new product and service concepts are designed, prototyped and tested.

The Role of Genes on Chromosome Locus 4q12 in Human Central Nervous System Tumors

Human central nervous system (CNS) tumors are a heterogeneous group of tumors occurring in brain, brainstem and spinal cord. Malignant gliomas (astrocytic and oligodendroglial tumors), which arise from the neuroepithelial cells are the most common CNS neoplasms in human. Malignant gliomas are highly aggressive and invasive tumors, and have a very poor prognosis. The development and progression of gliomas involve a stepwise accumulation of genetic alterations that generally affect either signal transduction pathways activated by receptor tyrosine kinases (RTKs), or cell cycle arrest pathways. Constitutive activation or deregulated signaling by RTKs is caused by gene amplification, overexpression or mutations. The aberrant RTK signaling results in turn in the activation of several downstream pathways, which ultimately lead to malignant transformation and tumor proliferation. Many genetic abnormalities implicated in nervous system tumors involve the genes located at the chromosomal region 4q12. This locus harbors the receptor tyrosine kinases KIT, PDGFRA and VEGFR2, and other genes (REST, LNX1) with neural function. Gene amplification and protein expression of KIT, PDGFRA, and VEGFR2 was studied using clinical tumor material. REST and LNX1, as well as NUMBL, the interaction partner of LNX1, were studied for their gene mutations and amplifications. In our studies, amplification of LNX1 was associated with KIT and PDGFRA amplification in glioblastomas, and coamplification of KIT, PDGFRA and VEGFR2 was detected in medulloblastomas and CNS primitive neuroectodermal tumors. PDGFRA amplification was also correlated with poor overall survival. Coamplification of KIT, PDGFRA and VEGFR2 was observed in a subset of human astrocytic and oligodendroglial tumors. We suggest that genes at 4q12 could be a part of a larger amplified region, which is deregulated in gliomas, and could be used as a prognostic marker of tumorigenic process. The signaling pathways activated due to gene amplifications, activating gene mutations, and overexpressed proteins may be useful as therapeutic targets for glioma treatment. This study also includes the characterization of KIT overexpressing astrocytes, analyzed by various in vitro functional assays. Our results show that overexpression of KIT in mouse astrocytes promotes cell proliferation and anchorage-independent growth, as well as phenotypic changes in the cells. Furthermore, the increased proliferation is partly inhibited by imatinib, a small molecule inhibitor of KIT. These results suggest that KIT may play a role in astrocyte growth regulation, and might have an oncogenic role in brain tumorigenesis. Elucidation of the altered signaling pathways due to specific gene amplifications, activating gene mutations, and overexpressed proteins may be useful as therapeutic targets for glioma treatment.

Social perception and cognition : processing of gestures, postures and facial expressions in the human brain

Humans are a social species with the internal capability to process social information from other humans. To understand others behavior and to react accordingly, it is necessary to infer their internal states, emotions and aims, which are conveyed by subtle nonverbal bodily cues such as postures, gestures, and facial expressions. This thesis investigates the brain functions underlying the processing of such social information. Studies I and II of this thesis explore the neural basis of perceiving pain from another person s facial expressions by means of functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). In Study I, observing another s facial expression of pain activated the affective pain system (previously associated with self-experienced pain) in accordance with the intensity of the observed expression. The strength of the response in anterior insula was also linked to the observer s empathic abilities. The cortical processing of facial pain expressions advanced from the visual to temporal-lobe areas at similar latencies (around 300 500 ms) to those previously shown for emotional expressions such as fear or disgust. Study III shows that perceiving a yawning face is associated with middle and posterior STS activity, and the contagiousness of a yawn correlates negatively with amygdalar activity. Study IV explored the brain correlates of interpreting social interaction between two members of the same species, in this case human and canine. Observing interaction engaged brain activity in very similar manner for both species. Moreover, the body and object sensitive brain areas of dog experts differentiated interaction from noninteraction in both humans and dogs whereas in the control subjects, similar differentiation occurred only for humans. Finally, Study V shows the engagement of the brain area associated with biological motion when exposed to the sounds produced by a single human being walking. However, more complex pattern of activation, with the walking sounds of several persons, suggests that as the social situation becomes more complex so does the brain response. Taken together, these studies demonstrate the roles of distinct cortical and subcortical brain regions in the perception and sharing of others internal states via facial and bodily gestures, and the connection of brain responses to behavioral attributes.

NMR Spectroscopy of Multi-domain Proteins : Immunoglobulin-like Domains of Human Filamin A

Proteins are complex biomacromolecules playing fundamental roles in the physiological processes of all living organisms. They function as structural units, enzymes, transporters, process regulators, and signal transducers. Defects in protein functions often derive from genetic mutations altering the protein structure, and impairment of essential protein functions manifests itself as pathological conditions. Proteins operate through interactions, and all protein functions depend on protein structure. In order to understand biological mechanisms at the molecular level, one has to know the structures of the proteins involved. This thesis covers structural and functional characterization of human filamins. Filamins are actin-binding and -bundling proteins that have numerous interaction partners. In addition to their actin-organizing functions, filamins are also known to have roles in cell adhesion and locomotion, and to participate in the logistics of cell membrane receptors, and in the coordination of intracellular signaling pathways. Filamin mutations in humans induce severe pathological conditions affecting the brain, bones, limbs, and the cardiovascular system. Filamins are large modular proteins composed of an N-terminal actin-binding domain and 24 consecutive immunoglobulin-like domains (IgFLNs). Nuclear magnetic resonance (NMR) spectroscopy is a versatile method of gaining insight into protein structure, dynamics and interactions. NMR spectroscopy was employed in this thesis to study the atomic structure and interaction mechanisms of C-terminal IgFLNs, which are known to house the majority of the filamin interaction sites. The structures of IgFLN single-domains 17 and 23 and IgFLN domain pairs 16-17 and 18-19 were determined using NMR spectroscopy. The structures of domain pairs 16 17 and 18 19 both revealed novel domain domain interaction modes of IgFLNs. NMR titrations were employed to characterize the interactions of filamins with glycoprotein Ibα, FilGAP, integrin β7 and dopamine receptors. Domain packing of IgFLN domain sextet 16 21 was further characterized using residual dipolar couplings and NMR relaxation analysis. This thesis demonstrates the versatility and potential of NMR spectroscopy in structural and functional studies of multi-domain proteins.

Microbe-induced Innate Immune Responses in Human Dendritic Cells

Two types of antigen-presenting cells (APCs), macrophages and dendritic cells (DCs), function at the interface of innate and adaptive immunity. Through recognition of conserved microbial patterns, they are able to detect the invading pathogens. This leads to activation of signal transduction pathways that in turn induce gene expression of various molecules required for immune responses and eventually pathogen clearance. Cytokines are among the genes induced upon detection of microbes. They play an important role in regulating host immune responses during microbial infection. Chemotactic cytokines, chemokines, are involved in migratory events of immune cells. Cytokines also promote the differentiation of distinct T cell responses. Because of the multiple roles of cytokines in the immune system, the cytokine network needs to be tightly regulated. In this work, the induction of innate immune responses was studied using human primary macrophages or DCs as cell models. Salmonella enterica serovar Typhimurium served as a model for an intracellular bacterium, whereas Sendai virus was used in virus experiments. The starting point of this study was that DCs of mouse origin had recently been characterized as host cells for Salmonella. However, only little was known about the immune responses initiated in Salmonella-infected human DCs. Thus, cellular responses of macrophages and DCs, in particular the pattern of cytokine production, to Salmonella infection were compared. Salmonella-induced macrophages and DCs were found to produce multiple cytokines including interferon (IFN) -gamma, which is conventionally produced by T and natural killer (NK) cells. Both macrophages and DCs also promoted the intracellular survival of the bacterium. Phenotypic maturation of DCs as characterized by upregulation of costimulatory and human leukocyte antigen (HLA) molecules, and production of CCL19 chemokine, were also detected upon infection with Salmonella. Another focus of this PhD work was to unravel the regulatory events controlling the expression of cytokine genes encoding for CCL19 and type III IFNs, which are central to DC biology. We found that the promoters of CCL19 and type III IFNs contain similar regulatory elements that bind nuclear factor kappaB (NF-kappaB) and interferon regulatory factors (IRFs), which could mediate transcriptional activation of the genes. The regulation of type III IFNs in virus infection resembled that of type I IFNs a cytokine class traditionally regarded as antiviral. The induction of type I and type III IFNs was also observed in response to bacterial infection. Taken together, this work identifies new details about the interaction of Salmonella with its phagocytic host cells of human origin. In addition, studies provide information on the regulatory events controlling the expression of CCL19 and the most recently identified IFN family genes, type III IFN genes.

Outer Membrane Protease/Adhesin PgtE of Salmonella enterica: Role in Salmonella-Host Interaction

Salmonella enterica serovar Typhimurium is a common cause of gastroenteritis in humans and, occasionally, also causes systemic infection. During systemic infection an important characteristic of Salmonella is its ability to survive and replicate within macrophages. The outer membrane protease PgtE of S. enterica is a member of the omptin family of outer membrane aspartate proteases, which are beta-barrel proteins with five surface-exposed loops. The main goals of this study were to characterize biological substrates and pathogenesis-associated functions of PgtE and to determine the conditions where PgtE is fully active. In this study we found that PgtE requires rough lipopolysaccharide (LPS) to be functional but is sterically inhibited by the long O-antigen side chain in smooth LPS. Salmonella isolates normally are smooth with a long oligosaccharide O-antigen, and PgtE remains functionally cryptic in wild-type Salmonella cultivated in vitro. Interestingly, our results showed that due to increased expression of PgtE and to reduced length of the LPS O-antigen chains, the wild-type Salmonella expresses highly functional PgtE when isolated from mouse macrophage-like J774A.1 cells. Salmonella is thought to be continuously released from macrophages to infect new ones, and our results suggest that PgtE is functional during these transient extracellular growth phases. Six novel host protein substrates were identified for PgtE in this work. PgtE was previously known to activate human plasminogen (Plg) to plasmin, a broad-spectrum serine protease, and in this study PgtE was shown to interfere with the Plg system by inactivating the main inhibitor of plasmin, alpha2-antiplasmin. PgtE also interferes with another important proteolytic system of mammals by activating pro-matrix metalloproteinase-9 to an active gelatinase. PgtE also directly degrades gelatin, a component of extracellular matrices. PgtE also increases bacterial resistance against complement-mediated killing in human serum and enhances survival of Salmonella within murine macrophages as well as in the liver and spleen of intraperitoneally infected mice. Taken together, the results in this study suggest that PgtE is a virulence factor of Salmonella that has adapted to interfere with host proteolytic systems and to modify extracellular matrix; these features likely assist the migration of Salmonella during systemic salmonellosis.

Tactile processing in human somatosensory and auditory cortices

Tactile sensation plays an important role in everyday life. While the somatosensory system has been studied extensively, the majority of information has come from studies using animal models. Recent development of high-resolution anatomical and functional imaging techniques has enabled the non-invasive study of human somatosensory cortex and thalamus. This thesis provides new insights into the functional organization of the human brain areas involved in tactile processing using magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). The thesis also demonstrates certain optimizations of MEG and fMRI methods. Tactile digit stimulation elicited stimulus-specific responses in a number of brain areas. Contralateral activation was observed in somatosensory thalamus (Study II), primary somatosensory cortex (SI; I, III, IV), and post-auditory belt area (III). Bilateral activation was observed in secondary somatosensory cortex (SII; II, III, IV). Ipsilateral activation was found in the post-central gyrus (area 2 of SI cortex; IV). In addition, phasic deactivation was observed within ipsilateral SI cortex and bilateral primary motor cortex (IV). Detailed investigation of the tactile responses demonstrated that the arrangement of distal-proximal finger representations in area 3b of SI in humans is similar to that found in monkeys (I). An optimized MEG approach was sufficient to resolve such fine detail in functional organization. The SII region appeared to contain double representations for fingers and toes (II). The detection of activations in the SII region and thalamus improved at the individual and group levels when cardiac-gated fMRI was used (II). Better detection of body part representations at the individual level is an important improvement, because identification of individual representations is crucial for studying brain plasticity in somatosensory areas. The posterior auditory belt area demonstrated responses to both auditory and tactile stimuli (III), implicating this area as a physiological substrate for the auditory-tactile interaction observed in earlier psychophysical studies. Comparison of different smoothing parameters (III) demonstrated that proper evaluation of co-activation should be based on individual subject analysis with minimal or no smoothing. Tactile input consistently influenced area 3b of the human ipsilateral SI cortex (IV). The observed phasic negative fMRI response is proposed to result from interhemispheric inhibition via trans-callosal connections. This thesis contributes to a growing body of human data suggesting that processing of tactile stimuli involves multiple brain areas, with different spatial patterns of cortical activation for different stimuli.