Studies of working memory in interrupted human-computer interaction

Distraction in the workplace is increasingly more common in the information age. Several tasks and sources of information compete for a worker's limited cognitive capacities in human-computer interaction (HCI). In some situations even very brief interruptions can have detrimental effects on memory. Nevertheless, in other situations where persons are continuously interrupted, virtually no interruption costs emerge. This dissertation attempts to reveal the mental conditions and causalities differentiating the two outcomes. The explanation, building on the theory of long-term working memory (LTWM; Ericsson and Kintsch, 1995), focuses on the active, skillful aspects of human cognition that enable the storage of task information beyond the temporary and unstable storage provided by short-term working memory (STWM). Its key postulate is called a retrieval structure an abstract, hierarchical knowledge representation built into long-term memory that can be utilized to encode, update, and retrieve products of cognitive processes carried out during skilled task performance. If certain criteria of practice and task processing are met, LTWM allows for the storage of large representations for long time periods, yet these representations can be accessed with the accuracy, reliability, and speed typical of STWM. The main thesis of the dissertation is that the ability to endure interruptions depends on the efficiency in which LTWM can be recruited for maintaing information. An observational study and a field experiment provide ecological evidence for this thesis. Mobile users were found to be able to carry out heavy interleaving and sequencing of tasks while interacting, and they exhibited several intricate time-sharing strategies to orchestrate interruptions in a way sensitive to both external and internal demands. Interruptions are inevitable, because they arise as natural consequences of the top-down and bottom-up control of multitasking. In this process the function of LTWM is to keep some representations ready for reactivation and others in a more passive state to prevent interference. The psychological reality of the main thesis received confirmatory evidence in a series of laboratory experiments. They indicate that after encoding into LTWM, task representations are safeguarded from interruptions, regardless of their intensity, complexity, or pacing. However, when LTWM cannot be deployed, the problems posed by interference in long-term memory and the limited capacity of the STWM surface. A major contribution of the dissertation is the analysis of when users must resort to poorer maintenance strategies, like temporal cues and STWM-based rehearsal. First, one experiment showed that task orientations can be associated with radically different patterns of retrieval cue encodings. Thus the nature of the processing of the interface determines which features will be available as retrieval cues and which must be maintained by other means. In another study it was demonstrated that if the speed of encoding into LTWM, a skill-dependent parameter, is slower than the processing speed allowed for by the task, interruption costs emerge. Contrary to the predictions of competing theories, these costs turned out to involve intrusions in addition to omissions. Finally, it was learned that in rapid visually oriented interaction, perceptual-procedural expectations guide task resumption, and neither STWM nor LTWM are utilized due to the fact that access is too slow. These findings imply a change in thinking about the design of interfaces. Several novel principles of design are presented, basing on the idea of supporting the deployment of LTWM in the main task.

Oscillatory brain activity in memory disorders

Neuronal oscillations are thought to underlie interactions between distinct brain regions required for normal memory functioning. This study aimed at elucidating the neuronal basis of memory abnormalities in neurodegenerative disorders. Magnetoencephalography (MEG) was used to measure oscillatory brain signals in patients with Alzheimer s disease (AD), a neurodegenerative disease causing progressive cognitive decline, and mild cognitive impairment (MCI), a disorder characterized by mild but clinically significant complaints of memory loss without apparent impairment in other cognitive domains. Furthermore, to help interpret our AD/MCI results and to develop more powerful oscillatory MEG paradigms for clinical memory studies, oscillatory neuronal activity underlying declarative memory, the function which is afflicted first in both AD and MCI, was investigated in a group of healthy subjects. An increased temporal-lobe contribution coinciding with parieto-occipital deficits in oscillatory activity was observed in AD patients: sources in the 6 12.5 Hz range were significantly stronger in the parieto-occipital and significantly weaker in the right temporal region in AD patients, as compared to MCI patients and healthy elderly subjects. Further, the auditory steady-state response, thought to represent both evoked and induced activity, was enhanced in AD patients, as compared to controls, possibly reflecting decreased inhibition in auditory processing and deficits in adaptation to repetitive stimulation with low relevance. Finally, the methodological study revealed that successful declarative encoding and retrieval is associated with increases in occipital gamma and right hemisphere theta power in healthy unmedicated subjects. This result suggests that investigation of neuronal oscillations during cognitive performance could potentially be used to investigate declarative memory deficits in AD patients. Taken together, the present results provide an insight on the role of brain oscillatory activity in memory function and memory disorders.

Storage and retrieval of individual genomes

A repetitive sequence collection is one where portions of a base sequence of length n are repeated many times with small variations, forming a collection of total length N. Examples of such collections are version control data and genome sequences of individuals, where the differences can be expressed by lists of basic edit operations. Flexible and efficient data analysis on a such typically huge collection is plausible using suffix trees. However, suffix tree occupies O(N log N) bits, which very soon inhibits in-memory analyses. Recent advances in full-text self-indexing reduce the space of suffix tree to O(N log σ) bits, where σ is the alphabet size. In practice, the space reduction is more than 10-fold, for example on suffix tree of Human Genome. However, this reduction factor remains constant when more sequences are added to the collection. We develop a new family of self-indexes suited for the repetitive sequence collection setting. Their expected space requirement depends only on the length n of the base sequence and the number s of variations in its repeated copies. That is, the space reduction factor is no longer constant, but depends on N / n. We believe the structures developed in this work will provide a fundamental basis for storage and retrieval of individual genomes as they become available due to rapid progress in the sequencing technologies.

Memory disorders in acute encephalitis : A neuropsychological study

Acute encephalitis is an inflammation of the brain, mostly caused by viral infection. A variety of cognitive symptoms may persist after the acute stage, and neuropsychological assessment is crucial in evaluation of the outcome. The most commonly reported sequelae are memory deficits. The main aims of this study were to investigate the types of memory impairment in various encephalitides, the frequency of global amnesia following encephalitis, and the changes in the deficits during follow-up. Between 1 January 1985 and 31 December 1994, 77 adult patients under the age of 75 with acute encephalitis but without alcohol abuse, or coexisting or previous neurological diseases were consecutively referred for neuropsychological examination at the Department of Neurology, Helsinki University Central Hospital. The aetiology was established in 44/77 (57%) patients; 17 had Herpes simplex virus encephalitis (HSVE). Transient amnesia (TENA) at the acute stage of the disease was found in 70% of patients. Furthermore, similarly to brain trauma, TENA was found to indicate cognitive outcome. The frequency of persisting global amnesia syndrome with both anterograde and retrograde amnesia in all encephalitic patients was 6%. One patient had isolated retrograde amnesia, which is very rare. In HSVE the frequency of global amnesia was 12.5%, which is lower than expected. As a group, HSVE patients were not found to have a homogeneous pattern of amnesia, instead subgroups among all encephalitic patients were observed: some patients had impaired semantic memory, some had difficulty predominantly with executive functions and some suffered from an increased forgetting rate. Herpes zoster encephalitis was found to result in mild memory impairment only, and the qualitative features indicated a subcortical dysfunction. On the whole, the cognitive deficits were predominantly found to diminish during follow-up. Progressive deterioration was often associated with intractable epilepsy. The frequency of dementia was 12.5%. In conclusion, the neuropsychological outcome, especially in HSVE, was more favourable than has previously been reported, possibly due to early acyclovir medication. Memory disorders after encephalitis should not be considered uniform, and the need for neuropsychological rehabilitation should be considered case-by-case

Semantic Fluency in Mild and Moderate Alzheimer's Disease

Alzheimer's disease (AD) is characterized by an impairment of the semantic memory responsible for processing meaning-related knowledge. This study was aimed at examining how Finnish-speaking healthy elderly subjects (n = 30) and mildly (n=20) and moderately (n = 20) demented AD patients utilize semantic knowledge to performa semantic fluency task, a method of studying semantic memory. In this task subjects are typically given 60 seconds to generate words belonging to the semantic category of animals. Successful task performance requires fast retrieval of subcategory exemplars in clusters (e.g., farm animals: 'cow', 'horse', 'sheep') and switching between subcategories (e.g., pets, water animals, birds, rodents). In this study, thescope of the task was extended to cover various noun and verb categories. The results indicated that, compared with normal controls, both mildly and moderately demented AD patients showed reduced word production, limited clustering and switching, narrowed semantic space, and an increase in errors, particularly perseverations. However, the size of the clusters, the proportion of clustered words, and the frequency and prototypicality of words remained relatively similar across the subject groups. Although the moderately demented patients showed a poor eroverall performance than the mildly demented patients in the individual categories, the error analysis appeared unaffected by the severity of AD. The results indicate a semantically rather coherent performance but less specific, effective, and flexible functioning of the semantic memory in mild and moderate AD patients. The findings are discussed in relation to recent theories of word production and semantic representation. Keywords: semantic fluency, clustering, switching, semantic category, nouns, verbs, Alzheimer's disease

Human brain networks of auditory attention and working memory

This thesis examines brain networks involved in auditory attention and auditory working memory using measures of task performance, brain activity, and neuroanatomical connectivity. Auditory orienting and maintenance of attention were compared with visual orienting and maintenance of attention, and top-down controlled attention was compared to bottom-up triggered attention in audition. Moreover, the effects of cognitive load on performance and brain activity were studied using an auditory working memory task. Corbetta and Shulman s (2002) model of visual attention suggests that what is known as the dorsal attention system (intraparietal sulcus/superior parietal lobule, IPS/SPL and frontal eye field, FEF) is involved in the control of top-down controlled attention, whereas what is known as the ventral attention system (temporo-parietal junction, TPJ and areas of the inferior/middle frontal gyrus, IFG/MFG) is involved in bottom-up triggered attention. The present results show that top-down controlled auditory attention also activates IPS/SPL and FEF. Furthermore, in audition, TPJ and IFG/MFG were activated not only by bottom-up triggered attention, but also by top-down controlled attention. In addition, the posterior cerebellum and thalamus were activated by top-down controlled attention shifts and the ventromedial prefrontal cortex (VMPFC) was activated by to-be-ignored, but attention-catching salient changes in auditory input streams. VMPFC may be involved in the evaluation of environmental events causing the bottom-up triggered engagement of attention. Auditory working memory activated a brain network that largely overlapped with the one activated by top-down controlled attention. The present results also provide further evidence of the role of the cerebellum in cognitive processing: During auditory working memory tasks, both activity in the posterior cerebellum (the crus I/II) and reaction speed increased when the cognitive load increased. Based on the present results and earlier theories on the role of the cerebellum in cognitive processing, the function of the posterior cerebellum in cognitive tasks may be related to the optimization of response speed.

Tarinakontekstin vaikutus sanojen muistamiseen kompleksisessa työmuistitehtävässä

Objectives. The sentence span task is a complex working memory span task used for estimating total working memory capacity for both processing (sentence comprehension) and storage (remembering a set of words). Several traditional models of working memory suggest that performance on these tasks relies on phonological short-term storage. However, long-term memory effects as well as the effects of expertise and strategies have challenged this view. This study uses a working memory task that aids the creation of retrieval structures in the form of stories, which have been shown to form integrated structures in longterm memory. The research question is whether sentence and story contexts boost memory performance in a complex working memory task. The hypothesis is that storage of the words in the task takes place in long-term memory. Evidence of this would be better recall for words as parts of sentences than for separate words, and, particularly, a beneficial effect for words as part of an organized story. Methods. Twenty stories consisting of five sentences each were constructed, and the stimuli in all experimental conditions were based on these sentences and sentence-final words, reordered and recombined for the other conditions. Participants read aloud sets of five sentences that either formed a story or not. In one condition they had to report all the last words at the end of the set, in another, they memorised an additional separate word with each sentence. The sentences were presented on the screen one word at a time (500 ms). After the presentation of each sentence, the participant verified a statement about the sentence. After five sentences, the participant repeated back the words in correct positions. Experiment 1 (n=16) used immediate recall, experiment 2 (n=21) both immediate recall and recall after a distraction interval (the operation span task). In experiment 2 a distracting mental arithmetic task was presented instead of recall in half of the trials, and an individual word was added before each sentence in the two experimental conditions when the participants were to memorize the sentence final words. Subjects also performed a listening span task (in exp.1) or an operation span task (exp.2) to allow comparison of the estimated span and performance in the story task. Results were analysed using correlations, repeated measures ANOVA and a chi-square goodness of fit test on the distribution of errors. Results and discussion. Both the relatedness of the sentences (the story condition) and the inclusion of the words into sentences helped memory. An interaction showed that the story condition had a greater effect on last words than separate words. The beneficial effect of the story was shown in all serial positions. The effects remained in delayed recall. When the sentences formed stories, performance in verification of the statements about sentence context was better. This, as well as the differing distributions of errors in different experimental conditions, suggest different levels of representation are in use in the different conditions. In the story condition, the nature of these representations could be in the form of an organized memory structure, a situation model. The other working memory tasks had only few week correlations to the story task. This could indicate that different processes are in use in the tasks. The results do not support short-term phonological storage, but instead are compatible with the words being encoded to LTM during the task.

Developmental, functional brain imaging and electrophysiological evidence of visual and auditory working memory

Intact function of working memory (WM) is essential for children and adults to cope with every day life. Children with deficits in WM mechanisms have learning difficulties that are often accompanied by behavioral problems. The neural processes subserving WM, and brain structures underlying this system, continue to develop during childhood till adolescence and young adulthood. With functional magnetic resonance imaging (fMRI) it is possible to investigate the organization and development of WM. The present thesis aimed to investigate, using behavioral and neuroimaging methods, whether mnemonic processing of spatial and nonspatial visual information is segregated in the developing and mature human brain. A further aim in this research was to investigate the organization and development of audiospatial and visuospatial information processing in WM. The behavioral results showed that spatial and nonspatial visual WM processing is segregated in the adult brain. The fMRI result in children suggested that memory load related processing of spatial and nonspatial visual information engages common cortical networks, whereas selective attention to either type of stimuli recruits partially segregated areas in the frontal, parietal and occipital cortices. Deactivation mechanisms that are important in the performance of WM tasks in adults are already operational in healthy school-aged children. Electrophysiological evidence suggested segregated mnemonic processing of visual and auditory location information. The results of the development of audiospatial and visuospatial WM demonstrate that WM performance improves with age, suggesting functional maturation of underlying cognitive processes and brain areas. The development of the performance of spatial WM tasks follows a different time course in boys and girls indicating a larger degree of immaturity in the male than female WM systems. Furthermore, the differences in mastering auditory and visual WM tasks may indicate that visual WM reaches functional maturity earlier than the corresponding auditory system. Spatial WM deficits may underlie some learning difficulties and behavioral problems related to impulsivity, difficulties in concentration, and hyperactivity. Alternatively, anxiety or depressive symptoms may affect WM function and the ability to concentrate, being thus the primary cause of poor academic achievement in children.

Remote sensing of leaf area index: enhanced retrieval from close-range and remotely sensed optical observations

A wide range of models used in agriculture, ecology, carbon cycling, climate and other related studies require information on the amount of leaf material present in a given environment to correctly represent radiation, heat, momentum, water, and various gas exchanges with the overlying atmosphere or the underlying soil. Leaf area index (LAI) thus often features as a critical land surface variable in parameterisations of global and regional climate models, e.g., radiation uptake, precipitation interception, energy conversion, gas exchange and momentum, as all areas are substantially determined by the vegetation surface. Optical wavelengths of remote sensing are the common electromagnetic regions used for LAI estimations and generally for vegetation studies. The main purpose of this dissertation was to enhance the determination of LAI using close-range remote sensing (hemispherical photography), airborne remote sensing (high resolution colour and colour infrared imagery), and satellite remote sensing (high resolution SPOT 5 HRG imagery) optical observations. The commonly used light extinction models are applied at all levels of optical observations. For the sake of comparative analysis, LAI was further determined using statistical relationships between spectral vegetation index (SVI) and ground based LAI. The study areas of this dissertation focus on two regions, one located in Taita Hills, South-East Kenya characterised by tropical cloud forest and exotic plantations, and the other in Gatineau Park, Southern Quebec, Canada dominated by temperate hardwood forest. The sampling procedure of sky map of gap fraction and size from hemispherical photographs was proven to be one of the most crucial steps in the accurate determination of LAI. LAI and clumping index estimates were significantly affected by the variation of the size of sky segments for given zenith angle ranges. On sloping ground, gap fraction and size distributions present strong upslope/downslope asymmetry of foliage elements, and thus the correction and the sensitivity analysis for both LAI and clumping index computations were demonstrated. Several SVIs can be used for LAI mapping using empirical regression analysis provided that the sensitivities of SVIs at varying ranges of LAI are large enough. Large scale LAI inversion algorithms were demonstrated and were proven to be a considerably efficient alternative approach for LAI mapping. LAI can be estimated nonparametrically from the information contained solely in the remotely sensed dataset given that the upper-end (saturated SVI) value is accurately determined. However, further study is still required to devise a methodology as well as instrumentation to retrieve on-ground green leaf area index . Subsequently, the large scale LAI inversion algorithms presented in this work can be precisely validated. Finally, based on literature review and this dissertation, potential future research prospects and directions were recommended.

Using Mathematical Morphology for Geometric Music Retrieval

The usual task in music information retrieval (MIR) is to find occurrences of a monophonic query pattern within a music database, which can contain both monophonic and polyphonic content. The so-called query-by-humming systems are a famous instance of content-based MIR. In such a system, the user's hummed query is converted into symbolic form to perform search operations in a similarly encoded database. The symbolic representation (e.g., textual, MIDI or vector data) is typically a quantized and simplified version of the sampled audio data, yielding to faster search algorithms and space requirements that can be met in real-life situations. In this thesis, we investigate geometric approaches to MIR. We first study some musicological properties often needed in MIR algorithms, and then give a literature review on traditional (e.g., string-matching-based) MIR algorithms and novel techniques based on geometry. We also introduce some concepts from digital image processing, namely the mathematical morphology, which we will use to develop and implement four algorithms for geometric music retrieval. The symbolic representation in the case of our algorithms is a binary 2-D image. We use various morphological pre- and post-processing operations on the query and the database images to perform template matching / pattern recognition for the images. The algorithms are basically extensions to classic image correlation and hit-or-miss transformation techniques used widely in template matching applications. They aim to be a future extension to the retrieval engine of C-BRAHMS, which is a research project of the Department of Computer Science at University of Helsinki.

Memory Meanders : Place, Home and Commemoration in an Ex-Rhodesian Diaspora Community

Memory Meanders is an ethnographic analysis of a postcolonial migrant community, white former Rhodesians, who have emigrated from Zimbabwe to South Africa after Zimbabwe s independence in 1980. An estimated 100 000 whites left the country during the first years of independence. Majority of these emigrants settled in South Africa. In recent years President Mugabe s land redistribution program has inflicted forced expulsions and violence against white farmers and black farm workers, and instigated a new wave of emigration. Concerning the study of Southern Africa, my research is therefore very topical. In recent years there has been a growing concern to study postcolonialism as it unfolds in the lived realities of actual postcolonies. A rising interest has also been cast on colonial cultures and white colonials within complex power relationships. My research offers insight to these discussions by investigating the ways in which the colonial past affects and effects in the present activities and ideas of former colonials. The study also takes part in discussing fundamental questions concerning how diaspora communities socially construct their place in the world in relation to the place left behind, to their current places of dwelling and to the community in dispersal. In spite of Rhodesia s incontestable ending, it is held close by social practices; by thoughts and talks, by material displays, and by webs of meaningful relationships. Such social memory practices, I suggest, are fundamental to how the community understands itself. The vantage points from which I examine how the ex-Rhodesians reminisce about Rhodesia concern ideas and practices related to place, home and commemoration. I first focus on the processes of symbolic investment that go into understanding place and landscape in Rhodesia and ask how the once dwelled-in places, iconic landscapes and experiences within places are reminisced about in diaspora. Secondly, I examine how home both as a mundanely organized sphere of everyday lives and as an idea of belonging is culturally configured, and analyze how and if homes travel in diaspora. In the final ethnographic section I focus on commemorative practices. I first analyze how food and culturally specific festive occasions of commensality are connected to social and sensual memory, considering the unique ways in which food acts as a mnemonic trigger in a diaspora community. The second example concerns the celebration of a centenary of Rhodesia in 1990. Through this case I describe how the mnemonic power of commemoration rests on the fact that culturally meaningful experiences are bodily re-enacted. I show how habitual memory connected to performance is one example of how memory gets passed-on in non-textual ways.

Fast in-memory XPath search using compressed indexes

A large fraction of an XML document typically consists of text data. The XPath query language allows text search via the equal, contains, and starts-with predicates. Such predicates can be efficiently implemented using a compressed self-index of the document's text nodes. Most queries, however, contain some parts querying the text of the document, plus some parts querying the tree structure. It is therefore a challenge to choose an appropriate evaluation order for a given query, which optimally leverages the execution speeds of the text and tree indexes. Here the SXSI system is introduced. It stores the tree structure of an XML document using a bit array of opening and closing brackets plus a sequence of labels, and stores the text nodes of the document using a global compressed self-index. On top of these indexes sits an XPath query engine that is based on tree automata. The engine uses fast counting queries of the text index in order to dynamically determine whether to evaluate top-down or bottom-up with respect to the tree structure. The resulting system has several advantages over existing systems: (1) on pure tree queries (without text search) such as the XPathMark queries, the SXSI system performs on par or better than the fastest known systems MonetDB and Qizx, (2) on queries that use text search, SXSI outperforms the existing systems by 1-3 orders of magnitude (depending on the size of the result set), and (3) with respect to memory consumption, SXSI outperforms all other systems for counting-only queries.

Task-dependent activations of human auditory cortex during pitch discrimination and pitch memory tasks

"The functional organization of auditory cortex (AC) is still poorly understood. Previous studies suggest segregation of auditory processing streams for spatial and nonspatial information located in the posterior and anterior AC, respectively (Rauschecker and Tian, 2000; Arnott et al., 2004; Lomber and Malhotra, 2008). Furthermore, previous studies have shown that active listening tasks strongly modulate AC activations (Petkov et al., 2004; Fritz et al., 2005; Polley et al., 2006). However, the task dependence of AC activations has not been systematically investigated. In the present study, we applied high-resolution functional magnetic resonance imaging of the AC and adjacent areas to compare activations during pitch discrimination and n-back pitch memory tasks that were varied parametrically in difficulty. We found that anterior AC activations were increased during discrimination but not during memory tasks, while activations in the inferior parietal lobule posterior to the AC were enhanced during memory tasks but not during discrimination. We also found that wide areas of the anterior AC and anterior insula were strongly deactivated during the pitch memory tasks. While these results are consistent with the proposition that the anterior and posterior AC belong to functionally separate auditory processing streams, our results show that this division is present also between tasks using spatially invariant sounds. Together, our results indicate that activations of human AC are strongly dependent on the characteristics of the behavioral task."