Studies of working memory in interrupted human-computer interaction

Distraction in the workplace is increasingly more common in the information age. Several tasks and sources of information compete for a worker's limited cognitive capacities in human-computer interaction (HCI). In some situations even very brief interruptions can have detrimental effects on memory. Nevertheless, in other situations where persons are continuously interrupted, virtually no interruption costs emerge. This dissertation attempts to reveal the mental conditions and causalities differentiating the two outcomes. The explanation, building on the theory of long-term working memory (LTWM; Ericsson and Kintsch, 1995), focuses on the active, skillful aspects of human cognition that enable the storage of task information beyond the temporary and unstable storage provided by short-term working memory (STWM). Its key postulate is called a retrieval structure an abstract, hierarchical knowledge representation built into long-term memory that can be utilized to encode, update, and retrieve products of cognitive processes carried out during skilled task performance. If certain criteria of practice and task processing are met, LTWM allows for the storage of large representations for long time periods, yet these representations can be accessed with the accuracy, reliability, and speed typical of STWM. The main thesis of the dissertation is that the ability to endure interruptions depends on the efficiency in which LTWM can be recruited for maintaing information. An observational study and a field experiment provide ecological evidence for this thesis. Mobile users were found to be able to carry out heavy interleaving and sequencing of tasks while interacting, and they exhibited several intricate time-sharing strategies to orchestrate interruptions in a way sensitive to both external and internal demands. Interruptions are inevitable, because they arise as natural consequences of the top-down and bottom-up control of multitasking. In this process the function of LTWM is to keep some representations ready for reactivation and others in a more passive state to prevent interference. The psychological reality of the main thesis received confirmatory evidence in a series of laboratory experiments. They indicate that after encoding into LTWM, task representations are safeguarded from interruptions, regardless of their intensity, complexity, or pacing. However, when LTWM cannot be deployed, the problems posed by interference in long-term memory and the limited capacity of the STWM surface. A major contribution of the dissertation is the analysis of when users must resort to poorer maintenance strategies, like temporal cues and STWM-based rehearsal. First, one experiment showed that task orientations can be associated with radically different patterns of retrieval cue encodings. Thus the nature of the processing of the interface determines which features will be available as retrieval cues and which must be maintained by other means. In another study it was demonstrated that if the speed of encoding into LTWM, a skill-dependent parameter, is slower than the processing speed allowed for by the task, interruption costs emerge. Contrary to the predictions of competing theories, these costs turned out to involve intrusions in addition to omissions. Finally, it was learned that in rapid visually oriented interaction, perceptual-procedural expectations guide task resumption, and neither STWM nor LTWM are utilized due to the fact that access is too slow. These findings imply a change in thinking about the design of interfaces. Several novel principles of design are presented, basing on the idea of supporting the deployment of LTWM in the main task.

Tohtoriksi tulemisen tarina

The study examines the dissertation process. The thesis is based on twenty-three autobiographical stories. They were collected from PhDs who had taken their doctoral degrees at the University of Helsinki. The purpose is to investigate the experiences of these PhD recipients and find out what they recall of the events, traditions, and meanings associated with their PhD studies. An anthology collected from PhDs in the historical sciences, which was published in 1998, was used as reference material. This study begins with a general presentation of the history and values of the university and the traditions of the academic community in Finland. Thereafter, the data and the methodological framework of the study are presented. Attention is paid to the discipline background, sex and age of these PhDs, which may have affected their storytelling. The former studies concerning the academic life together with the experiences of the writers revealed that the process of becoming a PhD graduate is generally considered as an academic rite of passage. The change from student status to becoming a fully accepted professional member of one’s field is real, transforming and at certain points, ritualized. The finding of an inner logic to the doctoral process, which included meaningful turning points, was central to the narrative analysis approach. During the process there were different kinds of struggles and highlights. The family background, university studies and time in employment had directed the doctoral studies of some PhD students enormously. But generally the most memorable and value laden were the phases of actually writing the dissertation, the public examination, and the celebration party in the evening called “karonkka”. The picture that emerged from these various life-stories demonstrated that there was no ivory tower where doctoral candidates live an isolated existence. Everyday cares in ordinary life and tasks in academic life are both demanding and rewarding at the same time. The main point is that the academic community in general and PhD supervisors in particular should concentrate not only on the thesis-writing process their students but also on the doctoral students´ wider life situation. Doctoral students need scientific, financial, and mental support. Somebody must be available to inspire and offer encouragement otherwise the process will be disturbed. The successful completion of the PhD ritual gives the PhD graduate self-confidence and respect but its’ influence on the doctors’ subsequent academic career is diverse. The dissertation process is a personal and unique experience. The aim of the PhD graduation is that this traditional ritual leads to a successful completion and ensures a positive reward for the PhD graduate and the academic community. Keywords: autobiography, narratives, academic traditions, PhD graduates.

Visual search and eye movements: Studies of perceptual span

In visual search one tries to find the currently relevant item among other, irrelevant items. In the present study, visual search performance for complex objects (characters, faces, computer icons and words) was investigated, and the contribution of different stimulus properties, such as luminance contrast between characters and background, set size, stimulus size, colour contrast, spatial frequency, and stimulus layout were investigated. Subjects were required to search for a target object among distracter objects in two-dimensional stimulus arrays. The outcome measure was threshold search time, that is, the presentation duration of the stimulus array required by the subject to find the target with a certain probability. It reflects the time used for visual processing separated from the time used for decision making and manual reactions. The duration of stimulus presentation was controlled by an adaptive staircase method. The number and duration of eye fixations, saccade amplitude, and perceptual span, i.e., the number of items that can be processed during a single fixation, were measured. It was found that search performance was correlated with the number of fixations needed to find the target. Search time and the number of fixations increased with increasing stimulus set size. On the other hand, several complex objects could be processed during a single fixation, i.e., within the perceptual span. Search time and the number of fixations depended on object type as well as luminance contrast. The size of the perceptual span was smaller for more complex objects, and decreased with decreasing luminance contrast within object type, especially for very low contrasts. In addition, the size and shape of perceptual span explained the changes in search performance for different stimulus layouts in word search. Perceptual span was scale invariant for a 16-fold range of stimulus sizes, i.e., the number of items processed during a single fixation was independent of retinal stimulus size or viewing distance. It is suggested that saccadic visual search consists of both serial (eye movements) and parallel (processing within perceptual span) components, and that the size of the perceptual span may explain the effectiveness of saccadic search in different stimulus conditions. Further, low-level visual factors, such as the anatomical structure of the retina, peripheral stimulus visibility and resolution requirements for the identification of different object types are proposed to constrain the size of the perceptual span, and thus, limit visual search performance. Similar methods were used in a clinical study to characterise the visual search performance and eye movements of neurological patients with chronic solvent-induced encephalopathy (CSE). In addition, the data about the effects of different stimulus properties on visual search in normal subjects were presented as simple practical guidelines, so that the limits of human visual perception could be taken into account in the design of user interfaces.

Achievement and Social Strategies and the Cumulation of Positive and Negative Experiences During Young Adulthood

These longitudinal studies focused on investigating young adults during transition into a new educational environment. The aims were to examine: (1) what kinds of achievement and social strategies young adults deploy, (2) whether the deployment of these strategies predicts people's success in their studies, their life events, their peer relationships, and their well-being, (3) whether young adults' success in dealing with educational transition (e.g. success in studies, life events, peer relationships and well-being) predict changes in their strategies and well-being, and (4) the associations between young adults' social strategies, interpersonal behaviour, person perception, and their peer relationships and satisfaction with them. The participants were students from Helsinki university and from two vocational institutes (the numbers ranging between 92 and 303). The results revealed that achievement and social strategies contributed to individuals' success in dealing with both the academic and interpersonal challenges of a new environment. Social strategies were also associated with online interpersonal behaviour and person perception, which mediated their impact on peer relationships. Achievement and social strategies changed as a result of environmental feedback. However, they also showed high stability, forming reciprocal and cumulative associations with the feedback the individuals received about their success in dealing with educational transition: the use of functional strategies, such as optimistic, defensive-pessimistic and planning-oriented strategies, increased their success, which in turn enhanced their well-being and further deployment of functional strategies. The opposite was true in the case of dysfunctional strategies, such as self-handicapping and avoidance. Key words : Achievement strategies, social strategies, transition, young adults, life events, sociometric status, social behaviour, person perception, well-being.

Determinants of vascular cognitive impairment : White matter lesions, brain atrophy and their neuropsychological correlates

The concept of vascular cognitive impairment (VCI) covers a wide spectrum of cognitive dysfunctions related to cerebrovascular disease. Among the pathophysiological determinants of VCI are cerebral stroke, white matter lesions and brain atrophy, which are known to be important risk factors for dementia. However, the specific mechanisms behind the brain abnormalities and cognitive decline are still poorly understood. The present study investigated the neuropsychological correlates of particular magnetic resonance imaging (MRI) findings, namely, medial temporal lobe atrophy (MTA), white matter hyperintensities (WMH), general cortical atrophy and corpus callosum (CC) atrophy in subjects with cerebrovascular disease. Furthermore, the cognitive profile of subcortical ischaemic vascular disease (SIVD) was examined. This study was conducted as part of two large multidisciplinary study projects, the Helsinki Stroke Aging Memory (SAM) Study and the multinational Leukoaraiosis and Disability (LADIS) Study. The SAM cohort consisted of 486 patients, between 55 and 85 years old, with ischaemic stroke from the Helsinki University Hospital, Helsinki, Finland. The LADIS Study included a mixed sample of subjects (n=639) with age-related WMH, between 65 and 84 years old, gathered from 11 centres around Europe. Both studies included comprehensive clinical and neuropsychological assessments and detailed brain MRI. The relationships between the MRI findings and the neuropsychological test performance were analysed by controlling for relevant confounding factors such as age, education and other coexisting brain lesions. The results revealed that in elderly patients with ischaemic stroke, moderate to severe MTA was specifically related to impairment of memory and visuospatial functions, but mild MTA had no clinical relevance. Instead, WMH were primarily associated with executive deficits and mental slowing. These deficits mediated the relationship between WMH and other, secondary cognitive deficits. Cognitive decline was best predicted by the overall degree of WMH, whereas the independent contribution of regional WMH measures was low. Executive deficits were the most prominent cognitive characteristic in SIVD. Compared to other stroke patients, the patients with SIVD also presented more severe memory deficits, which were related to MTA. The cognitive decline in SIVD occurred independently of depressive symptoms and, relative to healthy control subjects, it was substantial in severity. In stroke patients, general cortical atrophy also turned out to be a strong predictor of cognitive decline in a wide range of cognitive domains. Moreover, in elderly subjects with WMH, overall CC atrophy was related to reduction in mental speed, while anterior CC atrophy was independently associated with frontal lobe-mediated executive functions and attention. The present study provides cross-sectional evidence for the involvement of WMH, MTA, general cortical atrophy and CC atrophy in VCI. The results suggest that there are multifaceted pathophysiological mechanisms behind VCI in the elderly, including both vascular ischaemic lesions and neurodegenerative changes. The different pathological changes are highly interrelated processes and together they may produce cumulative effects on cognitive decline.

Käsityksistä ajatuksen poluille : Ammattikorkeakoulun tutkimus- ja kehitystoiminnan kehittäminen

This study deals with the formation of the idea of research and development (r&d) activity at one particular university of applied sciences in Finland. In this study I proceed from examining the conceptions of management regarding r&d-activities to exploring the development of the complex concept that guides these r&d-activities. The aim of this study is, first, to describe conceptions of r&d-activities, and then, second, to describe the formation of a new concept for r&d-activities in one field of study at the university of applied sciences. I used phenomenographic analysis to elucidate the conceptions and collected the data in interviews of personnel (22) who belong to the management of the university of applied sciences. The data for analysing the obstacles faced and the ways of overcoming them during the formation process of the new concept consist of the video-recorded material from ten change laboratory sessions held for the specific field of study at the university of applied sciences. In researching the formation of the concept of the activity, such research and analysis methods are used in which the concept is understood as a material construction, and with the help of which one can work out the formation of the concept during the development process. When the systematic development of the r&d-work was underway, the management of the university of applied sciences held differing views regarding the content, target, factors and organisation of the r&d-activities, as well as regarding the position of dissertations as a part of r&d-activity. Obstacles for developing rd concerned the unclear object and outcomes of the new activity, not having the tools (such as adequate know-how) for the new activity and not having found the necessary solutions for the distribution of work. In addition the rules that quide the activities of the university of applied sciences did not support working according to the new way. To improve r&d-activity, the school’s management defined three development strategies to encourage adequate rd-activity to support working life, regional development and learning. The strategies were based on the expansion of existing methods, such as service activities, dissertation work and the research activities of the teachers. Of the three possible routes, the concept of dissertation in this study was expanded to integrate the services of organisation development and the occupational growth of the students. A group of teachers in the field of social and health studies participated in the development work. The change laboratory method was used as a tool for this work. Analysis of the cognitive trails indicated that, in addition to the cycle of expansive learning activities, microcycles with different purposes can exist. According to Cussins (1992), something already in existence in each microcycle is destabilised, and something else is stabilised to replace it. In this study three microcycles were identified, during which the teachers first destabilised the existing concepts of dissertation work that guided their thinking, and then began to use new concepts to structure their thinking. Secondly, they undermined the existing concepts with material structure that defined their practical work, and developed new practical models to replace them. Thirdly, the problems accumulating in researching and testing the new activities caused the developers to destabilise the r&d-strategies of the university of applied sciences and to define a new conceptual model for r&d-work in the field. During this third cycle, the teachers developed a research arena model, which was a significant expansive innovation. In the cognitive trails developing the new concept for the r&d-activity, the teachers indeed faced the obstacles the management had described as their conceptions. These obstacles manifested themselves as contradictions. During the development process, the nature of the obstacles (i.e. contradictions) changed as the development proceeded. Solving the first and second degree contradictions highlighted the third and fourth degree contradictions. To overcome the obstacles, the teachers had to to articulate the value and motive of the development work throughout the development process. Developing the new concept for r&d-activity required many reconfigurations of practical solutions to overcome the obstacles. Developing the new concept for r&d-activity, both at the level of representations and of new practices, requires universities of applied sciences to adopt new methods in which the actors are partake in construction of new concepts of activity through adequate discussion, analysis and debate. R&d -activity can progress, if instead of implementing partial solutions, the totality of the activities (i.e. the group of partial solutions acting together) will be constructed. The development of the activity system requires many simultaneous changes as well as wide-ranging know-how and discussion related to these changes. What is perhaps most important, however, is that the group of developers mature into a determined collective actor which can engage in many agentive actions. The development of the agency is fundamental to progress.

Social perception and cognition : processing of gestures, postures and facial expressions in the human brain

Humans are a social species with the internal capability to process social information from other humans. To understand others behavior and to react accordingly, it is necessary to infer their internal states, emotions and aims, which are conveyed by subtle nonverbal bodily cues such as postures, gestures, and facial expressions. This thesis investigates the brain functions underlying the processing of such social information. Studies I and II of this thesis explore the neural basis of perceiving pain from another person s facial expressions by means of functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). In Study I, observing another s facial expression of pain activated the affective pain system (previously associated with self-experienced pain) in accordance with the intensity of the observed expression. The strength of the response in anterior insula was also linked to the observer s empathic abilities. The cortical processing of facial pain expressions advanced from the visual to temporal-lobe areas at similar latencies (around 300 500 ms) to those previously shown for emotional expressions such as fear or disgust. Study III shows that perceiving a yawning face is associated with middle and posterior STS activity, and the contagiousness of a yawn correlates negatively with amygdalar activity. Study IV explored the brain correlates of interpreting social interaction between two members of the same species, in this case human and canine. Observing interaction engaged brain activity in very similar manner for both species. Moreover, the body and object sensitive brain areas of dog experts differentiated interaction from noninteraction in both humans and dogs whereas in the control subjects, similar differentiation occurred only for humans. Finally, Study V shows the engagement of the brain area associated with biological motion when exposed to the sounds produced by a single human being walking. However, more complex pattern of activation, with the walking sounds of several persons, suggests that as the social situation becomes more complex so does the brain response. Taken together, these studies demonstrate the roles of distinct cortical and subcortical brain regions in the perception and sharing of others internal states via facial and bodily gestures, and the connection of brain responses to behavioral attributes.

Oscillatory brain activity in memory disorders

Neuronal oscillations are thought to underlie interactions between distinct brain regions required for normal memory functioning. This study aimed at elucidating the neuronal basis of memory abnormalities in neurodegenerative disorders. Magnetoencephalography (MEG) was used to measure oscillatory brain signals in patients with Alzheimer s disease (AD), a neurodegenerative disease causing progressive cognitive decline, and mild cognitive impairment (MCI), a disorder characterized by mild but clinically significant complaints of memory loss without apparent impairment in other cognitive domains. Furthermore, to help interpret our AD/MCI results and to develop more powerful oscillatory MEG paradigms for clinical memory studies, oscillatory neuronal activity underlying declarative memory, the function which is afflicted first in both AD and MCI, was investigated in a group of healthy subjects. An increased temporal-lobe contribution coinciding with parieto-occipital deficits in oscillatory activity was observed in AD patients: sources in the 6 12.5 Hz range were significantly stronger in the parieto-occipital and significantly weaker in the right temporal region in AD patients, as compared to MCI patients and healthy elderly subjects. Further, the auditory steady-state response, thought to represent both evoked and induced activity, was enhanced in AD patients, as compared to controls, possibly reflecting decreased inhibition in auditory processing and deficits in adaptation to repetitive stimulation with low relevance. Finally, the methodological study revealed that successful declarative encoding and retrieval is associated with increases in occipital gamma and right hemisphere theta power in healthy unmedicated subjects. This result suggests that investigation of neuronal oscillations during cognitive performance could potentially be used to investigate declarative memory deficits in AD patients. Taken together, the present results provide an insight on the role of brain oscillatory activity in memory function and memory disorders.