Studies of working memory in interrupted human-computer interaction

Distraction in the workplace is increasingly more common in the information age. Several tasks and sources of information compete for a worker's limited cognitive capacities in human-computer interaction (HCI). In some situations even very brief interruptions can have detrimental effects on memory. Nevertheless, in other situations where persons are continuously interrupted, virtually no interruption costs emerge. This dissertation attempts to reveal the mental conditions and causalities differentiating the two outcomes. The explanation, building on the theory of long-term working memory (LTWM; Ericsson and Kintsch, 1995), focuses on the active, skillful aspects of human cognition that enable the storage of task information beyond the temporary and unstable storage provided by short-term working memory (STWM). Its key postulate is called a retrieval structure an abstract, hierarchical knowledge representation built into long-term memory that can be utilized to encode, update, and retrieve products of cognitive processes carried out during skilled task performance. If certain criteria of practice and task processing are met, LTWM allows for the storage of large representations for long time periods, yet these representations can be accessed with the accuracy, reliability, and speed typical of STWM. The main thesis of the dissertation is that the ability to endure interruptions depends on the efficiency in which LTWM can be recruited for maintaing information. An observational study and a field experiment provide ecological evidence for this thesis. Mobile users were found to be able to carry out heavy interleaving and sequencing of tasks while interacting, and they exhibited several intricate time-sharing strategies to orchestrate interruptions in a way sensitive to both external and internal demands. Interruptions are inevitable, because they arise as natural consequences of the top-down and bottom-up control of multitasking. In this process the function of LTWM is to keep some representations ready for reactivation and others in a more passive state to prevent interference. The psychological reality of the main thesis received confirmatory evidence in a series of laboratory experiments. They indicate that after encoding into LTWM, task representations are safeguarded from interruptions, regardless of their intensity, complexity, or pacing. However, when LTWM cannot be deployed, the problems posed by interference in long-term memory and the limited capacity of the STWM surface. A major contribution of the dissertation is the analysis of when users must resort to poorer maintenance strategies, like temporal cues and STWM-based rehearsal. First, one experiment showed that task orientations can be associated with radically different patterns of retrieval cue encodings. Thus the nature of the processing of the interface determines which features will be available as retrieval cues and which must be maintained by other means. In another study it was demonstrated that if the speed of encoding into LTWM, a skill-dependent parameter, is slower than the processing speed allowed for by the task, interruption costs emerge. Contrary to the predictions of competing theories, these costs turned out to involve intrusions in addition to omissions. Finally, it was learned that in rapid visually oriented interaction, perceptual-procedural expectations guide task resumption, and neither STWM nor LTWM are utilized due to the fact that access is too slow. These findings imply a change in thinking about the design of interfaces. Several novel principles of design are presented, basing on the idea of supporting the deployment of LTWM in the main task.

Social perception and cognition : processing of gestures, postures and facial expressions in the human brain

Humans are a social species with the internal capability to process social information from other humans. To understand others behavior and to react accordingly, it is necessary to infer their internal states, emotions and aims, which are conveyed by subtle nonverbal bodily cues such as postures, gestures, and facial expressions. This thesis investigates the brain functions underlying the processing of such social information. Studies I and II of this thesis explore the neural basis of perceiving pain from another person s facial expressions by means of functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). In Study I, observing another s facial expression of pain activated the affective pain system (previously associated with self-experienced pain) in accordance with the intensity of the observed expression. The strength of the response in anterior insula was also linked to the observer s empathic abilities. The cortical processing of facial pain expressions advanced from the visual to temporal-lobe areas at similar latencies (around 300 500 ms) to those previously shown for emotional expressions such as fear or disgust. Study III shows that perceiving a yawning face is associated with middle and posterior STS activity, and the contagiousness of a yawn correlates negatively with amygdalar activity. Study IV explored the brain correlates of interpreting social interaction between two members of the same species, in this case human and canine. Observing interaction engaged brain activity in very similar manner for both species. Moreover, the body and object sensitive brain areas of dog experts differentiated interaction from noninteraction in both humans and dogs whereas in the control subjects, similar differentiation occurred only for humans. Finally, Study V shows the engagement of the brain area associated with biological motion when exposed to the sounds produced by a single human being walking. However, more complex pattern of activation, with the walking sounds of several persons, suggests that as the social situation becomes more complex so does the brain response. Taken together, these studies demonstrate the roles of distinct cortical and subcortical brain regions in the perception and sharing of others internal states via facial and bodily gestures, and the connection of brain responses to behavioral attributes.

The Freudian unconscious in the context of the cognitive orientation

For decades psychoanalysis was the discipline studying the unconscious, and other branches of study lacked competence to take a stand on the issues concerning the unconscious. From 1980s onwards intense study of the unconscious has been taken place in the scope of cognitive orientation. Thus, nowadays it is talked about both pyschoanalytic and cognitive unconscious. The aim of this thesis is to integrate psychoanalytic and cognitive views. When the "Freudian" conception of the unconscious is considered, there are four entangled issues: 1) what is the unconscious like, 2) how does the unconsciuos give rise to psychic disorders 3) why and how certain contents are missing from consciousness (repression of contents), 4) the emergence of those contents (becoming conscious of the repressed). The conventional psychoanalytic answer to the first question - and "the cornerstone of psychoanalysis" - is "the unconscious is mental". The issues 2)-4) depend radically on the answer given to the 1): "psychoanalytic" conceptualizations on them rest on the "cornerstone". That ground was challended in study I: it was argued that it has never been clear what does it mean that the unconscious is mental. Thus, it was stated that in the current state of art psychoanalysis should drop out the ephitet "mental" before the term unconscious. That claim creates a pressure to reappraise the convential "psychoanalytic" answers to the other questions, and that reappraisal was the aim of studies II and III. In study II the question 2) is approached in terms of implicit knowledge. Study III focuses on mechanisms, which determine which contents appear in the scope of consciousness, and also cause missing of contents from there (the questions 3) and 4)). In the core of study III there are distinctions concerning the processess occuring in the levels of the brain, consciousness, self-consciousness, and narrative self-consciousness. Studies I-III set "psychoanalytic" topics in the frames of cognitive view. The picture emerging from those studies is not especially useful for a clinican (psychotherapist). Studies IV and V focused that issue. Study IV is a rather serious critique toward neuropsychoanalysis. In it it was claimed that repressive functions of conscious states are in the core of clinical psychoanalysis, and functions in general cannot be reduced to neurophysiological terminology. Thus, the limits of neuropsychoanalysis are more strict than it has been realized: crucial clinical issues remain outside its scope. In study V it was focused on the confusing state of things that although unconscious fantasies do not exist, the idea on them has been an important conceptual tool for clinicans. When put in a larger context, the aim of study V is similar to that of study IV: to determine the relation between psychotherapists' and neuroscientists' terminologies. Studies III, IV and V apply the philosopher Daniel Dennett's model on different levels of explanation.

Minimalistinen yhdyssanateoria ja käsitteiden yhdistely

The thesis aims to link the biolinguistic research program and the results of studies in comceptual combination from cognitive psychology. The thesis derives a theory of syntactic structure of noun and adjectival compounds from the Empty Lexicon Hypothesis. Two compound-forming operations are described: root-compounding and word-compounding. The aptness of theory is tested with finnish and greek compounds. From the syntactic theory semantic requirements for conceptual system are derived, especially requirements for handling morphosyntactic features. These requirements are compared to three formidable theories of conceptual combination: relation theory CARIN, Dual-Process theory and C3-theory. The claims of explanatory power of relational distributions of modifier in CARIN-theory ared discarded, as the method for sampling and building relational distributions is not reliable and the algorithmic instantiation of theory does not compute what it claims to compute. From relational theory there still remains results supporting existence of 'easy' relations for certain concepts. Dual-Process theory is found to provide results that cannot in theory be affected by linguistic system, but the basic idea of property compounds is kept. C3-theory is found to be not computationally realistic, but the basic results of diagnosticity and local properties (domains) of conceptual system are solid. The three conceptual combination models are rethought as a problem of finding the shortest route between the two concepts. The new basis for modeling is suggested to be bare conceptual landscape with morphosyntactiic or semantic features working as guidance and structural features of landscape basically unknown, but such as they react to features from linguistic system. Minimalistic principles to conceptual modeling are suggested.

Neurocognitive processing of inflected and derived words

The representation of morphologically complex words in the mental lexicon and their neurocognitive processing has been a vigorously debated topic in psycholinguistics and the cognitive neuroscience of language. This thesis investigates the effect of stimulus modality on morphological processing, the spatiotemporal dynamics of the neural processing of inflected (e.g., work+ed ) and derived (e.g., work+er ) words and their interaction, using the Finnish language. Overall, the results suggest that the constituent morphemes of isolated written and spoken inflected words are accessed separately, whereas spoken derived words activate both their full form and the constituent morphemes. The processing of both spoken and written inflected words elicited larger N400 responses than monomorphemic words (Study I), whereas the responses to spoken derived words did not differ from those to monomorphemic words (Study IV). Spoken inflected words elicited a larger left-lateralized negativity and greater source strengths in the left temporal cortices than derived words (Study IV). Thus, the results suggest different cortical processing for derived and inflected words. Moreover, the neural mechanisms underlying inflection and derivation seem to be not only different, but also independent as indexed by the linear summation of the responses to derived and inflected stimuli in a combined (derivation+inflection) condition (Study III). Furthermore, the processing of meaningless, spoken derived pseudowords was more difficult than for existing derived words, indexed by a larger N400-type effect for the pseudowords. However, no differences were observed between meaningful derived pseudowords and existing derived words (Study II). The results of Study II suggest that semantic compatibility between morphemes seems to have a crucial role in a successful morphological analysis. As a methodological note, time-locking the auditory event-related potentials/fields (ERP/ERF) to the suffix onset revealed the processes related to morphological analysis more precisely (Studies II and IV), which also enables comparison of the neural processes in different modalities (Study I).