Investigating online reading with eye tracking and EEG : The influence of text format, reading task and parafoveal stimuli on reading processes

Research on reading has been successful in revealing how attention guides eye movements when people read single sentences or text paragraphs in simplified and strictly controlled experimental conditions. However, less is known about reading processes in more naturalistic and applied settings, such as reading Web pages. This thesis investigates online reading processes by recording participants eye movements. The thesis consists of four experimental studies that examine how location of stimuli presented outside the currently fixated region (Study I and III), text format (Study II), animation and abrupt onset of online advertisements (Study III), and phase of an online information search task (Study IV) affect written language processing. Furthermore, the studies investigate how the goal of the reading task affects attention allocation during reading by comparing reading for comprehension with free browsing, and by varying the difficulty of an information search task. The results show that text format affects the reading process, that is, vertical text (word/line) is read at a slower rate than a standard horizontal text, and the mean fixation durations are longer for vertical text than for horizontal text. Furthermore, animated online ads and abrupt ad onsets capture online readers attention and direct their gaze toward the ads, and distract the reading process. Compared to a reading-for-comprehension task, online ads are attended to more in a free browsing task. Moreover, in both tasks abrupt ad onsets result in rather immediate fixations toward the ads. This effect is enhanced when the ad is presented in the proximity of the text being read. In addition, the reading processes vary when Web users proceed in online information search tasks, for example when they are searching for a specific keyword, looking for an answer to a question, or trying to find a subjectively most interesting topic. A scanning type of behavior is typical at the beginning of the tasks, after which participants tend to switch to a more careful reading state before finishing the tasks in the states referred to as decision states. Furthermore, the results also provided evidence that left-to-right readers extract more parafoveal information to the right of the fixated word than to the left, suggesting that learning biases attentional orienting towards the reading direction.

Olivier Messiaenin väriharmoniat

Tutkielmassani käsittelen ranskalaisen säveltäjän Olivier Messiaenin (1908-1992) synestesiaan liittyvää harmonista ajattelua. Messiaen oli ääni-väri -synesteetikko, toisin sanoen hän näki kuulemansa musiikin väreinä. Tältä pohjalta hän kehitti monimutkaisten sointujen ja asteikkojen järjestelmän, jonka puitteissa hän pystyi ikään kuin maalaamaan musiikillaan haluamansa värit. Esittelen aluksi tämänhetkistä synestesiatutkimusta lähinnä kolmen vallitsevassa asemassa olevan päälinjan pohjalta. Näitä edustavat yhdysvaltalainen Richard Cytowic, britti Simon Baron-Cohen ja saksalainen Hinderk Emrich. Käyn läpi heidän määritelmänsä synestesiasta ja näkemyksiään synesteettisistä säveltäjistä. Messiaen on jossain määrin tuonut esiin musiikkinsa värejä lähes uransa alusta lähtien, mutta huomiot ovat yleensä olleet melko ylimalkaisia ja epäjärjestelmällisiä. Tuon esiin näitä eri lähteistä löytyneitä Messiaenin näkemyksiä ja kokemuksia tarkasteltuna Messiaenin henkilökohtaisen sävellysfilosofian pohjalta sekä tiettyihin Messiaenin sävellyksiin ja niiden harmonisiin rakenteisiin liittyen. Messiaenin kuoleman jälkeen vuosina 1994-2002 julkaistun seitsenosaisen teossarjan Traité de rythme, de couleur, et d'ornitologie viimeisessä osassa Messiaen esittää järjestelmällisesti kaikkien erikoissointujensa ja moodiensa värityksen. Tarkastelen Messiaenin käyttämiä harmonioita jakamalla ne yleisiin ja erikoissointuihin sekä moodeihin. Määritellessäni näitä harmonioita käytän metodina sävelluokkajoukkojen teoriaa, jonka avulla kaikki sävelyhdistelmät voidaan määritellä yksiselitteisesti. Esittelen joukkoteoriaa ja siihen liittyviä käsitteitä siinä määrin kuin se liittyy tutkielmaani, enkä siis yritäkään tuoda esiin analyysimetodia koko laajuudessaan. Olen rekonstruoinut visuaalisesti Messiaenin käyttämät noin 150 harmonista väriä, ja tutkielmani liitteenä olevalla CD-romilla esitän ne soivassa ja näkyvässä muodossa. Tällä CD-romilla on myös analyysiMessiaenin urkuteoksen Méditations sur le mystère de la Sainte Trinité 5. osan alusta, jossa tutkin käytännön tasolla värien ilmenemistä Messiaenin musiikissa. Tutkielmani keskeinen osa onkin tämä CD-rom, ja siitä ilmennee myös tutkielmani lähtökohta ja tavoite: luoda työkalu Messiaenin musiikin värisisältöjen hahmottamiseen.

Taustamusiikki markkinoinnin työvälineenä suomalaisissa tavarataloissa

The main research problem of this study was to explain how and why background music is used in Finnish department stores and how it is related to their marketing. The problem was investigated through the opinions, attitudes, and conceptions of the managers of Anttila, Sokos and Stockmann department stores. The data of study (N = 31) constituted of a www-survey to which the managers were asked to answer. In the first chapter of the study s theoretical section, the relationship between background music and an enterprise was examined. It was found that background music can serve as an aid in seeking competitive advantage. In the second chapter, the service encounter s environment and atmosphere in relation to marketing was examined and it was found that they are a part of customer s product or service experience. In the third chapter, the interaction process between service encounter atmosphere and consumer behaviour was examined and the essential finding was that atmospheric stimuli affects an individual through emotional, cognitive, and physiological processes, in which individual s personal characteristics are also in a great role. In the fourth chapter, the significance of background music s musical features was examined but the research results were found so contradictory that only the complexity of the studied phenomenon became clear. Findings from the study s empirical section showed that all examined department stores play background music and the usage of music is chain-controlled. The respondents considered background music in department stores as a fundamental element and they understood its significance in enterprise s marketing. The respondents also believed that customers consider background music important and pleasant. Respondents views on background music s effects to purchasing behaviour divided opinions more, but the majority however believed that background music has effects to purchasing behaviour. The main conclusion of the study was that background music is an important marketing tool, at least in a department store type service encounter.

Pitch discrimination in optimal and suboptimal acoustic environments: electroencephalographic, magnetoencephalographic, and behavioral evidence

Pitch discrimination is a fundamental property of the human auditory system. Our understanding of pitch-discrimination mechanisms is important from both theoretical and clinical perspectives. The discrimination of spectrally complex sounds is crucial in the processing of music and speech. Current methods of cognitive neuroscience can track the brain processes underlying sound processing either with precise temporal (EEG and MEG) or spatial resolution (PET and fMRI). A combination of different techniques is therefore required in contemporary auditory research. One of the problems in comparing the EEG/MEG and fMRI methods, however, is the fMRI acoustic noise. In the present thesis, EEG and MEG in combination with behavioral techniques were used, first, to define the ERP correlates of automatic pitch discrimination across a wide frequency range in adults and neonates and, second, they were used to determine the effect of recorded acoustic fMRI noise on those adult ERP and ERF correlates during passive and active pitch discrimination. Pure tones and complex 3-harmonic sounds served as stimuli in the oddball and matching-to-sample paradigms. The results suggest that pitch discrimination in adults, as reflected by MMN latency, is most accurate in the 1000-2000 Hz frequency range, and that pitch discrimination is facilitated further by adding harmonics to the fundamental frequency. Newborn infants are able to discriminate a 20% frequency change in the 250-4000 Hz frequency range, whereas the discrimination of a 5% frequency change was unconfirmed. Furthermore, the effect of the fMRI gradient noise on the automatic processing of pitch change was more prominent for tones with frequencies exceeding 500 Hz, overlapping with the spectral maximum of the noise. When the fundamental frequency of the tones was lower than the spectral maximum of the noise, fMRI noise had no effect on MMN and P3a, whereas the noise delayed and suppressed N1 and exogenous N2. Noise also suppressed the N1 amplitude in a matching-to-sample working memory task. However, the task-related difference observed in the N1 component, suggesting a functional dissociation between the processing of spatial and non-spatial auditory information, was partially preserved in the noise condition. Noise hampered feature coding mechanisms more than it hampered the mechanisms of change detection, involuntary attention, and the segregation of the spatial and non-spatial domains of working-memory. The data presented in the thesis can be used to develop clinical ERP-based frequency-discrimination protocols and combined EEG and fMRI experimental paradigms.

Visual completion in an illusory figure

The visual systems of humans and animals represent physical reality in a modified way, depending on the specific demands that the species in question has for survival. The ability to perceive visual illusions is found in independently evolved visual systems, from honeybees to humans. In humans, the ability emerges early, at the age of four months. Thus the perception of illusion is likely to reflect visual processes of fundamental importance for object perception in natural vision. The experiments reported in this thesis employed various modifications of the Kanizsa triangle, a drawn configuration composed of three black disks with missing sectors on a white background. The sectors appear to form the tips of a triangle. The visual system completes the physically empty area between the disks, generally called inducers, with giving the perception of an illusory triangle. The illusory triangle consists of an illusory surface bounded by illusory contours; the triangle appears brighter than and to lie above the background. If the sectors are coloured, the colour fills the illusory area, a phenomenon known as neon colour spreading . We investigated spatial limitations on the perception of Kanizsa-type illusions and how other stimuli and viewing parameters affected these limitations. We also studied complex configurations thick, bent, mobile and chromatic inducers - to determine whether illusions combining several attributes can be perceived. The results suggest that the visual system is highly effective in completing a percept. The perception of an illusory figure is spatially scale invariant when perceived at threshold. The processing time and the number of fixations modify the percept, making the perception of the illusion more probable in various viewing conditions. Furthermore, the fact that the illusion can be perceived when only one inducer is physically present at any given moment indicates the potential of single inducers. Apparently, modelling illusory figure perception will require a combination of low-level, local processes and higher-level integrative processes. Our studies with stimuli combining several attributes relevant to object perception demonstrate that the perception of an illusory figure is flexible and is maintained also when it contains colour and volume and when shown in movement. All in all, the results confirm the assumed importance of the visual processes related with the perception of illusory figures in everyday viewing. This is indicated by the variety of inducer modifications that can be made without destroying the percept. Furthermore, the illusion can acquire additional attributes from such modifications. Due to individual differences in the perception of illusory figures, universal values for absolute performance are not always meaningful, but stable trends and general relations do exist.

Cortical processing of speech and non-speech sounds in autism and Asperger syndrome

Autism and Asperger syndrome (AS) are neurodevelopmental disorders characterised by deficient social and communication skills, as well as restricted, repetitive patterns of behaviour. The language development in individuals with autism is significantly delayed and deficient, whereas in individuals with AS, the structural aspects of language develop quite normally. Both groups, however, have semantic-pragmatic language deficits. The present thesis investigated auditory processing in individuals with autism and AS. In particular, the discrimination of and orienting to speech and non-speech sounds was studied, as well as the abstraction of invariant sound features from speech-sound input. Altogether five studies were conducted with auditory event-related brain potentials (ERP); two studies also included a behavioural sound-identification task. In three studies, the subjects were children with autism, in one study children with AS, and in one study adults with AS. In children with autism, even the early stages of sound encoding were deficient. In addition, these children had altered sound-discrimination processes characterised by enhanced spectral but deficient temporal discrimination. The enhanced pitch discrimination may partly explain the auditory hypersensitivity common in autism, and it may compromise the filtering of relevant auditory information from irrelevant information. Indeed, it was found that when sound discrimination required abstracting invariant features from varying input, children with autism maintained their superiority in pitch processing, but lost it in vowel processing. Finally, involuntary orienting to sound changes was deficient in children with autism in particular with respect to speech sounds. This finding is in agreement with previous studies on autism suggesting deficits in orienting to socially relevant stimuli. In contrast to children with autism, the early stages of sound encoding were fairly unimpaired in children with AS. However, sound discrimination and orienting were rather similarly altered in these children as in those with autism, suggesting correspondences in the auditory phenotype in these two disorders which belong to the same continuum. Unlike children with AS, adults with AS showed enhanced processing of duration changes, suggesting developmental changes in auditory processing in this disorder.

Neuronal oscillations in gamma- and alpha-frequency bands: from object representations to sensory awareness

The synchronization of neuronal activity, especially in the beta- (14-30 Hz) /gamma- (30 80 Hz) frequency bands, is thought to provide a means for the integration of anatomically distributed processing and for the formation of transient neuronal assemblies. Thus non-stimulus locked (i.e. induced) gamma-band oscillations are believed to underlie feature binding and the formation of neuronal object representations. On the other hand, the functional roles of neuronal oscillations in slower theta- (4 8 Hz) and alpha- (8 14 Hz) frequency bands remain controversial. In addition, early stimulus-locked activity has been largely ignored, as it is believed to reflect merely the physical properties of sensory stimuli. With human neuromagnetic recordings, both the functional roles of gamma- and alpha-band oscillations and the significance of early stimulus-locked activity in neuronal processing were examined in this thesis. Study I of this thesis shows that even the stimulus-locked (evoked) gamma oscillations were sensitive to high-level stimulus features for speech and non-speech sounds, suggesting that they may underlie the formation of early neuronal object representations for stimuli with a behavioural relevance. Study II shows that neuronal processing for consciously perceived and unperceived stimuli differed as early as 30 ms after stimulus onset. This study also showed that the alpha band oscillations selectively correlated with conscious perception. Study III, in turn, shows that prestimulus alpha-band oscillations influence the subsequent detection and processing of sensory stimuli. Further, in Study IV, we asked whether phase synchronization between distinct frequency bands is present in cortical circuits. This study revealed prominent task-sensitive phase synchrony between alpha and beta/gamma oscillations. Finally, the implications of Studies II, III, and IV to the broader scientific context are analysed in the last study of this thesis (V). I suggest, in this thesis that neuronal processing may be extremely fast and that the evoked response is important for cognitive processes. I also propose that alpha oscillations define the global neuronal workspace of perception, action, and consciousness and, further, that cross-frequency synchronization is required for the integration of neuronal object representations into global neuronal workspace.

Processing of Stimulus Repetition and Change in the Somatosensory System: Recordings of Electrical and Magnetic Brain Responses

In the present work, effects of stimulus repetition and change in a continuous stimulus stream on the processing of somatosensory information in the human brain were studied. Human scalp-recorded somatosensory event-related potentials (ERPs) and magnetoencephalographic (MEG) responses rapidly diminished with stimulus repetition when mechanical or electric stimuli were applied to fingers. On the contrary, when the ERPs and multi-unit a ctivity (MUA) were directly recorded from the primary (SI) and secondary (SII) somatosensory cortices in a monkey, there was no marked decrement in the somatosensory responses as a function of stimulus repetition. These results suggest that this rate effect is not due to the response diminution in the SI and SII cortices. Obviously the responses to the first stimulus after a long "silent" period are nhanced due to unspecific initial orientation, originating in more broadly distributed and/or deeper neural structures, perhaps in the prefrontal cortices. With fast repetition rates not only the late unspecific but also some early specific somatosensory ERPs were diminished in amplitude. The fast decrease of the ERPs as a function of stimulus repetition is mainly due to the disappearance of the orientation effect and with faster repetition rates additively due to stimulus specific refractoriness. A sudden infrequent change in the continuous stimulus stream also enhanced somatosensory MEG responses to electric stimuli applied to different fingers. These responses were quite similar to those elicited by the deviant stimuli alone when the frequent standard stimuli were omitted. This enhancement was obviously due to the release from refractoriness because the neural structures generating the responses to the infrequent deviants had more time to recover from the refractoriness than the respective structures for the standards. Infrequent deviant mechanical stimuli among frequent standard stimuli also enhanced somatosensory ERPs and, in addition, they elicited a new negative wave which did not occur in the deviants-alone condition. This extra negativity could be recorded to deviations in the stimulation site and in the frequency of the vibratory stimuli. This response is probably a somatosensory analogue of the auditory mismatch negativity (MMN) which has been suggested to reflect a neural mismatch process between the sensory input and the sensory memory trace.

Neurochemical regulation of auditory information processing studied with EEG/MEG: application to schizophrenia

Cognitive impairments of attention, memory and executive functions are a fundamental feature of the pathophysiology of schizophrenia. The neurophysiological and neurochemical changes in the auditory cortex are shown to underlie cognitive impairmentsin schizophrenia patients. Functional state of the neural substrate of auditory information processing could be objectively and non-invasively probed with auditory event-related potentials (ERPs) and event- related fields (ERFs). In the current work, we explored the neurochemical effect on the neural origins of auditory information processing in relation to schizophrenia. By means of ERPs/ERFs we aimed to determine how neural substrates of auditory information processing are modulated by antipsychotic medication in schizophrenia spectrum patients (Studies I, II) and by neuropharmacological challenges in healthy human subjects (Studies III, IV). First, with auditory ERPs we investigated the effects of olanzapine (Study I) and risperidone (Study II) in a group of patients with schizophrenia spectrum disorders. After 2 and 4 weeks of treatment, olanzapine has no significant effects on mismatch negativity(MMN) and P300, which, as it has been suggested, respectively reflect preattentive and attention-dependent information processing. After 2 weeks of treatment, risperidone has no significant effect on P300, however risperidone reduces P200 amplitude. This latter effect of risperidone on neural resources responsible for P200 generation could be partly explained through the action of dopamine. Subsequently, we used simultaneous EEG/MEG to investigate the effects of memantine (Study III) and methylphenidate (Study IV) in healthy subjects. We found that memantine modulates MMN response without changing other ERP components. This could be interpreted as being due to the possible influence of memantine through the NMDA receptors on auditory change- detection mechanism, with processing of auditory stimuli remaining otherwise unchanged. Further, we found that methylphenidate does not modulate the MMN response. This finding could indicate no association between catecholaminergic activities and electrophysiological measures of preattentive auditory discrimination processes reflected in the MMN. However, methylphenidate decreases the P200 amplitudes. This could be interpreted as a modulation of auditory information processing reflected in P200 by dopaminergic and noradrenergic systems. Taken together, our set of studies indicates a complex pattern of neurochemical influences produced by the antipsychotic drugs in the neural substrate of auditory information processing in patients with schizophrenia spectrum disorders and by the pharmacological challenges in healthy subjects studied with ERPs and ERFs.

Constructing skilled images

When experts construct mental images, they do not rely only on perceptual features; they also access domain-specific knowledge and skills in long-term memory, which enables them to exceed the capacity limitations of the short-term working memory system. The central question of the present dissertation was whether the facilitating effect of long-term memory knowledge on working memory imagery tasks is primarily based on perceptual chunking or whether it relies on higher-level conceptual knowledge. Three domains of expertise were studied: chess, music, and taxi driving. The effects of skill level, stimulus surface features, and the stimulus structure on incremental construction of mental images were investigated. A method was developed to capture the chunking mechanisms that experts use in constructing images: chess pieces, street names, and visual notes were presented in a piecemeal fashion for later recall. Over 150 experts and non-experts participated in a total of 13 experiments, as reported in five publications. The results showed skill effects in all of the studied domains when experts performed memory and problem solving tasks that required mental imagery. Furthermore, only experts' construction of mental images benefited from meaningful stimuli. Manipulation of the stimulus surface features, such as replacing chess pieces with dots, did not significantly affect experts' performance in the imagery tasks. In contrast, the structure of the stimuli had a significant effect on experts' performance in every task domain. For example, taxi drivers recalled more street names from lists that formed a spatially continuous route than from alphabetically organised lists. The results suggest that the mechanisms of conceptual chunking rather than automatic perceptual pattern matching underlie expert performance, even though the tasks of the present studies required perception-like mental representations. The results show that experts are able to construct skilled images that surpass working memory capacity, and that their images are conceptually organised and interpreted rather than merely depictive.

Feature integration in human vision

The earliest stages of human cortical visual processing can be conceived as extraction of local stimulus features. However, more complex visual functions, such as object recognition, require integration of multiple features. Recently, neural processes underlying feature integration in the visual system have been under intensive study. A specialized mid-level stage preceding the object recognition stage has been proposed to account for the processing of contours, surfaces and shapes as well as configuration. This thesis consists of four experimental, psychophysical studies on human visual feature integration. In two studies, classification image a recently developed psychophysical reverse correlation method was used. In this method visual noise is added to near-threshold stimuli. By investigating the relationship between random features in the noise and observer s perceptual decision in each trial, it is possible to estimate what features of the stimuli are critical for the task. The method allows visualizing the critical features that are used in a psychophysical task directly as a spatial correlation map, yielding an effective "behavioral receptive field". Visual context is known to modulate the perception of stimulus features. Some of these interactions are quite complex, and it is not known whether they reflect early or late stages of perceptual processing. The first study investigated the mechanisms of collinear facilitation, where nearby collinear Gabor flankers increase the detectability of a central Gabor. The behavioral receptive field of the mechanism mediating the detection of the central Gabor stimulus was measured by the classification image method. The results show that collinear flankers increase the extent of the behavioral receptive field for the central Gabor, in the direction of the flankers. The increased sensitivity at the ends of the receptive field suggests a low-level explanation for the facilitation. The second study investigated how visual features are integrated into percepts of surface brightness. A novel variant of the classification image method with brightness matching task was used. Many theories assume that perceived brightness is based on the analysis of luminance border features. Here, for the first time this assumption was directly tested. The classification images show that the perceived brightness of both an illusory Craik-O Brien-Cornsweet stimulus and a real uniform step stimulus depends solely on the border. Moreover, the spatial tuning of the features remains almost constant when the stimulus size is changed, suggesting that brightness perception is based on the output of a single spatial frequency channel. The third and fourth studies investigated global form integration in random-dot Glass patterns. In these patterns, a global form can be immediately perceived, if even a small proportion of random dots are paired to dipoles according to a geometrical rule. In the third study the discrimination of orientation structure in highly coherent concentric and Cartesian (straight) Glass patterns was measured. The results showed that the global form was more efficiently discriminated in concentric patterns. The fourth study investigated how form detectability depends on the global regularity of the Glass pattern. The local structure was either Cartesian or curved. It was shown that randomizing the local orientation deteriorated the performance only with the curved pattern. The results give support for the idea that curved and Cartesian patterns are processed in at least partially separate neural systems.

Tarkkaavaisuuden vaikutus puheen merkityssisällön käsittelyyn: Aivojen herätevastetutkimus

Semantic processing can be studied with semantic priming. Target words that are preceded by semantically related prime words are recognized faster and more accurately than targets preceded by unrelated prime words. Semantic priming also affects the magnitude of the N400 event-related potential. The response is smaller to a target word when it is preceded by a related than an unrelated prime word. The effect is called the N400 effect. It is not yet clear, however, how attention modulates semantic priming and the N400 effect. This study investigated how the direction of attention affects the semantic processing of speech. The N400 effect was studied in experimental conditions in which the subjects attention was directed 1) away from the speech stimuli, 2) to phonological features of the speech stimuli, and 3) to semantic features of the speech stimuli. The first aim of the study was to investigate whether the N400 effect for spoken words is dependent on attention to the auditory information. The second aim was to study the differences in the N400 effect when attention is directed to the semantic or other features of speech stimuli. The results showed an N400 effect even when attention was directed away from the speech stimuli. The N400 effect was, however, stronger in conditions during which the speech stimuli were attended. The magnitude of the behavioral semantic priming and the N400 effect did not differ between the conditions during which attention was directed to the semantic or phonological features of the words. The findings indicate that the semantic processing of spoken words is not dependent on attention to auditory information. Furthermore, the results suggest that whether or not semantic processing is relevant for the task performance does not affect the semantic processing of attended spoken words.

Integrated semantic processing of complex pictures and spoken sentences - Evidence from event-related potentials

It has been suggested that semantic information processing is modularized according to the input form (e.g., visual, verbal, non-verbal sound). A great deal of research has concentrated on detecting a separate verbal module. Also, it has traditionally been assumed in linguistics that the meaning of a single clause is computed before integration to a wider context. Recent research has called these views into question. The present study explored whether it is reasonable to assume separate verbal and nonverbal semantic systems in the light of the evidence from event-related potentials (ERPs). The study also provided information on whether the context influences processing of a single clause before the local meaning is computed. The focus was on an ERP called N400. Its amplitude is assumed to reflect the effort required to integrate an item to the preceding context. For instance, if a word is anomalous in its context, it will elicit a larger N400. N400 has been observed in experiments using both verbal and nonverbal stimuli. Contents of a single sentence were not hypothesized to influence the N400 amplitude. Only the combined contents of the sentence and the picture were hypothesized to influence the N400. The subjects (n = 17) viewed pictures on a computer screen while hearing sentences through headphones. Their task was to judge the congruency of the picture and the sentence. There were four conditions: 1) the picture and the sentence were congruent and sensible, 2) the sentence and the picture were congruent, but the sentence ended anomalously, 3) the picture and the sentence were incongruent but sensible, 4) the picture and the sentence were incongruent and anomalous. Stimuli from the four conditions were presented in a semi-randomized sequence. Their electroencephalography was simultaneously recorded. ERPs were computed for the four conditions. The amplitude of the N400 effect was largest in the incongruent sentence-picture -pairs. The anomalously ending sentences did not elicit a larger N400 than the sensible sentences. The results suggest that there is no separate verbal semantic system, and that the meaning of a single clause is not processed independent of the context.

Tarinakontekstin vaikutus sanojen muistamiseen kompleksisessa työmuistitehtävässä

Objectives. The sentence span task is a complex working memory span task used for estimating total working memory capacity for both processing (sentence comprehension) and storage (remembering a set of words). Several traditional models of working memory suggest that performance on these tasks relies on phonological short-term storage. However, long-term memory effects as well as the effects of expertise and strategies have challenged this view. This study uses a working memory task that aids the creation of retrieval structures in the form of stories, which have been shown to form integrated structures in longterm memory. The research question is whether sentence and story contexts boost memory performance in a complex working memory task. The hypothesis is that storage of the words in the task takes place in long-term memory. Evidence of this would be better recall for words as parts of sentences than for separate words, and, particularly, a beneficial effect for words as part of an organized story. Methods. Twenty stories consisting of five sentences each were constructed, and the stimuli in all experimental conditions were based on these sentences and sentence-final words, reordered and recombined for the other conditions. Participants read aloud sets of five sentences that either formed a story or not. In one condition they had to report all the last words at the end of the set, in another, they memorised an additional separate word with each sentence. The sentences were presented on the screen one word at a time (500 ms). After the presentation of each sentence, the participant verified a statement about the sentence. After five sentences, the participant repeated back the words in correct positions. Experiment 1 (n=16) used immediate recall, experiment 2 (n=21) both immediate recall and recall after a distraction interval (the operation span task). In experiment 2 a distracting mental arithmetic task was presented instead of recall in half of the trials, and an individual word was added before each sentence in the two experimental conditions when the participants were to memorize the sentence final words. Subjects also performed a listening span task (in exp.1) or an operation span task (exp.2) to allow comparison of the estimated span and performance in the story task. Results were analysed using correlations, repeated measures ANOVA and a chi-square goodness of fit test on the distribution of errors. Results and discussion. Both the relatedness of the sentences (the story condition) and the inclusion of the words into sentences helped memory. An interaction showed that the story condition had a greater effect on last words than separate words. The beneficial effect of the story was shown in all serial positions. The effects remained in delayed recall. When the sentences formed stories, performance in verification of the statements about sentence context was better. This, as well as the differing distributions of errors in different experimental conditions, suggest different levels of representation are in use in the different conditions. In the story condition, the nature of these representations could be in the form of an organized memory structure, a situation model. The other working memory tasks had only few week correlations to the story task. This could indicate that different processes are in use in the tasks. The results do not support short-term phonological storage, but instead are compatible with the words being encoded to LTM during the task.