Investigating online reading with eye tracking and EEG : The influence of text format, reading task and parafoveal stimuli on reading processes

Research on reading has been successful in revealing how attention guides eye movements when people read single sentences or text paragraphs in simplified and strictly controlled experimental conditions. However, less is known about reading processes in more naturalistic and applied settings, such as reading Web pages. This thesis investigates online reading processes by recording participants eye movements. The thesis consists of four experimental studies that examine how location of stimuli presented outside the currently fixated region (Study I and III), text format (Study II), animation and abrupt onset of online advertisements (Study III), and phase of an online information search task (Study IV) affect written language processing. Furthermore, the studies investigate how the goal of the reading task affects attention allocation during reading by comparing reading for comprehension with free browsing, and by varying the difficulty of an information search task. The results show that text format affects the reading process, that is, vertical text (word/line) is read at a slower rate than a standard horizontal text, and the mean fixation durations are longer for vertical text than for horizontal text. Furthermore, animated online ads and abrupt ad onsets capture online readers attention and direct their gaze toward the ads, and distract the reading process. Compared to a reading-for-comprehension task, online ads are attended to more in a free browsing task. Moreover, in both tasks abrupt ad onsets result in rather immediate fixations toward the ads. This effect is enhanced when the ad is presented in the proximity of the text being read. In addition, the reading processes vary when Web users proceed in online information search tasks, for example when they are searching for a specific keyword, looking for an answer to a question, or trying to find a subjectively most interesting topic. A scanning type of behavior is typical at the beginning of the tasks, after which participants tend to switch to a more careful reading state before finishing the tasks in the states referred to as decision states. Furthermore, the results also provided evidence that left-to-right readers extract more parafoveal information to the right of the fixated word than to the left, suggesting that learning biases attentional orienting towards the reading direction.

Proactive control of sequential saccades in the human supplementary eye field

Our ability to regulate behavior based on past experience has thus far been examined using single movements. However, natural behavior typically involves a sequence of movements. Here, we examined the effect of previous trial type on the concurrent planning of sequential saccades using a unique paradigm. The task consisted of two trial types: no-shift trials, which implicitly encouraged the concurrent preparation of the second saccade in a subsequent trial; and target-shift trials, which implicitly discouraged the same in the next trial. Using the intersaccadic interval as an index of concurrent planning, we found evidence for context-based preparation of sequential saccades. We also used functional MRI-guided, single-pulse, transcranial magnetic stimulation on human subjects to test the role of the supplementary eye field (SEF) in the proactive control of sequential eye movements. Results showed that (i) stimulating the SEF in the previous trial disrupted the previous trial type-based preparation of the second saccade in the nonstimulated current trial, (ii) stimulating the SEF in the current trial rectified the disruptive effect caused by stimulation in the previous trial, and (iii) stimulating the SEF facilitated the preparation of second saccades based on previous trial type even when the previous trial was not stimulated. Taken together, we show how the human SEF is causally involved in proactive preparation of sequential saccades.

Neural Dynamics of Saccadic and Smooth Pursuit Eye Movement Coordination during Visual Tracking of Unpredictably Moving Targets

How does the brain use eye movements to track objects that move in unpredictable directions and speeds? Saccadic eye movements rapidly foveate peripheral visual or auditory targets and smooth pursuit eye movements keep the fovea pointed toward an attended moving target. Analyses of tracking data in monkeys and humans reveal systematic deviations from predictions of the simplest model of saccade-pursuit interactions, which would use no interactions other than common target selection and recruitment of shared motoneurons. Instead, saccadic and smooth pursuit movements cooperate to cancel errors of gaze position and velocity, and thus to maximize target visibility through time. How are these two systems coordinated to promote visual localization and identification of moving targets? How are saccades calibrated to correctly foveate a target despite its continued motion during the saccade? A neural model proposes answers to such questions. The modeled interactions encompass motion processing areas MT, MST, FPA, DLPN and NRTP; saccade planning and execution areas FEF and SC; the saccadic generator in the brain stem; and the cerebellum. Simulations illustrate the model’s ability to functionally explain and quantitatively simulate anatomical, neurophysiological and behavioral data about SAC-SPEM tracking.

Target Selection by Frontal Cortex During Coordinated Saccadic and Smooth Pursuit Eye Movement

Oculomotor tracking of moving objects is an important component of visually based cognition and planning. Such tracking is achieved by a combination of saccades and smooth pursuit eye movements. In particular, the saccadic and smooth pursuit systems interact to often choose the same target, and to maximize its visibility through time. How do multiple brain regions interact, including frontal cortical areas, to decide the choice of a target among several competing moving stimuli? How is target selection information that is created by a bias (e.g., electrical stimulation) transferred from one movement system to another? These saccade-pursuit interactions are clarified by a new computational neural model, which describes interactions among motion processing areas MT, MST, FPA, DLPN; saccade specification, selection, and planning areas LIP, FEF, SNr, SC; the saccadic generator in the brain stem; and the cerebellum. Model simulations explain a broad range of neuroanatomical and neurophysiological data. These results are in contrast with the simplest parallel model with no interactions between saccades and pursuit than common-target selection and recruitment of shared motoneurons. Actual tracking episodes in primates reveal multiple systematic deviations from predictions of the simplest parallel model, which are explained by the current model.

Division of labor in frontal eye field neurons during presaccadic remapping of visual receptive fields.

Our percept of visual stability across saccadic eye movements may be mediated by presaccadic remapping. Just before a saccade, neurons that remap become visually responsive at a future field (FF), which anticipates the saccade vector. Hence, the neurons use corollary discharge of saccades. Many of the neurons also decrease their response at the receptive field (RF). Presaccadic remapping occurs in several brain areas including the frontal eye field (FEF), which receives corollary discharge of saccades in its layer IV from a collicular-thalamic pathway. We studied, at two levels, the microcircuitry of remapping in the FEF. At the laminar level, we compared remapping between layers IV and V. At the cellular level, we compared remapping between different neuron types of layer IV. In the FEF in four monkeys (Macaca mulatta), we identified 27 layer IV neurons with orthodromic stimulation and 57 layer V neurons with antidromic stimulation from the superior colliculus. With the use of established criteria, we classified the layer IV neurons as putative excitatory (n = 11), putative inhibitory (n = 12), or ambiguous (n = 4). We found that just before a saccade, putative excitatory neurons increased their visual response at the RF, putative inhibitory neurons showed no change, and ambiguous neurons increased their visual response at the FF. None of the neurons showed presaccadic visual changes at both RF and FF. In contrast, neurons in layer V showed full remapping (at both the RF and FF). Our data suggest that elemental signals for remapping are distributed across neuron types in early cortical processing and combined in later stages of cortical microcircuitry.

Similar prevalence and magnitude of auditory-evoked and visually evoked activity in the frontal eye fields: implications for multisensory motor control.

Saccadic eye movements can be elicited by more than one type of sensory stimulus. This implies substantial transformations of signals originating in different sense organs as they reach a common motor output pathway. In this study, we compared the prevalence and magnitude of auditory- and visually evoked activity in a structure implicated in oculomotor processing, the primate frontal eye fields (FEF). We recorded from 324 single neurons while 2 monkeys performed delayed saccades to visual or auditory targets. We found that 64% of FEF neurons were active on presentation of auditory targets and 87% were active during auditory-guided saccades, compared with 75 and 84% for visual targets and saccades. As saccade onset approached, the average level of population activity in the FEF became indistinguishable on visual and auditory trials. FEF activity was better correlated with the movement vector than with the target location for both modalities. In summary, the large proportion of auditory-responsive neurons in the FEF, the similarity between visual and auditory activity levels at the time of the saccade, and the strong correlation between the activity and the saccade vector suggest that auditory signals undergo tailoring to match roughly the strength of visual signals present in the FEF, facilitating accessing of a common motor output pathway.

A pathway in primate brain for internal monitoring of movements.

It is essential to keep track of the movements we make, and one way to do that is to monitor correlates, or corollary discharges, of neuronal movement commands. We hypothesized that a previously identified pathway from brainstem to frontal cortex might carry corollary discharge signals. We found that neuronal activity in this pathway encodes upcoming eye movements and that inactivating the pathway impairs sequential eye movements consistent with loss of corollary discharge without affecting single eye movements. These results identify a pathway in the brain of the primate Macaca mulatta that conveys corollary discharge signals.

Reversible inactivation of macaque frontal eye field.

The macaque frontal eye field (FEF) is involved in the generation of saccadic eye movements and fixations. To better understand the role of the FEF, we reversibly inactivated a portion of it while a monkey made saccades and fixations in response to visual stimuli. Lidocaine was infused into a FEF and neural inactivation was monitored with a nearby microelectrode. We used two saccadic tasks. In the delay task, a target was presented and then extinguished, but the monkey was not allowed to make a saccade to its location until a cue to move was given. In the step task, the monkey was allowed to look at a target as soon as it appeared. During FEF inactivation, monkeys were severely impaired at making saccades to locations of extinguished contralateral targets in the delay task. They were similarly impaired at making saccades to locations of contralateral targets in the step task if the target was flashed for < or =100 ms, such that it was gone before the saccade was initiated. Deficits included increases in saccadic latency, increases in saccadic error, and increases in the frequency of trials in which a saccade was not made. We varied the initial fixation location and found that the impairment specifically affected contraversive saccades rather than affecting all saccades made into head-centered contralateral space. Monkeys were impaired only slightly at making saccades to contralateral targets in the step task if the target duration was 1000 ms, such that the target was present during the saccade: latency increased, but increases in saccadic error were mild and increases in the frequency of trials in which a saccade was not made were insignificant. During FEF inactivation there usually was a direct correlation between the latency and the error of saccades made in response to contralateral targets. In the delay task, FEF inactivation increased the frequency of making premature saccades to ipsilateral targets. FEF inactivation had inconsistent and mild effects on saccadic peak velocity. FEF inactivation caused impairments in the ability to fixate lights steadily in contralateral space. FEF inactivation always caused an ipsiversive deviation of the eyes in darkness. In summary, our results suggest that the FEF plays major roles in (1) generating contraversive saccades to locations of extinguished or flashed targets, (2) maintaining contralateral fixations, and (3) suppressing inappropriate ipsiversive saccades.

Visual attention to food cues in obesity: An eye‐tracking study.

Objective
Based on the theory of incentive sensitization, the aim of this study was to investigate differences in attentional processing of food-related visual cues between normal-weight and overweight/obese males and females.

Methods
Twenty-six normal-weight (14M, 12F) and 26 overweight/obese (14M, 12F) adults completed a visual probe task and an eye-tracking paradigm. Reaction times and eye movements to food and control images were collected during both a fasted and fed condition in a counterbalanced design.

Results
Participants had greater visual attention towards high-energy-density food images compared to low-energy-density food images regardless of hunger condition. This was most pronounced in overweight/obese males who had significantly greater maintained attention towards high-energy-density food images when compared with their normal-weight counterparts however no between weight group differences were observed for female participants.

Conclusions
High-energy-density food images appear to capture visual attention more readily than low-energy-density food images. Results also suggest the possibility of an altered visual food cue-associated reward system in overweight/obese males. Attentional processing of food cues may play a role in eating behaviors thus should be taken into consideration as part of an integrated approach to curbing obesity.

Attentional orienting to biologically fear-relevant stimuli: data from eye tracking

Snakes are thought as fear-relevant stimuli (biologically prepared to be associated with fear) which can lead to an enhanced attentional capture when compared fear-irrelevant stimuli. Inherent limitations related to the key-press behaviour might be bypassed with the measurement of eye movements, since they are more closely related to attentional processes than reaction times. An eye tracking technique was combined with the flicker paradigm in two studies. A sample of university students was gathered. In both studies, an instruction to detect changes between the pair of scenes was given. Attentional orienting for the changing element in the scene was analyzed, as well the role of fear of snakes as a moderator variable. The results for both studies revealed a significant shorter time to first fixation for snake stimuli when compared to control stimuli. A facilitating effect of fear of snakes was also found for snakes, presenting the highly fear participants a shorter a time to first fixation for snake stimuli when compared to low-feared participants. The results are in line with current research that supports the advantage of snakes to grab attention due their evo-biological significance.

Using eye-tracking to detect reading difficulties

Taking into account the study of Luegi (2006), where eye movements of 20 Portuguese university students while reading text passages were analyzed, in this article we discuss some methodological issues concerning eye tracking measures to evaluate reading difficulties. Relating syntactic complexity, grammaticality and ambiguity to eye movements, we will discuss the use of many different dependent variables that indicate the immediate and delayed processes in text processing. We propose a new measure that we called Progression-Path which permits analyzing, in the critical region, what happens when the reader proceeds on the sentence instead of going backwards to solve a problem that s/he found (which is the most common expected behavior but not the only one, as is illustrated by some of our examples).

Using eye-tracking to study anaphoric relations processing in European Portuguese

In this study we evaluate processing costs of different types of anaphoric expressions during reading. We consider three types of anaphoric expressions in Subject sentential position: a null pronoun (pro), and two gaps produced by syntactic movement: a WHvariable and a NP copy. Given that coreferential pro exhibits more referential weight than wh- and NP-gaps, and grounded on theories of referential processing based on relations of hierarchy and accessibility of the antecedent, we raise the hypothesis that the more dependent on its antecedent the anaphoric null constituent is, and the more minimal is the distance in terms of hierarchical structure between the anaphoric null element and its antecedent, the lower are the cognitive costs in processing. To test our hypothesis, we registered the eye movements with R6-HS ASL system of 20 Portuguese adult native speakers. Text regions including the selected anaphoric expressions were delimited and tagged. We analyzed the reading time of each region taking into account the number and duration of eye fixations per region; we used the reading time by character in milliseconds in order to compare values between regions of different length. We found a significant advantage in the reading time of the gaps arising from movement over the reading time of pro.

How active gaze informs the hand in sequential pointing movements

Visual information is vital for fast and accurate hand movements. It has been demonstrated that allowing free eye movements results in greater accuracy than when the eyes maintain centrally fixed. Three explanations as to why free gaze improves accuracy are: shifting gaze to a target allows visual feedback in guiding the hand to the target (feedback loop), shifting gaze generates ocular-proprioception which can be used to update a movement (feedback-feedforward), or efference copy could be used to direct hand movements (feedforward). In this experiment we used a double-step task and manipulated the utility of ocular-proprioceptive feedback from eye to head position by removing the second target during the saccade. We confirm the advantage of free gaze for sequential movements with a double-step pointing task and document eye-hand lead times of approximately 200 ms for both initial movements and secondary movements. The observation that participants move gaze well ahead of the current hand target dismisses foveal feedback as a major contribution. We argue for a feedforward model based on eye movement efference as the major factor in enabling accurate hand movements. The results with the double-step target task also suggest the need for some buffering of efference and ocular-proprioceptive signals to cope with the situation where the eye has moved to a location ahead of the current target for the hand movement. We estimate that this buffer period may range between 120 and 200 ms without significant impact on hand movement accuracy.

The integration of figurative language and static depictions: an eye movement study of fictive motion

Do we view the world differently if it is described to us in figurative rather than literal terms? An answer to this question would reveal something about both the conceptual representation of figurative language and the scope of top-down influences oil scene perception. Previous work has shown that participants will look longer at a path region of a picture when it is described with a type of figurative language called fictive motion (The road goes through the desert) rather than without (The road is in the desert). The current experiment provided evidence that such fictive motion descriptions affect eye movements by evoking mental representations of motion. If participants heard contextual information that would hinder actual motion, it influenced how they viewed a picture when it was described with fictive motion. Inspection times and eye movements scanning along the path increased during fictive motion descriptions when the terrain was first described as difficult (The desert is hilly) as compared to easy (The desert is flat); there were no such effects for descriptions without fictive motion. It is argued that fictive motion evokes a mental simulation of motion that is immediately integrated with visual processing, and hence figurative language can have a distinct effect on perception. (c) 2005 Elsevier B.V. All rights reserved.