Communicating eye gaze across a distance without rooting participants to the spot

Eye gaze is an important conversational resource that until now could only be supported across a distance if people were rooted to the spot. We introduce EyeCVE, the worldpsilas first tele-presence system that allows people in different physical locations to not only see what each other are doing but follow each otherpsilas eyes, even when walking about. Projected into each space are avatar representations of remote participants, that reproduce not only body, head and hand movements, but also those of the eyes. Spatial and temporal alignment of remote spaces allows the focus of gaze as well as activity and gesture to be used as a resource for non-verbal communication. The temporal challenge met was to reproduce eye movements quick enough and often enough to interpret their focus during a multi-way interaction, along with communicating other verbal and non-verbal language. The spatial challenge met was to maintain communicational eye gaze while allowing free movement of participants within a virtually shared common frame of reference. This paper reports on the technical and especially temporal characteristics of the system.

Eye tracking and gaze based interaction within immersive virtual environments

Our eyes are input sensors which Provide our brains with streams of visual data. They have evolved to be extremely efficient, and they will constantly dart to-and-fro to rapidly build up a picture of the salient entities in a viewed scene. These actions are almost subconscious. However, they can provide telling signs of how the brain is decoding the visuals and call indicate emotional responses, prior to the viewer becoming aware of them. In this paper we discuss a method of tracking a user's eye movements, and Use these to calculate their gaze within an immersive virtual environment. We investigate how these gaze patterns can be captured and used to identify viewed virtual objects, and discuss how this can be used as a, natural method of interacting with the Virtual Environment. We describe a flexible tool that has been developed to achieve this, and detail initial validating applications that prove the concept.

Eye gaze in virtual environments: evaluating the need and initial work on implementation

For efficient collaboration between participants, eye gaze is seen as being critical for interaction. Video conferencing either does not attempt to support eye gaze (e.g. AcessGrid) or only approximates it in round table conditions (e.g. life size telepresence). Immersive collaborative virtual environments represent remote participants through avatars that follow their tracked movements. By additionally tracking people's eyes and representing their movement on their avatars, the line of gaze can be faithfully reproduced, as opposed to approximated. This paper presents the results of initial work that tested if the focus of gaze could be more accurately gauged if tracked eye movement was added to that of the head of an avatar observed in an immersive VE. An experiment was conducted to assess the difference between user's abilities to judge what objects an avatar is looking at with only head movements being displayed, while the eyes remained static, and with eye gaze and head movement information being displayed. The results from the experiment show that eye gaze is of vital importance to the subjects correctly identifying what a person is looking at in an immersive virtual environment. This is followed by a description of the work that is now being undertaken following the positive results from the experiment. We discuss the integration of an eye tracker more suitable for immersive mobile use and the software and techniques that were developed to integrate the user's real-world eye movements into calibrated eye gaze in an immersive virtual world. This is to be used in the creation of an immersive collaborative virtual environment supporting eye gaze and its ongoing experiments. Copyright (C) 2009 John Wiley & Sons, Ltd.

Stars in their eyes: what eye-tracking reveal about multimedia perceptual quality

Perceptual multimedia quality is of paramount importance to the continued take-up and proliferation of multimedia applications: users will not use and pay for applications if they are perceived to be of low quality. Whilst traditionally distributed multimedia quality has been characterised by Quality of Service (QoS) parameters, these neglect the user perspective of the issue of quality. In order to redress this shortcoming, we characterise the user multimedia perspective using the Quality of Perception (QoP) metric, which encompasses not only a user’s satisfaction with the quality of a multimedia presentation, but also his/her ability to analyse, synthesise and assimilate informational content of multimedia. In recognition of the fact that monitoring eye movements offers insights into visual perception, as well as the associated attention mechanisms and cognitive processes, this paper reports on the results of a study investigating the impact of differing multimedia presentation frame rates on user QoP and eye path data. Our results show that provision of higher frame rates, usually assumed to provide better multimedia presentation quality, do not significantly impact upon the median coordinate value of eye path data. Moreover, higher frame rates do not significantly increase level of participant information assimilation, although they do significantly improve overall user enjoyment and quality perception of the multimedia content being shown.

Feasibility of using eye tracking to increase resolution for visual telepresence

Visual telepresence seeks to extend existing teleoperative capability by supplying the operator with a 3D interactive view of the remote environment. This is achieved through the use of a stereo camera platform which, through appropriate 3D display devices, provides a distinct image to each eye of the operator, and which is slaved directly from the operator's head and eye movements. However, the resolution within current head mounted displays remains poor, thereby reducing the operator's visual acuity. This paper reports on the feasibility of incorporation of eye tracking to increase resolution and investigates the stability and control issues for such a system. Continuous domain and discrete simulations are presented which indicates that eye tracking provides a stable feedback loop for tracking applications, though some empirical testing (currently being initiated) of such a system will be required to overcome indicated stability problems associated with micro saccades of the human operator.

Oxytocin increases eye contact during a real-time, naturalistic social interaction in males with and without autism

Autism spectrum conditions (autism) affect ~1% of the population and are characterized by deficits in social communication. Oxytocin has been widely reported to affect social-communicative function and its neural underpinnings. Here we report the first evidence that intranasal oxytocin administration improves a core problem that individuals with autism have in using eye contact appropriately in real-world social settings. A randomized double-blind, placebo-controlled, within-subjects design is used to examine how intranasal administration of 24 IU of oxytocin affects gaze behavior for 32 adult males with autism and 34 controls in a real-time interaction with a researcher. This interactive paradigm bypasses many of the limitations encountered with conventional static or computer-based stimuli. Eye movements are recorded using eye tracking, providing an objective measurement of looking patterns. The measure is shown to be sensitive to the reduced eye contact commonly reported in autism, with the autism group spending less time looking to the eye region of the face than controls. Oxytocin administration selectively enhanced gaze to the eyes in both the autism and control groups (transformed mean eye-fixation difference per second=0.082; 95% CI:0.025–0.14, P=0.006). Within the autism group, oxytocin has the most effect on fixation duration in individuals with impaired levels of eye contact at baseline (Cohen’s d=0.86). These findings demonstrate that the potential benefits of oxytocin in autism extend to a real-time interaction, providing evidence of a therapeutic effect in a key aspect of social communication.

Using E-Z Reader to examine the concurrent development of eye-movement control and reading skill

Compared to skilled adult readers, children typically make more fixations that are longer in duration, shorter saccades, and more regressions, thus reading more slowly (Blythe & Joseph, 2011). Recent attempts to understand the reasons for these differences have discovered some similarities (e.g., children and adults target their saccades similarly; Joseph, Liversedge, Blythe, White, & Rayner, 2009) and some differences (e.g., children’s fixation durations are more affected by lexical variables; Blythe, Liversedge, Joseph, White, & Rayner, 2009) that have yet to be explained. In this article, the E-Z Reader model of eye-movement control in reading (Reichle, 2011; Reichle, Pollatsek, Fisher, & Rayner, 1998) is used to simulate various eye-movement phenomena in adults versus children in order to evaluate hypotheses about the concurrent development of reading skill and eye-movement behavior. These simulations suggest that the primary difference between children and adults is their rate of lexical processing, and that different rates of (post-lexical) language processing may also contribute to some phenomena (e.g., children’s slower detection of semantic anomalies; Joseph et al., 2008). The theoretical implications of this hypothesis are discussed, including possible alternative accounts of these developmental changes, how reading skill and eye movements change across the entire lifespan (e.g., college-aged vs. elderly readers), and individual differences in reading ability.

Evaluation and revision of inferential comprehension in narrative texts: an eye movement study

We investigated the processes of how adult readers evaluate and revise their situation model during reading by monitoring their eye movements as they read narrative texts and subsequent critical sentences. In each narrative text, a short introduction primed a knowledge-based inference, followed by a target concept that was either expected (e.g., “oven”) or unexpected (e.g., “grill”) in relation to the inferred concept. Eye movements showed that readers detected a mismatch between the new unexpected information and their prior interpretation, confirming their ability to evaluate inferential information. Just below the narrative text, a critical sentence included a target word that was either congruent (e.g., “roasted”) or incongruent (e.g., “barbecued”) with the expected but not the unexpected concept. Readers spent less time reading the congruent than the incongruent target word, reflecting the facilitation of prior information. In addition, when the unexpected (but not expected) concept had been presented, participants with lower verbal (but not visuospatial) working memory span exhibited longer reading times and made more regressions (from the critical sentence to previous information) on encountering congruent information, indicating difficulty in inhibiting their initial incorrect interpretation and revising their situation model

Simulated hyperopic anisometropia and reading, visual information processing, and reading-related eye movement performance in children.

Purpose: This study investigated the impact of simulated hyperopic anisometropia and sustained near work on performance of academic-related measures in children.
Methods: Participants included 16 children (mean age: 11.1 ± 0.8 years) with minimal refractive error. Academic-related outcome measures included a reading test (Neale Analysis of Reading Ability), visual information–processing tests (Coding and Symbol Search subtests from the Wechsler Intelligence Scale for Children), and a reading-related eye movement test (Developmental Eye Movement test). Performance was assessed with and without 0.75 diopters of simulated monocular hyperopic defocus (administered in a randomized order), before and after 20 minutes of sustained near work. Unilateral hyperopic defocus was systematically assigned to either the dominant or nondominant sighting eye to evaluate the impact of ocular dominance on any performance decrements.
Results: Simulated hyperopic anisometropia and sustained near work both independently reduced performance on all of the outcome measures (P < 0.001). A significant interaction was also observed between simulated anisometropia and near work (P < 0.05), with the greatest decrement in performance observed during simulated anisometropia in combination with sustained near work. Laterality of the refractive error simulation (ocular dominance) did not significantly influence the outcome measures (P > 0.05). A reduction of up to 12% in performance was observed across the range of academic-related measures following sustained near work undertaken during the anisometropic simulation.
Conclusions: Simulated hyperopic anisometropia significantly impaired academic-related performance, particularly in combination with sustained near work. The impact of uncorrected habitual anisometropia on academic-related performance in children requires further investigation. © 2014 The Association for Research in Vision and Ophthalmology, Inc.

Analysis of slow- and fast-alpha band asymmetry during performance of a saccadic eye movement task: Dissociation between memory- and attention-driven systems

This study aimed at analyzing the relationship between slow- and fast-alpha asymmetry within frontal cortex and the planning, execution and voluntary control of saccadic eye movements (SEM), and quantitative electroencephalography (qEEG) was recorded using a 20-channel EEG system in 12 healthy participants performing a fixed (i.e., memory-driven) and a random SEM (i.e., stimulus-driven) condition. We find main effects for SEM condition in slow- and fast-alpha asymmetry at electrodes F3-F4, which are located over premotor cortex, specifically a negative asymmetry between conditions. When analyzing electrodes F7-F8, which are located over prefrontal cortex, we found a main effect for condition in slow-alpha asymmetry, particularly a positive asymmetry between conditions. In conclusion, the present approach supports the association of slow- and fast-alpha bands with the planning and preparation of SEM, and the specific role of these sub-bands for both, the attention network and the coordination and integration of sensory information with a (oculo)-motor response. (C) 2011 Elsevier B.V. All rights reserved.

Eye movement and diffusion tensor imaging analysis of treatment effects in a Niemann-Pick Type C patient

New treatment options for Niemann-Pick Type C (NPC) have recently become available. To assess the efficiency and efficacy of these new treatment markers for disease status and progression are needed. Both the diagnosis and the monitoring of disease progression are challenging and mostly rely on clinical impression and functional testing of horizontal eye movements. Diffusion tensor imaging (DTI) provides information about the microintegrity especially of white matter. We show here in a case report how DTI and measures derived from this imaging method can serve as adjunct quantitative markers for disease management in Niemann-Pick Type C. Two approaches are taken--first, we compare the fractional anisotropy (FA) in the white matter globally between a 29-year-old NPC patient and 18 healthy age-matched controls and show the remarkable difference in FA relatively early in the course of the disease. Second, a voxelwise comparison of FA values reveals where white matter integrity is compromised locally and demonstrate an individualized analysis of FA changes before and after 1year of treatment with Miglustat. This method might be useful in future treatment trials for NPC to assess treatment effects.

Loss of exploratory vertical saccades after unilateral frontal eye field damage

Despite their relevance for locomotion and social interaction in everyday situations, little is known about the cortical control of vertical saccades in humans. Results from microstimulation studies indicate that both frontal eye fields (FEFs) contribute to these eye movements. Here, we present a patient with a damaged right FEF, who hardly made vertical saccades during visual exploration. This finding suggests that, for the cortical control of exploratory vertical saccades, integrity of both FEFs is indeed important.

Saccadic eye movement disturbances in whiplash patients with persistent complaints.

In order to analyse the possible basis of subjective complaints following whiplash injury, horizontal eye movements were examined in subjects with persistent complaints ('symptomatic group') and subjects who had completely recovered ('recovered group'). The results for the symptomatic and recovered groups were compared with those for age-matched, healthy volunteers (control group). A battery of different saccade paradigms was employed: two were reflexive saccade tasks including a gap and an overlap task, and two were intentional saccade tasks consisting of an antisaccade and a memory-guided saccade task. In addition, the symptomatic and recovered groups also underwent psychiatric evaluation in a structured clinical interview, and all groups were assessed for emotional functioning using the Beck Depression Inventory (BDI). The recovered group did not differ significantly from the control group in saccade performance and emotional functioning. The symptomatic group showed dissociation of their performances of reflexive and intentional saccade tasks: performance in reflexive saccade tasks was normal, but in intentional saccade tasks the symptomatic group showed significantly impaired inhibition of unwanted reflexive saccades, impaired saccade triggering (i.e. increased latency) and a higher percentage error in amplitude in memory-guided saccades. Based on clinical interviews, no signs of major depression or dysthymia were found in any of the groups. Compared with the other two groups, the symptomatic group had significantly higher overall BDI scores, but these resulted from BDI dimensions that were non-specific to depression, viz. 'physiological manifestations' (e.g. fatigue, sleep disturbance) or 'performance difficulty' (e.g. work inhibition). In summary, in the symptomatic group the pattern of eye movement disturbances together with normal performance in reflexive saccade tasks and impaired performance in the intentional saccade tasks, especially impaired inhibitory function, suggests dysfunction of prefrontal and frontal cortical structures.

Spatial modulation of primate inferotemporal responses by eye position.

BACKGROUND: A key aspect of representations for object recognition and scene analysis in the ventral visual stream is the spatial frame of reference, be it a viewer-centered, object-centered, or scene-based coordinate system. Coordinate transforms from retinocentric space to other reference frames involve combining neural visual responses with extraretinal postural information. METHODOLOGY/PRINCIPAL FINDINGS: We examined whether such spatial information is available to anterior inferotemporal (AIT) neurons in the macaque monkey by measuring the effect of eye position on responses to a set of simple 2D shapes. We report, for the first time, a significant eye position effect in over 40% of recorded neurons with small gaze angle shifts from central fixation. Although eye position modulates responses, it does not change shape selectivity. CONCLUSIONS/SIGNIFICANCE: These data demonstrate that spatial information is available in AIT for the representation of objects and scenes within a non-retinocentric frame of reference. More generally, the availability of spatial information in AIT calls into questions the classic dichotomy in visual processing that associates object shape processing with ventral structures such as AIT but places spatial processing in a separate anatomical stream projecting to dorsal structures.

Eye Movement Measures of Cognitive Control in Children with Tourette Syndrome

Tourette Syndrome begins in childhood and is characterized by uncontrollable repetitive actions like neck craning or hopping and noises such as sniffing or chirping. Worst in early adolescence, these tics wax and wane in severity and occur in bouts unpredictably, often drawing unwanted attention from bystanders. Making matters worse, over half of children with Tourette Syndrome also suffer from comorbid, or concurrent, disorders such as attention deficit hyperactivity disorder (ADHD) and obsessive-compulsive disorder (OCD). These disorders introduce anxious thoughts, impulsivity, inattention, and mood variability that further disrupt children with Tourette Syndrome from focusing and performing well at school and home. Thus, deficits in the cognitive control functions of response inhibition, response generation, and working memory have long been ascribed to Tourette Syndrome. Yet, without considering the effect of medication, age, and comorbidity, this is a premature attribution. This study used an infrared eye tracking camera and various computer tasks requiring eye movement responses to evaluate response inhibition, response generation, and working memory in Tourette Syndrome. This study, the first to control for medication, age, and comorbidity, enrolled 39 unmedicated children with Tourette Syndrome and 29 typically developing peers aged 10-16 years who completed reflexive and voluntary eye movement tasks and diagnostic rating scales to assess symptom severities of Tourette Syndrome, ADHD, and OCD. Children with Tourette Syndrome and comorbid ADHD and/or OCD, but not children with Tourette Syndrome only, took longer to respond and made more errors and distracted eye movements compared to typically-developing children, displaying cognitive control deficits. However, increasing symptom severities of Tourette Syndrome, ADHD, and OCD correlated with one another. Thus, cognitive control deficits were not specific to Tourette Syndrome patients with comorbid conditions, but rather increase with increasing tic severity, suggesting that a majority of Tourette Syndrome patients, regardless of a clinical diagnosis of ADHD and/or OCD, have symptoms of cognitive control deficits at some level. Therefore, clinicians should evaluate and counsel all families of children with Tourette Syndrome, with or without currently diagnosed ADHD and/or OCD, about the functional ramifications of comorbid symptoms and that they may wax and wane with tic severity.