Dynamic vergence telepresence system incorporating active gaze control

Visual Telepresence system which utilize virtual reality style helmet mounted displays have a number of limitations. The geometry of the camera positions and of the display is fixed and is most suitable only for viewing elements of a scene at a particular distance. In such a system, the operator's ability to gaze around without use of head movement is severely limited. A trade off must be made between a poor viewing resolution or a narrow width of viewing field. To address these limitations a prototype system where the geometry of the displays and cameras is dynamically controlled by the eye movement of the operator has been developed. This paper explores the reasons why is necessary to actively adjust both the display system and the cameras and furthermore justifies the use of mechanical adjustment of the displays as an alternative to adjustment by electronic or image processing methods. The electronic and mechanical design is described including optical arrangements and control algorithms, An assessment of the performance of the system against a fixed camera/display system when operators are assigned basic tasks involving depth and distance/size perception. The sensitivity to variations in transient performance of the display and camera vergence is also assessed.

Coping with delays for real-time gaze control (The Fall and Rise of the Smiths Regulator)

In this paper we describe how to cope with the delays inherent in a real time control system for a steerable stereo head/eye platform. A purposive and reactive system requires the use of fast vision algorithms to provide the controller with the error signals to drive the platform. The time-critical implementation of these algorithms is necessary, not only to enable short latency reaction to real world events, but also to provide sufficiently high frequency results with small enough delays that controller remain stable. However, even with precise knowledge of that delay, nonlinearities in the plant make modelling of that plant impossible, thus precluding the use of a Smith Regulator. Moreover, the major delay in the system is in the feedback (image capture and vision processing) rather than feed forward (controller) loop. Delays ranging between 40msecs and 80msecs are common for the simple 2D processes, but might extend to several hundred milliseconds for more sophisticated 3D processes. The strategy presented gives precise control over the gaze direction of the cameras despite the lack of a priori knowledge of the delays involved. The resulting controller is shown to have a similar structure to the Smith Regulator, but with essential modifications.

From saccades to smooth pursuit: real-time gaze control using motion feedback

The authors present an active vision system which performs a surveillance task in everyday dynamic scenes. The system is based around simple, rapid motion processors and a control strategy which uses both position and velocity information. The surveillance task is defined in terms of two separate behavioral subsystems, saccade and smooth pursuit, which are demonstrated individually on the system. It is shown how these and other elementary responses to 2D motion can be built up into behavior sequences, and how judicious close cooperation between vision and control results in smooth transitions between the behaviors. These ideas are demonstrated by an implementation of a saccade to smooth pursuit surveillance system on a high-performance robotic hand/eye platform.

Oxytocin increases eye contact during a real-time, naturalistic social interaction in males with and without autism

Autism spectrum conditions (autism) affect ~1% of the population and are characterized by deficits in social communication. Oxytocin has been widely reported to affect social-communicative function and its neural underpinnings. Here we report the first evidence that intranasal oxytocin administration improves a core problem that individuals with autism have in using eye contact appropriately in real-world social settings. A randomized double-blind, placebo-controlled, within-subjects design is used to examine how intranasal administration of 24 IU of oxytocin affects gaze behavior for 32 adult males with autism and 34 controls in a real-time interaction with a researcher. This interactive paradigm bypasses many of the limitations encountered with conventional static or computer-based stimuli. Eye movements are recorded using eye tracking, providing an objective measurement of looking patterns. The measure is shown to be sensitive to the reduced eye contact commonly reported in autism, with the autism group spending less time looking to the eye region of the face than controls. Oxytocin administration selectively enhanced gaze to the eyes in both the autism and control groups (transformed mean eye-fixation difference per second=0.082; 95% CI:0.025–0.14, P=0.006). Within the autism group, oxytocin has the most effect on fixation duration in individuals with impaired levels of eye contact at baseline (Cohen’s d=0.86). These findings demonstrate that the potential benefits of oxytocin in autism extend to a real-time interaction, providing evidence of a therapeutic effect in a key aspect of social communication.

Using eye movements to investigate word frequency effects in children’s sentence reading

While eye movements have been used widely to investigate how skilled adult readers process written language, relatively little research has used this methodology with children. This is unfortunate as, as we discuss here, eye-movement studies have significant potential to inform our understanding of children’s reading development. We consider some of the empirical and theoretical issues that arise when using this methodology with children, illustrating our points with data from an experiment examining word frequency effects in 8-year-old children’s sentence reading. Children showed significantly longer gaze durations to low than high-frequency words, demonstrating that linguistic characteristics of text drive children’s eye movements as they read. We discuss these findings within the broader context of how eye-movement studies can inform our understanding of children’s reading, and can assist with the development of appropriately targeted interventions to support children as they learn to read.

Children’s and Adults’ Processing of Anomaly and Implausibility during Reading: Evidence from Eye Movements

The eye movements of 24 children and 24 adults were monitored to compare how they read sentences containing plausible, implausible, and anomalous thematic relations. In the implausible condition the incongruity occurred due to the incompatibility of two objects involved in the event denoted by the main verb. In the anomalous condition the direct object of the verb was not a possible verb argument. Adults exhibited immediate disruption with the anomalous sentences as compared to the implausible sentences as indexed by longer gaze durations on the target word. Children exhibited the same pattern of effects as adults as far as the anomalous sentences were concerned, but exhibited delayed effects of implausibility. These data indicate that while children and adults are alike in their basic thematic assignment processes during reading, children may be delayed in the efficiency with which they are able to integrate pragmatic and real world knowledge into their discourse representation.

Exploring combinations of different color and facial expression stimuli for gaze-independent BCIs

Background: Some studies have proven that a conventional visual brain computer interface (BCI) based on overt attention cannot be used effectively when eye movement control is not possible. To solve this problem, a novel visual-based BCI system based on covert attention and feature attention has been proposed and was called the gaze-independent BCI. Color and shape difference between stimuli and backgrounds have generally been used in examples of gaze-independent BCIs. Recently, a new paradigm based on facial expression changes has been presented, and obtained high performance. However, some facial expressions were so similar that users couldn't tell them apart, especially when they were presented at the same position in a rapid serial visual presentation (RSVP) paradigm. Consequently, the performance of the BCI is reduced. New Method: In this paper, we combined facial expressions and colors to optimize the stimuli presentation in the gaze-independent BCI. This optimized paradigm was called the colored dummy face pattern. It is suggested that different colors and facial expressions could help users to locate the target and evoke larger event-related potentials (ERPs). In order to evaluate the performance of this new paradigm, two other paradigms were presented, called the gray dummy face pattern and the colored ball pattern. Comparison with Existing Method(s): The key point that determined the value of the colored dummy faces stimuli in BCI systems was whether the dummy face stimuli could obtain higher performance than gray faces or colored balls stimuli. Ten healthy participants (seven male, aged 21–26 years, mean 24.5 ± 1.25) participated in our experiment. Online and offline results of four different paradigms were obtained and comparatively analyzed. Results: The results showed that the colored dummy face pattern could evoke higher P300 and N400 ERP amplitudes, compared with the gray dummy face pattern and the colored ball pattern. Online results showed that the colored dummy face pattern had a significant advantage in terms of classification accuracy (p < 0.05) and information transfer rate (p < 0.05) compared to the other two patterns. Conclusions: The stimuli used in the colored dummy face paradigm combined color and facial expressions. This had a significant advantage in terms of the evoked P300 and N400 amplitudes and resulted in high classification accuracies and information transfer rates. It was compared with colored ball and gray dummy face stimuli.

Effects of infant cleft lip on adult gaze and perceptions of ‘cuteness'

Objective: Early mother-infant interactions are impaired in the context of infant cleft lip, and are associated with adverse child psychological outcomes, but the nature of these interaction difficulties is not yet fully understood. The aim of this study was to explore adult gaze behaviour and cuteness perception, which are particularly important during early social exchanges, in response to infants with cleft lip, in order to investigate potential foundations for the interaction difficulties seen in this population. Methods: Using an eye-tracker, eye movements were recorded as adult participants viewed images of infant faces with and without cleft lip. Participants also rated each infant on a scale of cuteness. Results: Participants fixated significantly longer on the mouths of infants with cleft lip, which occurred at the expense of fixation on eyes. Severity of cleft lip was associated with the strength of fixation bias, with participants looking even longer at the mouths of infants with the most severe clefts. Infants with cleft lip were rated as significantly less cute than unaffected infants. Men rated infants as less cute than women overall, but gave particularly low ratings to infants with cleft lip Conclusions: Results demonstrate that the limited disturbance in infant facial configuration of cleft lip can significantly alter adult gaze patterns and cuteness perception. Our findings could have important implications for early interactions, and may help in the development of interventions to foster healthy development in infants with cleft lip.

Head, eye and arm coordination in table tennis

The aim of this study was to determine the role of head, eye and arm movements during the execution of a table tennis forehand stroke. Three-dimensional kinematic analysis of line-of-gaze, arm and ball was used to describe visual and motor behaviour. Skilled and less skilled participants returned the ball to cued right or left target areas under three levels of temporal constraint: pre-, early- and late-cue conditions. In the pre- and early-cue conditions, both high and low skill participants tracked the ball early in flight and kept gaze stable on a location in advance of the ball before ball-bat contact. Skilled participants demonstrated an earlier onset of ball tracking and recorded higher performance accuracy than less skilled counterparts. The manipulation of cue condition showed the limits of adaptation to maintain accuracy on the target. Participants were able to accommodate the constraints imposed by the early-cue condition by using a shorter quiet eye duration, earlier quiet eye offset and reduced arm velocity at contact. In the late-cue condition, modifications to gaze, head and arm movements were not sufficient to preserve accuracy. The findings highlight the functional coupling between perception and action during time-constrained, goal-directed actions.

Gaze pursuit and arm control of adolescent males diagnosed with attention deficit hyperactivity disorder (ADHD) and normal controls: evidence of a dissociation in processing visual information of short and long duration

Three-dimensional kinematic analysis of line of gaze, arm and ball was used to describe the visual and motor behaviour of male adolescents diagnosed with attention deficit hyperactivity disorder (ADHD). The ADHD participants were tested when both on (ADHD-On) and off (ADHD-Off) their medication and compared to age-matched normal controls in a modified table tennis task that required tracking the ball and hitting to cued right and left targets. Long-duration information was provided by a pre-cue, in which the target was illuminated approximately 2 s before the serve, and short-duration information by an early-cue illuminated about 350 ms after the serve, leaving -500 ms to select the target and perform the action. The ADHD groups differed significantly from the control group in both the pre-cue and early-cue conditions in being less accurate, in having a later onset and duration of pursuit tracking, and a higher frequency of gaze on and off the ball. The use of medication significantly reduced the gaze frequency of the ADHD participants, but surprisingly this did not lead to an increase in pursuit tracking, suggesting a barrier was reached beyond which ball flight information could not be processed. The control and ADHD groups did not differ in arm movement onset, duration and velocity in the short-duration early-cue condition; in the long-duration pre-cue condition, however, the ADHD group's movement time onset and arm velocity differed significantly from controls. The results show that the ADHD groups were able to process short-duration information without experiencing adverse effects on their motor behaviour; however, long-duration information contributed to irregular movement control.

Effects of saccadic eye movements on postural control stabilization

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Effects of saccadic eye movements on postural control in older adults

Saccadic eye movements have been shown to affect posture by decreasing the magnitude of body sway in young adults. However, there is no evidence of how the search for visual information that occurs during eye movements affects postural control in older adults. The purpose of the present study was to determine the influence of saccadic eye movements on postural control in older adults while they stood on 2 different bases of support. Twelve older adults stood upright in 70-s trials under 2 stance conditions (wide and narrow) and 3 gaze conditions (fixation, saccadic eye movements at 0.5 Hz, and saccadic eye movements at 1.1 Hz). Head and trunk sway amplitude and mean sway frequency were measured in both the anterior/posterior (AP) and medial/lateral (ML) directions. The results showed that the amplitude of body sway was reduced during saccades compared with fixation, as previously observed in young adults. However, older adults exhibited similar sway amplitude and frequency in the AP direction under the wide and narrow stance conditions, which is different from observations in young adults, who display larger sway in a narrow stance compared with a wide stance while performing saccades. These results suggest that although older adults are affected by saccadic eye movements by a decrease in the amplitude of body sway, as observed in young adults, they present a more rigid postural control strategy that does not allow larger sway during a more challenging stance condition.

Seeing Spaces: An Eye-Tracking Study Of Speech Segmentation

Speech is often a multimodal process, presented audiovisually through a talking face. One area of speech perception influenced by visual speech is speech segmentation, or the process of breaking a stream of speech into individual words. Mitchel and Weiss (2013) demonstrated that a talking face contains specific cues to word boundaries and that subjects can correctly segment a speech stream when given a silent video of a speaker. The current study expanded upon these results, using an eye tracker to identify highly attended facial features of the audiovisual display used in Mitchel and Weiss (2013). In Experiment 1, subjects were found to spend the most time watching the eyes and mouth, with a trend suggesting that the mouth was viewed more than the eyes. Although subjects displayed significant learning of word boundaries, performance was not correlated with gaze duration on any individual feature, nor was performance correlated with a behavioral measure of autistic-like traits. However, trends suggested that as autistic-like traits increased, gaze duration of the mouth increased and gaze duration of the eyes decreased, similar to significant trends seen in autistic populations (Boratston & Blakemore, 2007). In Experiment 2, the same video was modified so that a black bar covered the eyes or mouth. Both videos elicited learning of word boundaries that was equivalent to that seen in the first experiment. Again, no correlations were found between segmentation performance and SRS scores in either condition. These results, taken with those in Experiment, suggest that neither the eyes nor mouth are critical to speech segmentation and that perhaps more global head movements indicate word boundaries (see Graf, Cosatto, Strom, & Huang, 2002). Future work will elucidate the contribution of individual features relative to global head movements, as well as extend these results to additional types of speech tasks.

Gaze perception in social anxiety and social anxiety disorder

Clinical observations suggest abnormal gaze perception to be an important indicator of social anxiety disorder (SAD). Experimental research has yet paid relatively little attention to the study of gaze perception in SAD. In this article we first discuss gaze perception in healthy human beings before reviewing self-referential and threat-related biases of gaze perception in clinical and non-clinical socially anxious samples. Relative to controls, socially anxious individuals exhibit an enhanced self-directed perception of gaze directions and demonstrate a pronounced fear of direct eye contact, though findings are less consistent regarding the avoidance of mutual gaze in SAD. Prospects for future research and clinical implications are discussed.

Spatial modulation of primate inferotemporal responses by eye position.

BACKGROUND: A key aspect of representations for object recognition and scene analysis in the ventral visual stream is the spatial frame of reference, be it a viewer-centered, object-centered, or scene-based coordinate system. Coordinate transforms from retinocentric space to other reference frames involve combining neural visual responses with extraretinal postural information. METHODOLOGY/PRINCIPAL FINDINGS: We examined whether such spatial information is available to anterior inferotemporal (AIT) neurons in the macaque monkey by measuring the effect of eye position on responses to a set of simple 2D shapes. We report, for the first time, a significant eye position effect in over 40% of recorded neurons with small gaze angle shifts from central fixation. Although eye position modulates responses, it does not change shape selectivity. CONCLUSIONS/SIGNIFICANCE: These data demonstrate that spatial information is available in AIT for the representation of objects and scenes within a non-retinocentric frame of reference. More generally, the availability of spatial information in AIT calls into questions the classic dichotomy in visual processing that associates object shape processing with ventral structures such as AIT but places spatial processing in a separate anatomical stream projecting to dorsal structures.