Eye tracking and gaze based interaction within immersive virtual environments

Our eyes are input sensors which Provide our brains with streams of visual data. They have evolved to be extremely efficient, and they will constantly dart to-and-fro to rapidly build up a picture of the salient entities in a viewed scene. These actions are almost subconscious. However, they can provide telling signs of how the brain is decoding the visuals and call indicate emotional responses, prior to the viewer becoming aware of them. In this paper we discuss a method of tracking a user's eye movements, and Use these to calculate their gaze within an immersive virtual environment. We investigate how these gaze patterns can be captured and used to identify viewed virtual objects, and discuss how this can be used as a, natural method of interacting with the Virtual Environment. We describe a flexible tool that has been developed to achieve this, and detail initial validating applications that prove the concept.

An assessment of eye-gaze potential within immersive virtual environments

In collaborative situations, eye gaze is a critical element of behavior which supports and fulfills many activities and roles. In current computer-supported collaboration systems, eye gaze is poorly supported. Even in a state-of-the-art video conferencing system such as the access grid, although one can see the face of the user, much of the communicative power of eye gaze is lost. This article gives an overview of some preliminary work that looks towards integrating eye gaze into an immersive collaborative virtual environment and assessing the impact that this would have on interaction between the users of such a system. Three experiments were conducted to assess the efficacy of eye gaze within immersive virtual environments. In each experiment, subjects observed on a large screen the eye-gaze behavior of an avatar. The eye-gaze behavior of that avatar had previously been recorded from a user with the use of a head-mounted eye tracker. The first experiment was conducted to assess the difference between users' abilities to judge what objects an avatar is looking at with only head gaze being viewed and also with eye- and head-gaze data being displayed. The results from the experiment show that eye gaze is of vital importance to the subjects, correctly identifying what a person is looking at in an immersive virtual environment. The second experiment examined whether a monocular or binocular eye-tracker would be required. This was examined by testing subjects' ability to identify where an avatar was looking from their eye direction alone, or by eye direction combined with convergence. This experiment showed that convergence had a significant impact on the subjects' ability to identify where the avatar was looking. The final experiment looked at the effects of stereo and mono-viewing of the scene, with the subjects being asked to identify where the avatar was looking. This experiment showed that there was no difference in the subjects' ability to detect where the avatar was gazing. This is followed by a description of how the eye-tracking system has been integrated into an immersive collaborative virtual environment and some preliminary results from the use of such a system.

Eye gaze in virtual environments: evaluating the need and initial work on implementation

For efficient collaboration between participants, eye gaze is seen as being critical for interaction. Video conferencing either does not attempt to support eye gaze (e.g. AcessGrid) or only approximates it in round table conditions (e.g. life size telepresence). Immersive collaborative virtual environments represent remote participants through avatars that follow their tracked movements. By additionally tracking people's eyes and representing their movement on their avatars, the line of gaze can be faithfully reproduced, as opposed to approximated. This paper presents the results of initial work that tested if the focus of gaze could be more accurately gauged if tracked eye movement was added to that of the head of an avatar observed in an immersive VE. An experiment was conducted to assess the difference between user's abilities to judge what objects an avatar is looking at with only head movements being displayed, while the eyes remained static, and with eye gaze and head movement information being displayed. The results from the experiment show that eye gaze is of vital importance to the subjects correctly identifying what a person is looking at in an immersive virtual environment. This is followed by a description of the work that is now being undertaken following the positive results from the experiment. We discuss the integration of an eye tracker more suitable for immersive mobile use and the software and techniques that were developed to integrate the user's real-world eye movements into calibrated eye gaze in an immersive virtual world. This is to be used in the creation of an immersive collaborative virtual environment supporting eye gaze and its ongoing experiments. Copyright (C) 2009 John Wiley & Sons, Ltd.

Soya phytoestrogens change cortical and hippocampal expression of BDNF mRNA in male rats

Adult male hooded Lister rats were either fed a diet containing 150 microg/g soya phytoestrogens or a soya-free diet for 18 days. This concentration of phytoestrogens should have been sufficient to occupy the oestrogen-beta, but not the oestrogen-alpha, receptors. Using in situ hybridisation, significant reductions were found in brain-derived neurotrophic factor (BDNF) mRNA expression in the CA3 and CA4 region of the hippocampus and in the cerebral cortex in the rats fed the diet containing phytoestrogens, compared with those on the soya-free diet. No changes in glutamic acid decarboxylase-67 or glial fibrillary acidic protein mRNA were found. This suggests a role for oestrogen-beta receptors in regulating BDNF mRNA expression.

A helmet mounted display system with active gaze control for visual telepresence

The objective of a Visual Telepresence System is to provide the operator with a high fidelity image from a remote stereo camera pair linked to a pan/tilt device such that the operator may reorient the camera position by use of head movement. Systems such as these which utilise virtual reality style helmet mounted displays have a number of limitations. The geometry of the camera positions and of the displays is generally fixed and is most suitable only for viewing elements of a scene at a particular distance. To address such limitations, a prototype system has been developed where the geometry of the displays and cameras is dynamically controlled by the eye movement of the operator. This paper explores why it is necessary to actively adjust the display system as well as the cameras and justifies the use of mechanical adjustment of the displays as an alternative to adjustment by electronic or image processing methods. The electronic and mechanical design is described including optical arrangements and control algorithms. The performance and accuracy of the system is assessed with respect to eye movement.

Variation in the human Cannabinoid Receptor (CNR1) gene modulates gaze duration for happy faces

BACKGROUND: Humans from an early age look longer at preferred stimuli, and also typically look longer at facial expressions of emotion, particularly happy faces. Atypical gaze patterns towards social stimuli are common in Autism Spectrum Conditions (ASC). However, it is unknown if gaze fixation patterns have any genetic basis. In this study, we tested if variations in the cannabinoid receptor 1 (CNR1) gene are associated with gaze duration towards happy faces. This gene was selected because CNR1 is a key component of the endocannabinoid system, involved in processing reward, and in our previous fMRI study we found variations in CNR1 modulates the striatal response to happy (but not disgust) faces. The striatum is involved in guiding gaze to rewarding aspects of a visual scene. We aimed to validate and extend this result in another sample using a different technique (gaze tracking). METHODS: 30 volunteers (13 males, 17 females) from the general population observed dynamic emotion expressions on a screen while their eye movements were recorded. They were genotyped for the identical four SNPs in the CNR1 gene tested in our earlier fMRI study. RESULTS: Two SNPs (rs806377 and rs806380) were associated with differential gaze duration for happy (but not disgust) faces. Importantly, the allelic groups associated with greater striatal response to happy faces in the fMRI study were associated with longer gaze duration for happy faces. CONCLUSIONS: These results suggest CNR1 variations modulate striatal function that underlies the perception of signals of social reward such as happy faces. This suggests CNR1 is a key element in the molecular architecture of perception of certain basic emotions. This may have implications for understanding neurodevelopmental conditions marked by atypical eye contact and facial emotion processing, such as ASC.

A behavioral comparison of male and female adults with high functioning autism spectrum conditions

Autism spectrum conditions (ASC) affect more males than females in the general population. However, within ASC it is unclear if there are phenotypic sex differences. Testing for similarities and differences between the sexes is important not only for clinical assessment but also has implications for theories of typical sex differences and of autism. Using cognitive and behavioral measures, we investigated similarities and differences between the sexes in age- and IQ-matched adults with ASC (high-functioning autism or Asperger syndrome). Of the 83 (45 males and 38 females) participants, 62 (33 males and 29 females) met Autism Diagnostic Interview-Revised (ADI-R) cut-off criteria for autism in childhood and were included in all subsequent analyses. The severity of childhood core autism symptoms did not differ between the sexes. Males and females also did not differ in self-reported empathy, systemizing, anxiety, depression, and obsessive-compulsive traits/symptoms or mentalizing performance. However, adult females with ASC showed more lifetime sensory symptoms (p = 0.036), fewer current socio-communication difficulties (p = 0.001), and more self-reported autistic traits (p = 0.012) than males. In addition, females with ASC who also had developmental language delay had lower current performance IQ than those without developmental language delay (p<0.001), a pattern not seen in males. The absence of typical sex differences in empathizing-systemizing profiles within the autism spectrum confirms a prediction from the extreme male brain theory. Behavioral sex differences within ASC may also reflect different developmental mechanisms between males and females with ASC. We discuss the importance of the superficially better socio-communication ability in adult females with ASC in terms of why females with ASC may more often go under-recognized, and receive their diagnosis later, than males.

Blood pressure lowering effects of beetroot juice and novel beetroot enriched bread products in normotensive male subjects

A number of vegetables have a high nitrate content which after ingestion can be reduced to 36 nitrite by oral bacteria, and further to vasoprotective nitric oxide endogenously. Two separate 37 randomly controlled, single blind, cross-over, postprandial studies were performed in 38 normotensive volunteers. Ambulatory blood pressure was measured over a 24 h period 39 following consumption of either four doses of beetroot juice (BJ) 0 g, 100 g, 250 g and 500 g 40 (n = 18) or three bread products, control bread (0 g beetroot), red beetroot and white beetroot 41 enriched breads (n =14). Total urinary nitrate/nitrite (NOx) was measured at baseline, 2, 4 42 and 24 h post ingestion. BJ consumption significantly, and in a near dose dependent manner, 43 lowered systolic (P <0.01) and diastolic BP (P <0.001) over a period of 24 h, compared to 44 water control. Furthermore, bread products enriched with 100 g red or white beetroot lowered 45 systolic and diastolic BP over a period of 24 h (red beetroot enriched bread, P <0.05), with no 46 statistical differences between varieties. Total urinary NOx significantly increased following 47 consumption of 100 g (P<0.01), 250 g (P <0.001) and 500 g BJ (P <0.001) and after red 48 beetroot bread (P <0.05), but did not reach significance for white beetroot bread compared to 49 the no beetroot condition. These studies demonstrated significant hypotensive effects of a low 50 dose (100 g) of beetroot which was unaffected by processing, or the presence of betacyanins. 51 This data strengthens the evidence for cardioprotective BP lowering effects of dietary nitrate-52 rich vegetables.

Dynamic vergence telepresence system incorporating active gaze control

Visual Telepresence system which utilize virtual reality style helmet mounted displays have a number of limitations. The geometry of the camera positions and of the display is fixed and is most suitable only for viewing elements of a scene at a particular distance. In such a system, the operator's ability to gaze around without use of head movement is severely limited. A trade off must be made between a poor viewing resolution or a narrow width of viewing field. To address these limitations a prototype system where the geometry of the displays and cameras is dynamically controlled by the eye movement of the operator has been developed. This paper explores the reasons why is necessary to actively adjust both the display system and the cameras and furthermore justifies the use of mechanical adjustment of the displays as an alternative to adjustment by electronic or image processing methods. The electronic and mechanical design is described including optical arrangements and control algorithms, An assessment of the performance of the system against a fixed camera/display system when operators are assigned basic tasks involving depth and distance/size perception. The sensitivity to variations in transient performance of the display and camera vergence is also assessed.

Coping with delays for real-time gaze control (The Fall and Rise of the Smiths Regulator)

In this paper we describe how to cope with the delays inherent in a real time control system for a steerable stereo head/eye platform. A purposive and reactive system requires the use of fast vision algorithms to provide the controller with the error signals to drive the platform. The time-critical implementation of these algorithms is necessary, not only to enable short latency reaction to real world events, but also to provide sufficiently high frequency results with small enough delays that controller remain stable. However, even with precise knowledge of that delay, nonlinearities in the plant make modelling of that plant impossible, thus precluding the use of a Smith Regulator. Moreover, the major delay in the system is in the feedback (image capture and vision processing) rather than feed forward (controller) loop. Delays ranging between 40msecs and 80msecs are common for the simple 2D processes, but might extend to several hundred milliseconds for more sophisticated 3D processes. The strategy presented gives precise control over the gaze direction of the cameras despite the lack of a priori knowledge of the delays involved. The resulting controller is shown to have a similar structure to the Smith Regulator, but with essential modifications.

From saccades to smooth pursuit: real-time gaze control using motion feedback

The authors present an active vision system which performs a surveillance task in everyday dynamic scenes. The system is based around simple, rapid motion processors and a control strategy which uses both position and velocity information. The surveillance task is defined in terms of two separate behavioral subsystems, saccade and smooth pursuit, which are demonstrated individually on the system. It is shown how these and other elementary responses to 2D motion can be built up into behavior sequences, and how judicious close cooperation between vision and control results in smooth transitions between the behaviors. These ideas are demonstrated by an implementation of a saccade to smooth pursuit surveillance system on a high-performance robotic hand/eye platform.