Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments

Participants' eye-gaze is generally not captured or represented in immersive collaborative virtual environment (ICVE) systems. We present EyeCVE. which uses mobile eye-trackers to drive the gaze of each participant's virtual avatar, thus supporting remote mutual eye-contact and awareness of others' gaze in a perceptually unfragmented shared virtual workspace. We detail trials in which participants took part in three-way conferences between remote CAVE (TM) systems linked via EyeCVE. Eye-tracking data was recorded and used to evaluate interaction, confirming; the system's support for the use of gaze as a communicational and management resource in multiparty conversational scenarios. We point toward subsequent investigation of eye-tracking in ICVEs for enhanced remote social-interaction and analysis.

Immersive open surgery simulation

Realistic medical simulation has great potential for augmenting or complimenting traditional medical training or surgery planning, and Virtual Reality (VR) is a key enabling technology for delivering this goal. Although, medical simulators are now widely used in medical institutions, the majority of them are still reliant on desktop monitor displays, and many are restricted in their modelling capability to minimally invasive or endoscopic surgery scenarios. Whilst useful, such models lack the realism and interaction of the operating theatre. In this paper, we describe how we are advancing the technology by simulating open surgery procedures in an Immersive Projection Display CAVE environment thereby enabling medical practitioners to interact with their virtual patients in a more realistic manner.

Eye tracking and gaze based interaction within immersive virtual environments

Our eyes are input sensors which Provide our brains with streams of visual data. They have evolved to be extremely efficient, and they will constantly dart to-and-fro to rapidly build up a picture of the salient entities in a viewed scene. These actions are almost subconscious. However, they can provide telling signs of how the brain is decoding the visuals and call indicate emotional responses, prior to the viewer becoming aware of them. In this paper we discuss a method of tracking a user's eye movements, and Use these to calculate their gaze within an immersive virtual environment. We investigate how these gaze patterns can be captured and used to identify viewed virtual objects, and discuss how this can be used as a, natural method of interacting with the Virtual Environment. We describe a flexible tool that has been developed to achieve this, and detail initial validating applications that prove the concept.

Immersive co-operative psychological virtual environments (ICPVE)

Virtual Reality (VR) has been used in a variety of forms to assist in the treatment of a wide range of psychological illness. VR can also fulfil the need that psychologists have for safe environments in which to conduct experiments. Currently the main barrier against using this technology is the complexity in developing applications. This paper presents two different co-operative psychological applications which have been developed using a single framework. These applications require different levels of co-operation between the users and clients, ranging from full psychologist involvement to their minimal intervention. This paper will also discuss our approach to developing these different environments and our experiences to date in utilising these environments.

An assessment of eye-gaze potential within immersive virtual environments

In collaborative situations, eye gaze is a critical element of behavior which supports and fulfills many activities and roles. In current computer-supported collaboration systems, eye gaze is poorly supported. Even in a state-of-the-art video conferencing system such as the access grid, although one can see the face of the user, much of the communicative power of eye gaze is lost. This article gives an overview of some preliminary work that looks towards integrating eye gaze into an immersive collaborative virtual environment and assessing the impact that this would have on interaction between the users of such a system. Three experiments were conducted to assess the efficacy of eye gaze within immersive virtual environments. In each experiment, subjects observed on a large screen the eye-gaze behavior of an avatar. The eye-gaze behavior of that avatar had previously been recorded from a user with the use of a head-mounted eye tracker. The first experiment was conducted to assess the difference between users' abilities to judge what objects an avatar is looking at with only head gaze being viewed and also with eye- and head-gaze data being displayed. The results from the experiment show that eye gaze is of vital importance to the subjects, correctly identifying what a person is looking at in an immersive virtual environment. The second experiment examined whether a monocular or binocular eye-tracker would be required. This was examined by testing subjects' ability to identify where an avatar was looking from their eye direction alone, or by eye direction combined with convergence. This experiment showed that convergence had a significant impact on the subjects' ability to identify where the avatar was looking. The final experiment looked at the effects of stereo and mono-viewing of the scene, with the subjects being asked to identify where the avatar was looking. This experiment showed that there was no difference in the subjects' ability to detect where the avatar was gazing. This is followed by a description of how the eye-tracking system has been integrated into an immersive collaborative virtual environment and some preliminary results from the use of such a system.

Low-cost optical tracking for collaborative applications in immersive virtual environments

We present a novel way of interacting with an immersive virtual environment which involves inexpensive motion-capture using the Wii Remote®. A software framework is also presented to visualize and share this information across two remote CAVETM-like environments. The resulting application can be used to assist rehabilitation by sending motion information across remote sites. The application’s software and hardware components are scalable enough to be used on a desktop computer when home-based rehabilitation is preferred.

Designing in caves: using immersive visualisations in design practice

This paper describes a study of the use of immersive Virtual reality technologies in the design of a new hospital. It uses Schön’s concept of reflective practice and video-based methods to analyse the ways design teams approach and employ a full scale 3D immersive environment – a CAVE – in collaborative design work. The analysis describes four themes relating to reflective practice occurring in the setting: orienting to the CAVE technology itself, orienting to the representation of the specific design within the CAVE, activities accounting for, or exploring alternatives within the design for the use and users of the space, and more strategic interactions around how to best represent the design and model to the client within the CAVE setting. The analysis also reveals some unique aspects of design work in this environment. Perhaps most significantly, rather than enhancing or adding to an existing understanding of design through paper based or non-immersive digital representations, it is often acting to challenge or surprise the participants as they experience the immersive, full scale version of their own design.

`Future is old': immersive learning with generation Y engineering students

This paper explores the application of four elements deemed to be essential to immersive learning; immersion, engagement, risk/creativity and agency. The authors discuss the implementation of these four elements within two very different classroom environments, one secondary and one tertiary, to illustrate the importance of students' active participation in the planning, design and development processes of their own learning environments. By transforming both the conceptual and operational environments of both cohorts the authors illustrate the benefits of the development of immersive learning environments. Importantly, such environments must reflect the choices and preferences of the learner as they negotiate their way through the learning journey that best suits their needs. Rather than the imposition of a learning regime considered to be irrelevant, but more importantly 'boring' to the student cohort, the authors argue that immersive learning environments are successful because they are reliant on the students' enthusiastic creation of their own knowledge in collaboration with adults and student colleagues. This study can be applied in all spectrums of education (primary, secondary and tertiary), and in this case, specifically engineering education.

Haptically enabled interactivity and immersive virtual assembly

Virtual training systems are attracting paramount attention from the manufacturing industries due to their potential advantages over the conventional training practices such as general assembly. Within this virtual training realm for general assembly, a haptically enabled interactive and immersive virtual reality (HIVEx) system is presented. The idea is to imitate real assembly training scenarios by providing comprehensive user interaction as well as by enforcing physical constraints within the virtual environment through the use of haptics technology. The developed system employs a modular system approach providing flexibility of reconfiguration and scalability as well as better utilization of the current multi-core computer architecture. The user interacts with the system using haptics device and data glove while fully immersed into the virtual environment with depth perception. An evaluation module, incorporated into the system, automatically logs and evaluates the information through the simulation providing user performance and improvements over time. A ruggedized portable version of the system is also developed and presented with full system capabilities allowing easy relocation with different factory environments. A number of training scenarios has been developed with varying degree of complexity to exploit the potential of the presented system. The presented system can be employed for teaching and training of existing assembly processes as well as the design of new optimised assembly operations. Furthermore, the presented system can assist in optimizing existing practices by evaluating the effectiveness and the level of knowledge transfer involved in the process. Within the aforementioned conceptual. framework, a working prototype is developed.

Through the looking glass : immersive interfaces for participant engagement in blended e-learning environments

This chapter outlines current research profiling students and educators participating in and constructing immersive interfaces in blended e-learning settings. Multi User Virtual Environments (MUVEs) and real world settings augmented with virtual information can generate problem-solving communities where participants gain greater technical knowledge and skills through meaningful and frequent interaction. MUVEs can also generate technical innovation amongst students from diverse disciplinary backgrounds provided students are encouraged to help each other and learn together. After detailing some false assumptions about computer literacy that can stifle meaningful exploration with new technologies in contemporary education, this chapter documents an exemplar involving extensive collaboration between students from different educational backgrounds with diverse technical competencies. The success of this initiative hinges on the willingness of educators to provide a shared learning experience where technology is used to facilitate increased student communication and offers a site for invention, informed critique, industry participation, and a sense of community.

Vision, ambiguity and perceptual possibility : why pictures matter in the critique of stereo-immersive VR

The acquisition of three-dimensional immersive space through advanced digitial imaging technology, suggests a profound shift from the relatively impoverished representational stratgies of two-dimensional pictorial imagery.  This entails an epistemological shift from pictorial representation, to the presentation of actual three-dimensional space through stereoscopic (3D) imagery.  Moreover, it suggests that visuality rather than 'virtuality' is the core issue in understanding the nature of the epistemological shift associated with stereo-immersive VR.

A shift in visual epistemology from 'flat' pictorial representation to three-dimensional stereo-immersion suggests a gainful move toward a visuality imbued with spatial possibilities.  In quantitative terms, these visual-spatial gains may seem self-evident.  However, certain aspects peculiar to pictorial representation are missing from stereo-immersive imaging.  That is lost in stereo-immersive imaging, and how it can be measured?

This thesis proposes that the inherent ambiguity of two-dimensional representations of three-dimensional spatial structure in pictures, invokes a perceptual response in which pictorial spaces elicit 'perceptual possibility'.  The robust three-dimentional spatiality of stereo-immersvie VR foreclosures such possibility.  Through examining stereo-immersive VR in terms of its visuality, the thesis develops a new appraoch to understanding VR that solves some of the issues associated with the problematic concept of 'virtuality'.  In addition, the thesis finds that by placing stereo-immersive VR and pictures within the shared paradigm of 'the visual', an important dimension of pictures that has been overlooked in past analyses re-emerges : the thesis proposes the concept of 'artifactuality' to account for the way pictures are fundamentally, and in the first instance, aesthetic objects for visual perception.  It is in their manner of appearing as pictures, that they are perceived as pictures of something.  It is from this fundamental basis that their many levels of meaning and signification - their manifold 'realisms' - can arise.

This thesis therefore addresses two intersecting problems within the paradigm of the visual: it proposes 1) that analyses of 'the virtual' be grounded in the 'artifactuality' of pictorial perception, and 2) that the spatiality of stereo-immersive VR be reinvigorated by purposefully 'under-contraining' its key percept - the robust, 'solid' stereoscopic structuring of visual space.  This approach opens up the discourse of stereo-immersive VR to new visual paradigms.  The thesis proposes that these be modelled not on the impossibility of 'the virtual', but on the possibilities of visual ambiguity drawn from the analysis of pictorial perception.