Vision, ambiguity and perceptual possibility : why pictures matter in the critique of stereo-immersive VR

The acquisition of three-dimensional immersive space through advanced digitial imaging technology, suggests a profound shift from the relatively impoverished representational stratgies of two-dimensional pictorial imagery.  This entails an epistemological shift from pictorial representation, to the presentation of actual three-dimensional space through stereoscopic (3D) imagery.  Moreover, it suggests that visuality rather than 'virtuality' is the core issue in understanding the nature of the epistemological shift associated with stereo-immersive VR.

A shift in visual epistemology from 'flat' pictorial representation to three-dimensional stereo-immersion suggests a gainful move toward a visuality imbued with spatial possibilities.  In quantitative terms, these visual-spatial gains may seem self-evident.  However, certain aspects peculiar to pictorial representation are missing from stereo-immersive imaging.  That is lost in stereo-immersive imaging, and how it can be measured?

This thesis proposes that the inherent ambiguity of two-dimensional representations of three-dimensional spatial structure in pictures, invokes a perceptual response in which pictorial spaces elicit 'perceptual possibility'.  The robust three-dimentional spatiality of stereo-immersvie VR foreclosures such possibility.  Through examining stereo-immersive VR in terms of its visuality, the thesis develops a new appraoch to understanding VR that solves some of the issues associated with the problematic concept of 'virtuality'.  In addition, the thesis finds that by placing stereo-immersive VR and pictures within the shared paradigm of 'the visual', an important dimension of pictures that has been overlooked in past analyses re-emerges : the thesis proposes the concept of 'artifactuality' to account for the way pictures are fundamentally, and in the first instance, aesthetic objects for visual perception.  It is in their manner of appearing as pictures, that they are perceived as pictures of something.  It is from this fundamental basis that their many levels of meaning and signification - their manifold 'realisms' - can arise.

This thesis therefore addresses two intersecting problems within the paradigm of the visual: it proposes 1) that analyses of 'the virtual' be grounded in the 'artifactuality' of pictorial perception, and 2) that the spatiality of stereo-immersive VR be reinvigorated by purposefully 'under-contraining' its key percept - the robust, 'solid' stereoscopic structuring of visual space.  This approach opens up the discourse of stereo-immersive VR to new visual paradigms.  The thesis proposes that these be modelled not on the impossibility of 'the virtual', but on the possibilities of visual ambiguity drawn from the analysis of pictorial perception.

Instrumental vision

This chapter interrogates stereo-immersive ‘virtual reality’ (VR), the technology that enables a perceiver to experience what it is like to be immersed in a simulated environment. While the simulation is powered by the “geometry engine” (Cutting, 1997: 31) associated with high-end computer imaging technology, the visual experience itself is powered by ordinary human vision: the vision system’s innate capacity to see “in 3D”. To understand and critically appraise stereo-immersive VR, we should study not its purported ‘virtuality’, but its specific visuality, because the ‘reality’ of a so-called ‘virtual environment’ is afforded by the stereoacuity of binocular vision itself. By way of such a critique of the visuality of stereo-immersive VR, this chapter suggests that we think about the ‘practice’ of vision, and consider on what basis vision can have its own ‘materiality’. Pictorial perception is proposed as an exemplary visual mode in which the possibilities of perception might emerge. Against the ‘possibilities’ of vision associated with pictures, the visuality of stereo-immersive VR emerges as a harnessing, or ‘instrumentalisation’ of vision’s innate capabilities. James J. Gibson’s ‘ecological’ approach to vision studies is referenced to show the degree to which developers of VR have sought — and succeeded — to mimic the ‘realness’ of ordinary perceptual reality. This raises a question concerning whether the success of stereo-immersive VR is simultaneously the source of its own perceptual redundancy: for to bring into being the perceptual basis of ordinary ‘real’ reality, is to return the perceiver to what is already familiar and known.

VR Based Visualization and Exploration of Plant Biological Data

This paper investigates the use of virtual reality (VR) technologies to facilitate the analysis of plant biological data in distinctive steps in the application pipeline. Reconstructed three-dimensional biological models (primary polygonal models) transferred to a virtual environment support scientists' collaborative exploration of biological datasets so that they obtain accurate analysis results and uncover information hidden in the data. Examples of the use of virtual reality in practice are provided and a complementary user study was performed.

Augmenting a Laser Pointer with a Diffraction Grating for Monoscopic 6DOF Detection

This article illustrates the detection of 6 degrees of freedom (DOF) for Virtual Environment interactions using a modified simple laser pointer device and a camera. The laser pointer is combined with a diffraction rating to project a unique laser grid onto the projection planes used in projection-based immersive VR setups. The distortion of the projected grid is used to calculate the translational and rotational degrees of freedom required for human-computer interaction purposes.

XSAMPL3D: An Action Description Language for the Animation of Virtual Characters

In this paper we present XSAMPL3D, a novel language for the high-level representation of actions performed on objects by (virtual) humans. XSAMPL3D was designed to serve as action representation language in an imitation-based approach to character animation: First, a human demonstrates a sequence of object manipulations in an immersive Virtual Reality (VR) environment. From this demonstration, an XSAMPL3D description is automatically derived that represents the actions in terms of high-level action types and involved objects. The XSAMPL3D action description can then be used for the synthesis of animations where virtual humans of different body sizes and proportions reproduce the demonstrated action. Actions are encoded in a compact and human-readable XML-format. Thus, XSAMPL3D describtions are also amenable to manual authoring, e.g. for rapid prototyping of animations when no immersive VR environment is at the animator's disposal. However, when XSAMPL3D descriptions are derived from VR interactions, they can accomodate many details of the demonstrated action, such as motion trajectiories,hand shapes and other hand-object relations during grasping. Such detail would be hard to specify with manual motion authoring techniques only. Through the inclusion of language features that allow the representation of all relevant aspects of demonstrated object manipulations, XSAMPL3D is a suitable action representation language for the imitation-based approach to character animation.

The development and evaluation of a medical imaging training immersive environment

Introduction A novel realistic 3D virtual reality (VR) application has been developed to allow medical imaging students at Queensland University of Technology to practice radiographic techniques independently outside the usual radiography laboratory. Methods A flexible agile development methodology was used to create the software rapidly and effectively. A 3D gaming environment and realistic models were used to engender presence in the software while tutor-determined gold standards enabled students to compare their performance and learn in a problem-based learning pedagogy. Results Students reported high levels of satisfaction and perceived value and the software enabled up to 40 concurrent users to prepare for clinical practice. Student feedback also indicated that they found 3D to be of limited value in the desktop version compared to the usual 2D approach. A randomised comparison between groups receiving software-based and traditional practice measured performance in a formative role play with real equipment. The results of this work indicated superior performance with the equipment for the VR trained students (P = 0.0366) and confirmed the value of VR for enhancing 3D equipment-based problem-solving skills. Conclusions Students practising projection techniques virtually performed better at role play assessments than students practising in a traditional radiography laboratory only. The application particularly helped with 3D equipment configuration, suggesting that teaching 3D problem solving is an ideal use of such medical equipment simulators. Ongoing development work aims to establish the role of VR software in preparing students for clinical practice with a range of medical imaging equipment.

The Development of an Immersive Virtual Reality Environment for Manufacturing and Improving Organizational Learning

This paper examines the applicability of an immersive virtual reality (VR) system to the process of organizational learning in a manufacturing context. The work focuses on the extent to which realism has to be represented in a simulated product build scenario in order to give the user an effective learning experience for an assembly task. Current technologies allow the visualization and manipulation of objects in VR systems but physical behaviors such as contact between objects and the effects of gravity are not commonly represented in off the shelf simulation solutions and the computational power required to facilitate these functions remains a challenge. This work demonstrates how physical behaviors can be coded and represented through the development of more effective mechanisms for the computer aided design (CAD) and VR interface.

Manufacturing System Simulation & Human Robot Interaction in VR:Manufacturing system simulation: Methodologies, software tools, case study in transportation design

Virtual Reality techniques are relatively new, having experienced significant development only during the last few years, in accordance with the progress achieved by computer science and hardware and software technologies. The study of such advanced design systems has led to the realization of an immersive environment in which new procedures for the evaluation of product prototypes, ergonomics and manufacturing operations have been simulated. The application of the environment realized to robotics, ergonomics, plant simulations and maintainability verifications has allowed us to highlight the advantages offered by a design methodology: the possibility of working on the industrial product in the first phase of conception; of placing the designer in front of the virtual reproduction of the product in a realistic way; and of interacting with the same concept. The aim of this book is to present an updated vision of VM through different aspects. We will describe the trends and results achieved in the automotive, aerospace and railway fields, in terms of the Digital Product Creation Process to design the product and the manufacturing process.

Development of a novel immersive interactive virtual reality cricket simulator for cricket batting

Research in the field of sports performance is constantly developing new technology to help extract meaningful data to aid in understanding in a multitude of areas such as improving technical or motor performance. Video playback has previously been extensively used for exploring anticipatory behaviour. However, when using such systems, perception is not active. This loses key information that only emerges from the dynamics of the action unfolding over time and the active perception of the observer. Virtual reality (VR) may be used to overcome such issues. This paper presents the architecture and initial implementation of a novel VR cricket simulator, utilising state of the art motion capture technology (21 Vicon cameras capturing kinematic profile of elite bowlers) and emerging VR technology (Intersense IS-900 tracking combined with Qualisys Motion capture cameras with visual display via Sony Head Mounted Display HMZ-T1), applied in a cricket scenario to examine varying components of decision and action for cricket batters. This provided an experience with a high level of presence allowing for a real-time egocentric view-point to be presented to participants. Cyclical user-testing was carried out, utilisng both qualitative and quantitative approaches, with users reporting a positive experience in use of the system.

Stereo and motion parallax cues in human 3D vision: can they vanish without a trace?

In an immersive virtual reality environment, subjects fail to notice when a scene expands or contracts around them, despite correct and consistent information from binocular stereopsis and motion parallax, resulting in gross failures of size constancy (A. Glennerster, L. Tcheang, S. J. Gilson, A. W. Fitzgibbon, & A. J. Parker, 2006). We determined whether the integration of stereopsis/motion parallax cues with texture-based cues could be modified through feedback. Subjects compared the size of two objects, each visible when the room was of a different size. As the subject walked, the room expanded or contracted, although subjects failed to notice any change. Subjects were given feedback about the accuracy of their size judgments, where the “correct” size setting was defined either by texture-based cues or (in a separate experiment) by stereo/motion parallax cues. Because of feedback, observers were able to adjust responses such that fewer errors were made. For texture-based feedback, the pattern of responses was consistent with observers weighting texture cues more heavily. However, for stereo/motion parallax feedback, performance in many conditions became worse such that, paradoxically, biases moved away from the point reinforced by the feedback. This can be explained by assuming that subjects remap the relationship between stereo/motion parallax cues and perceived size or that they develop strategies to change their criterion for a size match on different trials. In either case, subjects appear not to have direct access to stereo/motion parallax cues.

Humans ignore motion and stereo cues in favor of a fictional stable world

As we move through the world, our eyes acquire a sequence of images. The information from this sequence is sufficient to determine the structure of a three-dimensional scene, up to a scale factor determined by the distance that the eyes have moved [1, 2]. Previous evidence shows that the human visual system accounts for the distance the observer has walked [3,4] and the separation of the eyes [5-8] when judging the scale, shape, and distance of objects. However, in an immersive virtual-reality environment, observers failed to notice when a scene expanded or contracted, despite having consistent information about scale from both distance walked and binocular vision. This failure led to large errors in judging the size of objects. The pattern of errors cannot be explained by assuming a visual reconstruction of the scene with an incorrect estimate of interocular separation or distance walked. Instead, it is consistent with a Bayesian model of cue integration in which the efficacy of motion and disparity cues is greater at near viewing distances. Our results imply that observers are more willing to adjust their estimate of interocular separation or distance walked than to accept that the scene has changed in size.

Students' assessment of immersive virtual reality as a pedagogic medium

Over the past few years a number of research studies, mainly involving desktop-based or head-mounted Virtual Reality (VR) systems, have been undertaken to determine what VR can contribute to the education process. In our study we have used the findings from a number of these studies to help in formulating a new study into the perceived merits and limitations of using VR in general, and immersive CAVE-like systems in particular, as an education tool. We conducted our study with a group of final year undergraduate students who were registered on a module that described VR in terms of the scientific issues, application areas, and strengths and weaknesses of the technology.

Immersive open surgery simulation

Realistic medical simulation has great potential for augmenting or complimenting traditional medical training or surgery planning, and Virtual Reality (VR) is a key enabling technology for delivering this goal. Although, medical simulators are now widely used in medical institutions, the majority of them are still reliant on desktop monitor displays, and many are restricted in their modelling capability to minimally invasive or endoscopic surgery scenarios. Whilst useful, such models lack the realism and interaction of the operating theatre. In this paper, we describe how we are advancing the technology by simulating open surgery procedures in an Immersive Projection Display CAVE environment thereby enabling medical practitioners to interact with their virtual patients in a more realistic manner.