Reinforcement learning utilizes proxemics: an avatar learns to manipulate the position of people in immersive virtual reality

A reinforcement learning (RL) method was used to train a virtual character to move participants to a specified location. The virtual environment depicted an alleyway displayed through a wide field-of-view head-tracked stereo head-mounted display. Based on proxemics theory, we predicted that when the character approached within a personal or intimate distance to the participants, they would be inclined to move backwards out of the way. We carried out a between-groups experiment with 30 female participants, with 10 assigned arbitrarily to each of the following three groups: In the Intimate condition the character could approach within 0.38m and in the Social condition no nearer than 1.2m. In the Random condition the actions of the virtual character were chosen randomly from among the same set as in the RL method, and the virtual character could approach within 0.38m. The experiment continued in each case until the participant either reached the target or 7 minutes had elapsed. The distributions of the times taken to reach the target showed significant differences between the three groups, with 9 out of 10 in the Intimate condition reaching the target significantly faster than the 6 out of 10 who reached the target in the Social condition. Only 1 out of 10 in the Random condition reached the target. The experiment is an example of applied presence theory: we rely on the many findings that people tend to respond realistically in immersive virtual environments, and use this to get people to achieve a task of which they had been unaware. This method opens up the door for many such applications where the virtual environment adapts to the responses of the human participants with the aim of achieving particular goals.

Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments

In this paper address we the question as to why participants tend to respond realistically to situations and events portrayed within an Immersive Virtual Reality (IVR) system. The idea is put forward, based on experience of a large number of experimental studies, that there are two orthogonal components that contribute to this realistic response. The first is"being there", often called"presence", the qualia of having a sensation of being in a real place. We call this Place Illusion (PI). Second, Plausibility Illusion (Psi) refers to the illusion that the scenario being depicted is actually occurring. In the case of both PI and Psi the participant knows for sure that that they are not"there" and that the events are not occurring. PI is constrained by the sensorimotor contingencies afforded by the virtual reality system. Psi is determined by the extent to which the system can produce events that directly relate to the participant, and the overall credibility of the scenario being depicted in comparison with expectations. We argue that when both PI and Psi occur, participants will respond realistically to the virtual reality.

Correlations between vocal input and visual response apparently enhance presence in a virtual environment

This work investigates novel alternative means of interaction in a virtual environment (VE).We analyze whether humans can remap established body functions to learn to interact with digital information in an environment that is cross-sensory by nature and uses vocal utterances in order to influence (abstract) virtual objects. We thus establish a correlation among learning, control of the interface, and the perceived sense of presence in the VE. The application enables intuitive interaction by mapping actions (the prosodic aspects of the human voice) to a certain response (i.e., visualization). A series of single-user and multiuser studies shows that users can gain control of the intuitive interface and learn to adapt to new and previously unseen tasks in VEs. Despite the abstract nature of the presented environment, presence scores were generally very high.

The effect of virtual reality on visual vertigo symptoms in patients with peripheral vestibular dysfunction: a pilot study

Individuals with vestibular dysfunction may experience visual vertigo (VV), in which symptoms are provoked or exacerbated by excessive or disorientating visual stimuli (e.g. supermarkets). VV can significantly improve when customized vestibular rehabilitation exercises are combined with exposure to optokinetic stimuli. Virtual reality (VR), which immerses patients in realistic, visually challenging environments, has also been suggested as an adjunct to VR to improve VV symptoms. This pilot study compared the responses of sixteen patients with unilateral peripheral vestibular disorder randomly allocated to a VR regime incorporating exposure to a static (Group S) or dynamic (Group D) VR environment. Participants practiced vestibular exercises, twice weekly for four weeks, inside a static (Group S) or dynamic (Group D) virtual crowded square environment, presented in an immersive projection theatre (IPT), and received a vestibular exercise program to practice on days not attending clinic. A third Group D1 completed both the static and dynamic VR training. Treatment response was assessed with the Dynamic Gait Index and questionnaires concerning symptom triggers and psychological state. At final assessment, significant betweengroup differences were noted between Groups D (p = 0.001) and D1 (p = 0.03) compared to Group S for VV symptoms with the former two showing a significant 59.2% and 25.8% improvement respectively compared to 1.6% for the latter. Depression scores improved only for Group S (p = 0.01) while a trend towards significance was noted for Group D regarding anxiety scores (p = 0.07). Conclusion: Exposure to dynamic VR environments should be considered as a useful adjunct to vestibular rehabilitation programs for patients with peripheral vestibular disorders and VV symptoms.

Drift and ownership toward a distant virtual body.

In body ownership illusions participants feel that a mannequin or virtual body (VB) is their own. Earlier results suggest that body ownership over a body seen from behind in extra personal space is possible when the surrogate body is visually stroked and tapped on its back, while spatially and temporal synchronous tactile stimulation is applied to the participant's back. This result has been disputed with the claim that the results can be explained by self-recognition rather than somatic body ownership. We carried out an experiment with 30 participants in a between-groups design. They all saw the back of a VB 1.2 m in front, that moved in real-time determined by upper body motion capture. All felt tactile stimulation on their back, and for 15 of them this was spatially and temporally synchronous with stimulation that they saw on the back of the VB, but asynchronous for the other 15. After 3 min a revolving fan above the VB descended and stopped at the position of the VB neck. A questionnaire assessed referral of touch to the VB, body ownership, the illusion of drifting forwards toward the VB, and the VB drifting backwards. Heart rate deceleration (HRD) and the amount of head movement during the threat period were used to assess the response to the threat from the fan. Results showed that although referral of touch was significantly greater in the synchronous condition than the asynchronous, there were no other differences between the conditions. However, a further multivariate analysis revealed that in the visuotactile synchronous condition HRD and head movement increased with the illusion of forward drift and decreased with backwards drift. Body ownership contributed positively to these drift sensations. Our conclusion is that the setup results in a contradiction-somatic feelings associated with a distant body-that the brain attempts to resolve by generating drift illusions that would make the two bodies coincide.

Touching the void: exploring virtual objects through a vibrotactile glove

This paper describes a simple low-cost approach toadding an element of haptic interaction within a virtualenvironment. Using off-the-shelf hardware and software wedescribe a simple setup that can be used to explore physically virtual objects in space. This setup comprises of a prototype glove with a number of vibrating actuators to provide the haptic feedback, a Kinect camera for the tracking of the user's hand and a virtual reality development environment. As proof of concept and to test the efficiency of the system as well as its potential applications, we developed a simple application where we created 4 different shapes within a virtual environment in order to try toexplore them and guess their shape through touch alone.

Proxemics with multiple dynamic characters in an immersive virtual environment

An experiment was carried out to examine the impact on electrodermal activity of people when approached by groups of one or four virtual characters at varying distances. It was premised on the basis of proxemics theory that the closer the approach of the virtual characters to the participant, the greater the level of physiological arousal. Physiological arousal was measured by the number of skin conductance responses within a short time period after the approach, and the maximum change in skin conductance level 5 s after the approach. The virtual characters were each either female or a cylinder of human size, and one or four characters approached each subject a total of 12 times. Twelve male subjects were recruited for the experiment. The results suggest that the number of skin conductance responses after the approach and the change in skin conductance level increased the closer the virtual characters approached toward the participants. Moreover, these response variables were inversely correlated with the number of visits, showing a typical adaptation effect. There was some evidence to suggest that the number of characters who simultaneously approached (one or four) was positively associated with the responses. Surprisingly there was no evidence of a difference in response between the humanoid characters and cylinders on the basis of this physiological data. It is suggested that the similarity in this quantitative arousal response to virtual characters and virtual objects might mask a profound difference in qualitative response, an interpretation supported by questionnaire and interview results. Overall the experiment supported the premise that people exhibit heightened physiological arousal the closer they are approached by virtual characters.

Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion

Previous studies have examined the experience of owning a virtual surrogate body or body part through specific combinations of cross-modal multisensory stimulation. Both visuomotor (VM) and visuotactile (VT) synchronous stimulation have been shown to be important for inducing a body ownership illusion, each tested separately or both in combination. In this study we compared the relative importance of these two cross-modal correlations, when both are provided in the same immersive virtual reality setup and the same experiment. We systematically manipulated VT and VM contingencies in order to assess their relative role and mutual interaction. Moreover, we present a new method for measuring the induced body ownership illusion through time, by recording reports of breaks in the illusion of ownership ("breaks") throughout the experimental phase. The balance of the evidence, from both questionnaires and analysis of the breaks, suggests that while VM synchronous stimulation contributes the greatest to the attainment of the illusion, a disruption of either (through asynchronous stimulation) contributes equally to the probability of a break in the illusion.

Efectividad del trabajo de doble tarea cognitivo-motora con incorporación de realidad virtual para el tratamiento del equilibrio en pacientes con Traumatismo Craneoencefálico.

Pregunta de revisión: ¿El tratamiento de doble tarea con la incorporación del sistema de realidad virtual BioTrack favorece el equilibrio en los pacientes que han sufrido un traumatismo craneoencefálico? Objetivo: Determinar si el trabajo de doble tarea cognitivo-motor con la incorporación de realidad virtual favorece el equilibrio de los pacientes con traumatismo craneoencefálico. Metodología: se realizará un diseño de tipo experimental, aleatorio controlado y prospectivo. Se planteará un proyecto de 36 sesiones dividido en seis meses basado en trabajo de doble tarea cognitivo-motor con realidad virtual. Consta de un tratamiento combinado de ejercicios para el equilibrio con el soporte de realidad virtual BioTrack. Al mismo tiempo se trabajará la memoria y la atención para el trabajo cognitivo. Se compara con el grupo control que solo realiza ejercicios con BioTrack. La medida principal es el equilibrio medido con la berg balance scale y la influencia del trabajo cognitivo.

An Analysis of Eye Scanpath Entropy in a Progressively Forming Virtual Environment

A sign of presence in virtual environments is that people respond to situations and events as if they were real, where response may be considered at many different levels, ranging from unconscious physiological responses through to overt behavior,emotions, and thoughts. In this paper we consider two responses that gave different indications of the onset of presence in a gradually forming environment. Two aspects of the response of people to an immersive virtual environment were recorded: their eye scanpath, and their skin conductance response (SCR). The scenario was formed over a period of 2 min, by introducing an increasing number of its polygons in random order in a head-tracked head-mounted display. For one group of experimental participants (n 8) the environment formed into one in which they found themselves standing on top of a 3 m high column. For a second group of participants (n 6) the environment was otherwise the same except that the column was only 1 cm high, so that they would be standing at normal ground level. For a third group of participants (n 14) the polygons never formed into a meaningful environment. The participants who stood on top of the tall column exhibited a significant decrease in entropy of the eye scanpath and an increase in the number of SCR by 99 s into the scenario, at a time when only 65% of the polygons had been displayed. The ground level participants exhibited a similar decrease in scanpath entropy, but not the increase in SCR. The random scenario grouping did not exhibit this decrease in eye scanpath entropy. A drop in scanpath entropy indicates that the environment had cohered into a meaningful perception. An increase in the rate of SCR indicates the perception of an aversive stimulus. These results suggest that on these two dimensions (scanpath entropy and rate of SCR) participants were responding realistically to the scenario shown in the virtual environment. In addition, the response occurred well before the entire scenario had been displayed, suggesting that once a set of minimal cues exists within a scenario,it is enough to form a meaningful perception. Moreover, at the level of the sympathetic nervous system, the participants who were standing on top of the column exhibited arousal as if their experience might be real. This is an important practical aspect of the concept of presence.

The Impact of a Character Posture Model on the Communication of Affect in an Immersive Virtual Environment

This paper presents the quantitative and qualitative findings from an experiment designed to evaluate a developing model of affective postures for full-body virtual characters in immersive virtual environments (IVEs). Forty-nine participants were each requested to explore a virtual environment by asking two virtual characters for instructions. The participants used a CAVE-like system to explore the environment. Participant responses and their impression of the virtual characters were evaluated through a wide variety of both quantitative and qualitative methods. Combining a controlled experimental approach with various data-collection methods provided a number of advantages such as providing a reason to the quantitative results. The quantitative results indicate that posture plays an important role in the communication of affect by virtual characters. The qualitative findings indicated that participants attribute a variety of psychological states to the behavioral cues displayed by virtual characters. In addition, participants tended to interpret the social context portrayed by the virtual characters in a holistic manner. This suggests that one aspect of the virtual scene colors the perception of the whole social context portrayed by the virtual characters. We conclude by discussing the importance of designing holistically congruent virtual characters especially in immersive settings.

Virtual hand illusion induced by visuomotor correlations

Our body schema gives the subjective impression of being highly stable. However, a number of easily-evoked illusions illustrate its remarkable malleability. In the rubber-hand illusion, illusory ownership of a rubber-hand is evoked by synchronous visual and tactile stimulation on a visible rubber arm and on the hidden real arm. Ownership is concurrent with a proprioceptive illusion of displacement of the arm position towards the fake arm. We have previously shown that this illusion of ownership plus the proprioceptive displacement also occurs towards a virtual 3D projection of an arm when the appropriate synchronous visuotactile stimulation is provided. Our objective here was to explore whether these illusions (ownership and proprioceptive displacement) can be induced by only synchronous visuomotor stimulation, in the absence of tactile stimulation.

Drumming in immersive virtual reality: the body shapes the way we play

It has been shown that it is possible to generate perceptual illusions of ownership in immersive virtual reality (IVR) over a virtual body seen from first person perspective, in other words over a body that visually substitutes the person's real body. This can occur even when the virtual body is quite different in appearance from the person's real body. However, investigation of the psychological, behavioral and attitudinal consequences of such body transformations remains an interesting problem with much to be discovered. Thirty six Caucasian people participated in a between-groups experiment where they played a West-African Djembe hand drum while immersed in IVR and with a virtual body that substituted their own. The virtual hand drum was registered with a physical drum. They were alongside a virtual character that played a drum in a supporting, accompanying role. In a baseline condition participants were represented only by plainly shaded white hands, so that they were able merely to play. In the experimental condition they were represented either by a casually dressed dark-skinned virtual body (Casual Dark-Skinned - CD) or by a formal suited light-skinned body (Formal Light-Skinned - FL). Although participants of both groups experienced a strong body ownership illusion towards the virtual body, only those with the CD representation showed significant increases in their movement patterns for drumming compared to the baseline condition and compared with those embodied in the FL body. Moreover, the stronger the illusion of body ownership in the CD condition, the greater this behavioral change. A path analysis showed that the observed behavioral changes were a function of the strength of the illusion of body ownership towards the virtual body and its perceived appropriateness for the drumming task. These results demonstrate that full body ownership illusions can lead to substantial behavioral and possibly cognitive changes depending on the appearance of the virtual body. This could be important for many applications such as learning, education, training, psychotherapy and rehabilitation using IVR.

Extending body space in immersive virtual reality: a very long arm illusion

Recent studies have shown that a fake body part can be incorporated into human body representation through synchronous multisensory stimulation on the fake and corresponding real body part- the most famous example being the Rubber Hand Illusion. However, the extent to which gross asymmetries in the fake body can be assimilated remains unknown. Participants experienced, through a head-tracked stereo head-mounted display a virtual body coincident with their real body. There were 5 conditions in a between-groups experiment, with 10 participants per condition. In all conditions there was visuo-motor congruence between the real and virtual dominant arm. In an Incongruent condition (I), where the virtual arm length was equal to the real length, there was visuo-tactile incongruence. In four Congruent conditions there was visuo-tactile congruence, but the virtual arm lengths were either equal to (C1), double (C2), triple (C3) or quadruple (C4) the real ones. Questionnaire scores and defensive withdrawal movements in response to a threat showed that the overall level of ownership was high in both C1 and I, and there was no significant difference between these conditions. Additionally, participants experienced ownership over the virtual arm up to three times the length of the real one, and less strongly at four times the length. The illusion did decline, however, with the length of the virtual arm. In the C2-C4 conditions although a measure of proprioceptive drift positively correlated with virtual arm length, there was no correlation between the drift and ownership of the virtual arm, suggesting different underlying mechanisms between ownership and drift. Overall, these findings extend and enrich previous results that multisensory and sensorimotor information can reconstruct our perception of the body shape, size and symmetry even when this is not consistent with normal body proportions.

Inducing illusory ownership of a virtual body

We discuss three experiments that investigate how virtual limbs and bodies can come to feel like real limbs and bodies. The fi rst experiment shows that an illusion of ownership of a virtual arm appearing to project out of a person"s shoulder can be produced by tactile stimulation on a person"s hidden real hand and synchronous stimulation on the seen virtual hand. The second shows that the illusion can be produced by synchronous movement of the person"s hidden real hand and a virtual hand. The third shows that a weaker form of the illusion can be produced when a brain-computer interface is employed to move the virtual hand by means of motor imagery without any tactile stimulation. We discuss related studies that indicate that the ownership illusion may be generated for an entire body. This has important implications for the scientific understanding of body ownership and several practical applications.