A Framework for Human-like Behavior in an Immersive Virtual World

Just as readers feel immersed when the story line adheres to their experiences, users will more easily feel immersed in a virtual environment if the behavior of the characters in that environment adheres to their expectations, based on their lifelong observations in the real world. This paper introduces a framework that allows authors to establish natural, human-like behavior, physical interaction and emotional engagement of characters living in a virtual environment. Represented by realistic virtual characters, this framework allows people to feel immersed in an Internet based virtual world in which they can meet and share experiences in a natural way as they can meet and share experiences in real life. Rather than just being visualized in a 3D space, the virtual characters (autonomous agents as well as avatars representing users) in the immersive environment facilitate social interaction and multi-party collaboration, mixing virtual with real.

Advancements and Challenges towards a Collaborative Framework for 3D Tele-Immersive Social Networking

Social experiences realized through teleconferencing systems are still quite different from face to face meetings. The awareness that we are online and in a, to some extent, lesser real world are preventing us from really engaging and enjoying the event. Several reasons account for these differences and have been identified. We think it is now time to bridge these gaps and propose inspiring and innovative solutions in order to provide realistic, believable and engaging online experiences. We present a distributed and scalable framework named REVERIE that faces these challenges and provides a mix of these solutions. Applications built on top of the framework will be able to provide interactive, truly immersive, photo-realistic experiences to a multitude of users that for them will feel much more similar to having face to face meetings than the experience offered by conventional teleconferencing systems.

REVERIE: Natural Human Interaction in Virtual Immersive Environments

REVERIE (REal and Virtual Engagement in Realistic Immersive Environments [1]) targets novel research to address the demanding challenges involved with developing state-of-the-art technologies for online human interaction. The REVERIE framework enables users to meet, socialise and share experiences online by integrating cutting-edge technologies for 3D data acquisition and processing, networking, autonomy and real-time rendering. In this paper, we describe the innovative research that is showcased through the REVERIE integrated framework through richly defined use-cases which demonstrate the validity and potential for natural interaction in a virtual immersive and safe environment. Previews of the REVERIE demo and its key research components can be viewed at www.youtube.com/user/REVERIEFP7.

Playing immersive games on the REVERIE platform

REVERIE (REal and Virtual Engagement in Realistic Immersive Environments) [1] is a multimedia and multimodal framework, which supports the creation of immersive games. The framework supports the creation of games integrating technologies such as 3D spatial audio, detection of the player’s body movement using Kinect and WIMO sensors, NPCs (Non-Playable Characters) with advanced AI capabilities featuring various levels of representation and gameplay into an immersive 3D environment. A demonstration game was developed for REVERIE, which is an adapted version of the popular Simon Says game. In the REVERIE version, a player tries to follow physical instructions issued by two autonomous agents with different degrees of realism. If a player follows a physical instruction correctly, they are awarded one point. If not, they are deducted one point. This paper presents a technical overview of the game technologies integrated in the Simon Says demo and its evaluation by players with variable computer literacy skills. Finally the potential of REVERIE as an immersive framework for gaming is discussed, followed by recommendations for improvements in future versions of the framework.

A Methodological Approach to User Evaluation and Assessment of a Virtual Environment Hangout

Innovation in virtual reality and motion sensing devices is pushing the development of virtual communication platforms towards completely immersive scenarios, which require full user interaction and create complex sensory experiences. This evolution influences user experiences and creates new paradigms for interaction, leading to an increased importance of user evaluation and assessment on new systems interfaces and usability, to validate platform design and development from the users’ point of view. The REVERIE research project aims to develop a virtual environment service for realistic inter-personal interaction. This paper describes the design challenges faced during the development process of user interfaces and the adopted methodological approach to user evaluation and assessment.

Modulation of pain threshold by virtual body ownership

Background Appropriate sensorimotor correlations can result in the illusion of ownership of exogenous body parts. Nevertheless, whether and how the illusion of owning a new body part affects human perception, and in particular pain detection, is still poorly investigated. Recent findings have shown that seeing one’s own body is analgesic, but it is not known whether this effect is transferable to newly embodied, but exogenous, body parts. In recent years, results from our laboratory have demonstrated that a virtual body can be felt as one’s own, provided realistic multisensory correlations. Methods The current work aimed at investigating the impact of virtual body ownership on pain threshold. An immersive virtual environment allowed a first-person perspective of a virtual body that replaced the own. Passive movement of the index finger congruent with the movement of the virtual index finger was used in the “synchronous” condition to induce ownership of the virtual arm. The pain threshold was tested by thermal stimulation under four conditions: 1) synchronous movements of the real and virtual fingers, 2) asynchronous movements, 3) seeing a virtual object instead of an arm, and 4) not seeing any limb in real world. Results Our results show that, independently of attentional and stimulus adaptation processes, the ownership of a virtual arm per se can significantly increase the thermal pain threshold. Conclusions This finding may be relevant for the development and improvement of digital solutions for rehabilitation and pain treatment.

A Framework for Realistic 3D Tele-Immersion

Meeting, socializing and conversing online with a group of people using teleconferencing systems is still quite different from the experience of meeting face to face. We are abruptly aware that we are online and that the people we are engaging with are not in close proximity. Analogous to how talking on the telephone does not replicate the experience of talking in person. Several causes for these differences have been identified and we propose inspiring and innovative solutions to these hurdles in attempt to provide a more realistic, believable and engaging online conversational experience. We present the distributed and scalable framework REVERIE that provides a balanced mix of these solutions. Applications build on top of the REVERIE framework will be able to provide interactive, immersive, photo-realistic experiences to a multitude of users that for them will feel much more similar to having face to face meetings than the experience offered by conventional teleconferencing systems.

Investigating the effects of caffeine on executive functions using traditional Stroop and a new ecologically-valid virtual reality task, the Jansari assessment of Executive Functions (JEF©)

Objective: Caffeine has been shown to have effects on certain areas of cognition, but in executive functioning the research is limited and also inconsistent. One reason could be the need for a more sensitive measure to detect the effects of caffeine on executive function. This study used a new non-immersive virtual reality assessment of executive functions known as JEF© (the Jansari Assessment of Executive Function) alongside the ‘classic’ Stroop Colour- Word task to assess the effects of a normal dose of caffeinated coffee on executive function. Method: Using a double-blind, counterbalanced within participants procedure 43 participants were administered either a caffeinated or decaffeinated coffee and completed the ‘JEF©’ and Stroop tasks, as well as a subjective mood scale and blood pressure pre- and post condition on two separate occasions a week apart. JEF© yields measures for eight separate aspects of executive functions, in addition to a total average score. Results: Findings indicate that performance was significantly improved on the planning, creative thinking, event-, time- and action-based prospective memory, as well as total JEF© score following caffeinated coffee relative to the decaffeinated coffee. The caffeinated beverage significantly decreased reaction times on the Stroop task, but there was no effect on Stroop interference. Conclusion: The results provide further support for the effects of a caffeinated beverage on cognitive functioning. In particular, it has demonstrated the ability of JEF© to detect the effects of caffeine across a number of executive functioning constructs, which weren’t shown in the Stroop task, suggesting executive functioning improvements as a result of a ‘typical’ dose of caffeine may only be detected by the use of more real-world, ecologically valid tasks.