Reinforcement learning utilizes proxemics: an avatar learns to manipulate the position of people in immersive virtual reality

A reinforcement learning (RL) method was used to train a virtual character to move participants to a specified location. The virtual environment depicted an alleyway displayed through a wide field-of-view head-tracked stereo head-mounted display. Based on proxemics theory, we predicted that when the character approached within a personal or intimate distance to the participants, they would be inclined to move backwards out of the way. We carried out a between-groups experiment with 30 female participants, with 10 assigned arbitrarily to each of the following three groups: In the Intimate condition the character could approach within 0.38m and in the Social condition no nearer than 1.2m. In the Random condition the actions of the virtual character were chosen randomly from among the same set as in the RL method, and the virtual character could approach within 0.38m. The experiment continued in each case until the participant either reached the target or 7 minutes had elapsed. The distributions of the times taken to reach the target showed significant differences between the three groups, with 9 out of 10 in the Intimate condition reaching the target significantly faster than the 6 out of 10 who reached the target in the Social condition. Only 1 out of 10 in the Random condition reached the target. The experiment is an example of applied presence theory: we rely on the many findings that people tend to respond realistically in immersive virtual environments, and use this to get people to achieve a task of which they had been unaware. This method opens up the door for many such applications where the virtual environment adapts to the responses of the human participants with the aim of achieving particular goals.

Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments

In this paper address we the question as to why participants tend to respond realistically to situations and events portrayed within an Immersive Virtual Reality (IVR) system. The idea is put forward, based on experience of a large number of experimental studies, that there are two orthogonal components that contribute to this realistic response. The first is"being there", often called"presence", the qualia of having a sensation of being in a real place. We call this Place Illusion (PI). Second, Plausibility Illusion (Psi) refers to the illusion that the scenario being depicted is actually occurring. In the case of both PI and Psi the participant knows for sure that that they are not"there" and that the events are not occurring. PI is constrained by the sensorimotor contingencies afforded by the virtual reality system. Psi is determined by the extent to which the system can produce events that directly relate to the participant, and the overall credibility of the scenario being depicted in comparison with expectations. We argue that when both PI and Psi occur, participants will respond realistically to the virtual reality.

Correlations between vocal input and visual response apparently enhance presence in a virtual environment

This work investigates novel alternative means of interaction in a virtual environment (VE).We analyze whether humans can remap established body functions to learn to interact with digital information in an environment that is cross-sensory by nature and uses vocal utterances in order to influence (abstract) virtual objects. We thus establish a correlation among learning, control of the interface, and the perceived sense of presence in the VE. The application enables intuitive interaction by mapping actions (the prosodic aspects of the human voice) to a certain response (i.e., visualization). A series of single-user and multiuser studies shows that users can gain control of the intuitive interface and learn to adapt to new and previously unseen tasks in VEs. Despite the abstract nature of the presented environment, presence scores were generally very high.

The effect of virtual reality on visual vertigo symptoms in patients with peripheral vestibular dysfunction: a pilot study

Individuals with vestibular dysfunction may experience visual vertigo (VV), in which symptoms are provoked or exacerbated by excessive or disorientating visual stimuli (e.g. supermarkets). VV can significantly improve when customized vestibular rehabilitation exercises are combined with exposure to optokinetic stimuli. Virtual reality (VR), which immerses patients in realistic, visually challenging environments, has also been suggested as an adjunct to VR to improve VV symptoms. This pilot study compared the responses of sixteen patients with unilateral peripheral vestibular disorder randomly allocated to a VR regime incorporating exposure to a static (Group S) or dynamic (Group D) VR environment. Participants practiced vestibular exercises, twice weekly for four weeks, inside a static (Group S) or dynamic (Group D) virtual crowded square environment, presented in an immersive projection theatre (IPT), and received a vestibular exercise program to practice on days not attending clinic. A third Group D1 completed both the static and dynamic VR training. Treatment response was assessed with the Dynamic Gait Index and questionnaires concerning symptom triggers and psychological state. At final assessment, significant betweengroup differences were noted between Groups D (p = 0.001) and D1 (p = 0.03) compared to Group S for VV symptoms with the former two showing a significant 59.2% and 25.8% improvement respectively compared to 1.6% for the latter. Depression scores improved only for Group S (p = 0.01) while a trend towards significance was noted for Group D regarding anxiety scores (p = 0.07). Conclusion: Exposure to dynamic VR environments should be considered as a useful adjunct to vestibular rehabilitation programs for patients with peripheral vestibular disorders and VV symptoms.

Drift and ownership toward a distant virtual body.

In body ownership illusions participants feel that a mannequin or virtual body (VB) is their own. Earlier results suggest that body ownership over a body seen from behind in extra personal space is possible when the surrogate body is visually stroked and tapped on its back, while spatially and temporal synchronous tactile stimulation is applied to the participant's back. This result has been disputed with the claim that the results can be explained by self-recognition rather than somatic body ownership. We carried out an experiment with 30 participants in a between-groups design. They all saw the back of a VB 1.2 m in front, that moved in real-time determined by upper body motion capture. All felt tactile stimulation on their back, and for 15 of them this was spatially and temporally synchronous with stimulation that they saw on the back of the VB, but asynchronous for the other 15. After 3 min a revolving fan above the VB descended and stopped at the position of the VB neck. A questionnaire assessed referral of touch to the VB, body ownership, the illusion of drifting forwards toward the VB, and the VB drifting backwards. Heart rate deceleration (HRD) and the amount of head movement during the threat period were used to assess the response to the threat from the fan. Results showed that although referral of touch was significantly greater in the synchronous condition than the asynchronous, there were no other differences between the conditions. However, a further multivariate analysis revealed that in the visuotactile synchronous condition HRD and head movement increased with the illusion of forward drift and decreased with backwards drift. Body ownership contributed positively to these drift sensations. Our conclusion is that the setup results in a contradiction-somatic feelings associated with a distant body-that the brain attempts to resolve by generating drift illusions that would make the two bodies coincide.

Touching the void: exploring virtual objects through a vibrotactile glove

This paper describes a simple low-cost approach toadding an element of haptic interaction within a virtualenvironment. Using off-the-shelf hardware and software wedescribe a simple setup that can be used to explore physically virtual objects in space. This setup comprises of a prototype glove with a number of vibrating actuators to provide the haptic feedback, a Kinect camera for the tracking of the user's hand and a virtual reality development environment. As proof of concept and to test the efficiency of the system as well as its potential applications, we developed a simple application where we created 4 different shapes within a virtual environment in order to try toexplore them and guess their shape through touch alone.

Proxemics with multiple dynamic characters in an immersive virtual environment

An experiment was carried out to examine the impact on electrodermal activity of people when approached by groups of one or four virtual characters at varying distances. It was premised on the basis of proxemics theory that the closer the approach of the virtual characters to the participant, the greater the level of physiological arousal. Physiological arousal was measured by the number of skin conductance responses within a short time period after the approach, and the maximum change in skin conductance level 5 s after the approach. The virtual characters were each either female or a cylinder of human size, and one or four characters approached each subject a total of 12 times. Twelve male subjects were recruited for the experiment. The results suggest that the number of skin conductance responses after the approach and the change in skin conductance level increased the closer the virtual characters approached toward the participants. Moreover, these response variables were inversely correlated with the number of visits, showing a typical adaptation effect. There was some evidence to suggest that the number of characters who simultaneously approached (one or four) was positively associated with the responses. Surprisingly there was no evidence of a difference in response between the humanoid characters and cylinders on the basis of this physiological data. It is suggested that the similarity in this quantitative arousal response to virtual characters and virtual objects might mask a profound difference in qualitative response, an interpretation supported by questionnaire and interview results. Overall the experiment supported the premise that people exhibit heightened physiological arousal the closer they are approached by virtual characters.

Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion

Previous studies have examined the experience of owning a virtual surrogate body or body part through specific combinations of cross-modal multisensory stimulation. Both visuomotor (VM) and visuotactile (VT) synchronous stimulation have been shown to be important for inducing a body ownership illusion, each tested separately or both in combination. In this study we compared the relative importance of these two cross-modal correlations, when both are provided in the same immersive virtual reality setup and the same experiment. We systematically manipulated VT and VM contingencies in order to assess their relative role and mutual interaction. Moreover, we present a new method for measuring the induced body ownership illusion through time, by recording reports of breaks in the illusion of ownership ("breaks") throughout the experimental phase. The balance of the evidence, from both questionnaires and analysis of the breaks, suggests that while VM synchronous stimulation contributes the greatest to the attainment of the illusion, a disruption of either (through asynchronous stimulation) contributes equally to the probability of a break in the illusion.

Functional independence within the self-memory system: new insights from two cases of developmental amnesia.

Neuropsychological and neuroimaging data suggest that the self-memory system can be fractionated into three functionally independent systems processing personal information at several levels of abstraction, including episodic memories of one's life (episodic autobiographical memory, EAM), semantic knowledge of facts about one's life (semantic autobiographical memory, SAM), and semantic knowledge of one's personality [conceptual self, (CS)]. Through the study of two developmental amnesic patients suffering of neonatal brain injuries, we explored how the different facets of the self-memory system develop when growing up with bilateral hippocampal atrophy. Neuropsychological evaluations showed that both of them suffered from dramatic episodic learning disability with no sense of recollection (Remember/Know procedure), whereas their semantic abilities differed, being completely preserved (Valentine) or not (Jocelyn). Magnetic resonance imaging, including quantitative volumetric measurements of the hippocampus and adjacent (entorhinal, perirhinal, and temporopolar) cortex, showed severe bilateral atrophy of the hippocampus in both patients, with additional atrophy of adjacent cortex in Jocelyn. Exploration of EAM and SAM according to lifetime periods covering the entire lifespan (TEMPAu task, Piolino et al., 2009) showed that both patients had marked impairments in EAM, as they lacked specificity, details and sense of recollection, whereas SAM was completely normal in Valentine, but impaired in Jocelyn. Finally, measures of patients' CS (Tennessee Self-Concept Scale, Fitts and Warren, 1996), checked by their mothers, were generally within normal range, but both patients showed a more positive self-concept than healthy controls. These two new cases support a modular account of the medial-temporal lobe with episodic memory and recollection depending on the hippocampus, and semantic memory and familiarity on adjacent cortices. Furthermore, they highlight developmental episodic and semantic functional independence within the self-memory system suggesting that SAM and CS may be acquired without episodic memories.

LRCW.NET: A web site with a virtual laboratory for the study of coarse and cooking wares in the Late Antique Mediterranean

This contribution presents the LRCW.net a website with a virtual laboratory intranet devoted to the study of coarse and cooking wares in the late Antique Mediterranean. It is designed as a public website with a virtual laboratory intranet. There, all institutions and researchers interested in the subject can work together towards a specific purpose such as the creation of an on-line ‘encyclopedia’ for these categories of ceramics. The LRCW.net website and the associated virtual laboratory are just a small part of a wider initiative that aims to create an on-line Encyclopedia for Ancient Ceramics in the Mediterranean.

Nano-particle vaccination combined with TLR-7 and -9 ligands triggers memory and effector CD8⁺ T-cell responses in melanoma patients.

Optimal vaccine strategies must be identified for improving T-cell vaccination against infectious and malignant diseases. MelQbG10 is a virus-like nano-particle loaded with A-type CpG-oligonucleotides (CpG-ODN) and coupled to peptide(16-35) derived from Melan-A/MART-1. In this phase IIa clinical study, four groups of stage III-IV melanoma patients were vaccinated with MelQbG10, given (i) with IFA (Montanide) s.c.; (ii) with IFA s.c. and topical Imiquimod; (iii) i.d. with topical Imiquimod; or (iv) as intralymph node injection. In total, 16/21 (76%) patients generated ex vivo detectable Melan-A/MART-1-specific T-cell responses. T-cell frequencies were significantly higher when IFA was used as adjuvant, resulting in detectable T-cell responses in all (11/11) patients, with predominant generation of effector-memory-phenotype cells. In turn, Imiquimod induced higher proportions of central-memory-phenotype cells and increased percentages of CD127(+) (IL-7R) T cells. Direct injection of MelQbG10 into lymph nodes resulted in lower T-cell frequencies, associated with lower proportions of memory and effector-phenotype T cells. Swelling of vaccine site draining lymph nodes, and increased glucose uptake at PET/CT was observed in 13/15 (87%) of evaluable patients, reflecting vaccine triggered immune reactions in lymph nodes. We conclude that the simultaneous use of both Imiquimod and CpG-ODN induced combined memory and effector CD8(+) T-cell responses.

Multisensory context portends object memory.

Multisensory processes facilitate perception of currently-presented stimuli and can likewise enhance later object recognition. Memories for objects originally encountered in a multisensory context can be more robust than those for objects encountered in an exclusively visual or auditory context [1], upturning the assumption that memory performance is best when encoding and recognition contexts remain constant [2]. Here, we used event-related potentials (ERPs) to provide the first evidence for direct links between multisensory brain activity at one point in time and subsequent object discrimination abilities. Across two experiments we found that individuals showing a benefit and those impaired during later object discrimination could be predicted by their brain responses to multisensory stimuli upon their initial encounter. These effects were observed despite the multisensory information being meaningless, task-irrelevant, and presented only once. We provide critical insights into the advantages associated with multisensory interactions; they are not limited to the processing of current stimuli, but likewise encompass the ability to determine the benefit of one's memories for object recognition in later, unisensory contexts.

Glucose metabolism, gray matter structure, and memory decline in subjective memory impairment.

OBJECTIVE: To identify biological evidence for Alzheimer disease (AD) in individuals with subjective memory impairment (SMI) and unimpaired cognitive performance and to investigate the longitudinal cognitive course in these subjects. METHOD: [¹⁸F]fluoro-2-deoxyglucose PET (FDG-PET) and structural MRI were acquired in 31 subjects with SMI and 56 controls. Cognitive follow-up testing was performed (average follow-up time: 35 months). Differences in baseline brain imaging data and in memory decline were assessed between both groups. Associations of memory decline with brain imaging data were tested. RESULTS: The SMI group showed hypometabolism in the right precuneus and hypermetabolism in the right medial temporal lobe. Gray matter volume was reduced in the right hippocampus in the SMI group. At follow-up, subjects with SMI showed a poorer performance than controls on measures of episodic memory. Longitudinal memory decline in the SMI group was associated with reduced glucose metabolism in the right precuneus at baseline. CONCLUSION: The cross-sectional difference in 2 independent neuroimaging modalities indicates early AD pathology in SMI. The poorer memory performance at follow-up and the association of reduced longitudinal memory performance with hypometabolism in the precuneus at baseline support the concept of SMI as the earliest manifestation of AD.

Customization, transparency and proximity: a user- centered content strategy applied to the design of a virtual library website

We present a case study of the redesign of the organizational presentation and content of the Virtual Library website at the Universitat Oberta de Catalunya (Open University of Catalonia, UOC), based on a user-centered design strategy. The aim of the redesign was to provide users with more intuitive, usable and understandable content (textual content, resources and services) by implementing criteria of customization, transparency and proximity. The study also presents a selection of best practices for applying these criteria to the design of other library websites.