The MIRELA framework: modeling and analyzing mixed reality applications using timed automata

Mixed Reality (MR) aims to link virtual entities with the real world and has many applications such as military and medical domains [JBL+00, NFB07]. In many MR systems and more precisely in augmented scenes, one needs the application to render the virtual part accurately at the right time. To achieve this, such systems acquire data related to the real world from a set of sensors before rendering virtual entities. A suitable system architecture should minimize the delays to keep the overall system delay (also called end-to-end latency) within the requirements for real-time performance. In this context, we propose a compositional modeling framework for MR software architectures in order to specify, simulate and validate formally the time constraints of such systems. Our approach is first based on a functional decomposition of such systems into generic components. The obtained elements as well as their typical interactions give rise to generic representations in terms of timed automata. A whole system is then obtained as a composition of such defined components. To write specifications, a textual language named MIRELA (MIxed REality LAnguage) is proposed along with the corresponding compilation tools. The generated output contains timed automata in UPPAAL format for simulation and verification of time constraints. These automata may also be used to generate source code skeletons for an implementation on a MR platform. The approach is illustrated first on a small example. A realistic case study is also developed. It is modeled by several timed automata synchronizing through channels and including a large number of time constraints. Both systems have been simulated in UPPAAL and checked against the required behavioral properties.

Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an EEG and psychophysiology study

Using electroencephalography (EEG), psychophysiology, and psychometric measures, this is the first study which investigated the neurophysiological underpinnings of spatial presence. Spatial presence is considered a sense of being physically situated within a spatial environment portrayed by a medium (e.g., television, virtual reality). Twelve healthy children and 11 healthy adolescents were watching different virtual roller coaster scenarios. During a control session, the roller coaster cab drove through a horizontal roundabout track. The following realistic roller coaster rides consisted of spectacular ups, downs, and loops. Low-resolution brain electromagnetic tomography (LORETA) and event-related desynchronization (ERD) were used to analyze the EEG data. As expected, we found that, compared to the control condition, experiencing a virtual roller coaster ride evoked in both groups strong SP experiences, increased electrodermal reactions, and activations in parietal brain areas known to be involved in spatial navigation. In addition, brain areas that receive homeostatic afferents from somatic and visceral sensations of the body were strongly activated. Most interesting, children (as compared to adolescents) reported higher spatial presence experiences and demonstrated a different frontal activation pattern. While adolescents showed increased activation in prefrontal areas known to be involved in the control of executive functions, children demonstrated a decreased activity in these brain regions. Interestingly, recent neuroanatomical and neurophysiological studies have shown that the frontal brain continues to develop to adult status well into adolescence. Thus, the result of our study implies that the increased spatial presence experience in children may result from the not fully developed control functions of the frontal cortex.

A data-globe and immersive virtual reality environment for upper limb rehabilitation after spinal cord injury

While a number of virtual data-gloves have been used in stroke, there is little evidence about their use in spinal cord injury (SCI). A pilot clinical experience with nine SCI subjects was performed comparing two groups: one carried out a virtual rehabilitation training based on the use of a data glove, CyberTouch combined with traditional rehabilitation, during 30 minutes a day twice a week along two weeks; while the other made only conventional rehabilitation. Furthermore, two functional indexes were developed in order to assess the patient’s performance of the sessions: normalized trajectory lengths and repeatability. While differences between groups were not statistically significant, the data-glove group seemed to obtain better results in the muscle balance and functional parameters, and in the dexterity, coordination and fine grip tests. Related to the indexes that we implemented, normalized trajectory lengths and repeatability, every patient showed an improvement in at least one of the indexes, either along Y-axis trajectory or Z-axis trajectory. This study might be a step in investigating new ways of treatments and objective measures in order to obtain more accurate data about the patient’s evolution, allowing the clinicians to develop rehabilitation treatments, adapted to the abilities and needs of the patients.

Does Training Laparoscopic Skills in a Virtual Reality Simulator Improve Surgical Performance?

Background and Purpose: Several different methods of teaching laparoscopic skills have been advocated, with virtual reality surgical simulation (VRSS) being the most popular. Its effectiveness in improving surgical performance is not a consensus yet, however. The purpose of this study was to determine whether practicing surgical skills in a virtual reality simulator results in improved surgical performance. Materials and Methods: Fifteen medical students recruited for the study were divided into three groups. Group I (control) did not receive any VRSS training. For 10 weeks, group II trained basic laparoscopic skills (camera handling, cutting skill, peg transfer skill, and clipping skill) in a VRSS laparoscopic skills simulator. Group III practiced the same skills and, in addition, performed a simulated cholecystectomy. All students then performed a cholecystectomy in a swine model. Their performance was reviewed by two experienced surgeons. The following parameters were evaluated: Gallbladder pedicle dissection time, clipping time, time for cutting the pedicle, gallbladder removal time, total procedure time, and blood loss. Results: With practice, there was improvement in most of the evaluated parameters by each of the individuals. There were no statistical differences in any of evaluated parameters between those who did and did not undergo VRSS training, however. Conclusion: VRSS training is assumed to be an effective tool for learning and practicing laparoscopic skills. In this study, we could not demonstrate that VRSS training resulted in improved surgical performance. It may be useful, however, in familiarizing surgeons with laparoscopic surgery. More effective methods of teaching laparoscopic skills should be evaluated to help in improving surgical performance.

Virtual reality in assessment and treatment of schizophrenia: a systematic review

Objective To conduct a systematic review about the use of virtual reality (VR) for evaluation, treatment and/or rehabilitation of patients with schizophrenia, focused on: areas, fields and objectives; methodological issues; features of the VR used; viability and efficiency of this resource. Methods Searches were performed about schizophrenia and virtual reality in PsycINFO, Academic Search Complete, MEDLINE Complete, CINAHL with Full Text, Web of Science and Business Source Premier databases, using the following keywords: [“schizophrenia”] AND [“virtual reality” OR “serious game”] AND [“treatment” OR “therapy” OR “rehabilitation”]. The search was carried out between November 2013 and June 2014 without using any search limiters. Results A total of 101 papers were identified, and after the application of exclusion criteria, 33 papers remained. The studies analysed focused on the use of VR for the evaluation of cognitive, social, perceptual and sensory skills, and the vast majority were experimental studies, with virtual reality specifically created for them. All the reviewed papers point towards a reliable and safe use of VR for evaluating and treating cognitive and social deficits in patients with schizophrenia, with different results in terms of generalisation, motivation, assertiveness and task participation rate. Some problems were highlighted, such as its high cost and a constant need for software maintenance. Conclusion The studies show that using the virtual reality may streamline traditional evaluation/rehabilitation programmes, allowing to enhance the results achieved, both in the cognitive and in the social field, helping for the legitimisation of this population’s psycho-social inclusion.

integrated method for improving risk analysis using human factors methods and virtual reality

Magdeburg, Univ., Fak. für Informatik, Diss., 2009

Exploration of the visual system : Part 2. In vivo analysis methods : virtual-reality optomotor system, fundus examination and fluorescent angiography

The mouse has emerged as an animal model for many diseases. At IRO, we have used this animal to understand the development of many eye diseases and treatment of some of them. Precise evaluation of vision is a prerequisite for both these approaches. In this unit we describe three ways to measure vision: testing the optokinetic response, and evaluating the fundus by direct observation and by fluorescent angiography.

Animal dispersal modelling: handling landscape features and related animal choices

Animal dispersal in a fragmented landscape depends on the complex interaction between landscape structure and animal behavior. To better understand how individuals disperse, it is important to explicitly represent the properties of organisms and the landscape in which they move. A common approach to modelling dispersal includes representing the landscape as a grid of equal sized cells and then simulating individual movement as a correlated random walk. This approach uses a priori scale of resolution, which limits the representation of all landscape features and how different dispersal abilities are modelled. We develop a vector-based landscape model coupled with an object-oriented model for animal dispersal. In this spatially explicit dispersal model, landscape features are defined based on their geographic and thematic properties and dispersal is modelled through consideration of an organism's behavior, movement rules and searching strategies (such as visual cues). We present the model's underlying concepts, its ability to adequately represent landscape features and provide simulation of dispersal according to different dispersal abilities. We demonstrate the potential of the model by simulating two virtual species in a real Swiss landscape. This illustrates the model's ability to simulate complex dispersal processes and provides information about dispersal such as colonization probability and spatial distribution of the organism's path.

Personality traits and virtual reality performance.

BACKGROUND: Surgeons' personalities have been described as different from those of the general population, but this was based on small descriptive studies limited by the choice of evaluation instrument. Furthermore, although the importance of the human factor in team performance has been recognized, the effect of personality traits on technical performance is unknown. This study aimed to compare surgical residents' personality traits with those of the general population and to evaluate whether an association exists between their personality traits and technical performance using a virtual reality (VR) laparoscopy simulator. METHODS: In this study, 95 participants (54 residents with basic, 29 with intermediate laparoscopic experience, and 12 students) underwent personality assessment using the NEO-Five Factor Inventory and performed five VR tasks of the Lap Mentor? basic tasks module. The residents' personality traits were compared with those of the general population, and the association between VR performance and personality traits was investigated. RESULTS: Surgical residents showed personality traits different from those of the general population, demonstrating lower neuroticism, higher extraversion and conscientiousness, and male residents showed greater openness. In the multivariable analysis, adjusted for gender and surgical experience, none of the personality traits was found to be an independent predictor of technical performance. CONCLUSIONS: Surgical residents present distinct personality traits that differ from those of the general population. These traits were not found to be associated with technical performance in a virtual environment. The traits may, however, play an important role in team performance, which in turn is highly relevant for optimal surgical performance.

Assessment of construct validity of a virtual reality laparoscopy simulator.

BACKGROUND: The aim of this study was to assess whether virtual reality (VR) can discriminate between the skills of novices and intermediate-level laparoscopic surgical trainees (construct validity), and whether the simulator assessment correlates with an expert's evaluation of performance. METHODS: Three hundred and seven (307) participants of the 19th-22nd Davos International Gastrointestinal Surgery Workshops performed the clip-and-cut task on the Xitact LS 500 VR simulator (Xitact S.A., Morges, Switzerland). According to their previous experience in laparoscopic surgery, participants were assigned to the basic course (BC) or the intermediate course (IC). Objective performance parameters recorded by the simulator were compared to the standardized assessment by the course instructors during laparoscopic pelvitrainer and conventional surgery exercises. RESULTS: IC participants performed significantly better on the VR simulator than BC participants for the task completion time as well as the economy of movement of the right instrument, not the left instrument. Participants with maximum scores in the pelvitrainer cholecystectomy task performed the VR trial significantly faster, compared to those who scored less. In the conventional surgery task, a significant difference between those who scored the maximum and those who scored less was found not only for task completion time, but also for economy of movement of the right instrument. CONCLUSIONS: VR simulation provides a valid assessment of psychomotor skills and some basic aspects of spatial skills in laparoscopic surgery. Furthermore, VR allows discrimination between trainees with different levels of experience in laparoscopic surgery establishing construct validity for the Xitact LS 500 clip-and-cut task. Virtual reality may become the gold standard to assess and monitor surgical skills in laparoscopic surgery.

The future of patient safety: Surgical trainees accept virtual reality as a new training tool.

BACKGROUND: The use of virtual reality (VR) has gained increasing interest to acquire laparoscopic skills outside the operating theatre and thus increasing patients' safety. The aim of this study was to evaluate trainees' acceptance of VR for assessment and training during a skills course and at their institution. METHODS: All 735 surgical trainees of the International Gastrointestinal Surgery Workshop 2006-2008, held in Davos, Switzerland, were given a minimum of 45 minutes for VR training during the course. Participants' opinion on VR was analyzed with a standardized questionnaire. RESULTS: Fivehundred-twenty-seven participants (72%) from 28 countries attended the VR sessions and answered the questionnaires. The possibility of using VR at the course was estimated as excellent or good in 68%, useful in 21%, reasonable in 9% and unsuitable or useless in 2%. If such VR simulators were available at their institution, most course participants would train at least one hour per week (46%), two or more hours (42%) and only 12% wouldn't use VR. Similarly, 63% of the participants would accept to operate on patients only after VR training and 55% to have VR as part of their assessment. CONCLUSION: Residents accept and appreciate VR simulation for surgical assessment and training. The majority of the trainees are motivated to regularly spend time for VR training if accessible.

Feasibility of Visualization and Simulation Applications to Improve Work Zone Safety and Mobility

Visualization is a relatively recent tool available to engineers for enhancing transportation project design through improved communication, decision making, and stakeholder feedback. Current visualization techniques include image composites, video composites, 2D drawings, drive-through or fly-through animations, 3D rendering models, virtual reality, and 4D CAD. These methods are used mainly to communicate within the design and construction team and between the team and external stakeholders. Use of visualization improves understanding of design intent and project concepts and facilitates effective decision making. However, visualization tools are typically used for presentation only in large-scale urban projects. Visualization is not widely accepted due to a lack of demonstrated engineering benefits for typical agency projects, such as small- and medium-sized projects, rural projects, and projects where external stakeholder communication is not a major issue. Furthermore, there is a perceived high cost of investment of both financial and human capital in adopting visualization tools. The most advanced visualization technique of virtual reality has only been used in academic research settings, and 4D CAD has been used on a very limited basis for highly complicated specialty projects. However, there are a number of less intensive visualization methods available which may provide some benefit to many agency projects. In this paper, we present the results of a feasibility study examining the use of visualization and simulation applications for improving highway planning, design, construction, and safety and mobility.

Feasibility of Visualization and Simulation Applications, Tech Transfer Summary, 2008

Visualization is a relatively recent tool available to engineers for enhancing transportation project design through improved communication, decision making, and stakeholder feedback. Current visualization techniques include image composites, video composites, 2D drawings, drive-through or fly-through animations, 3D rendering models, virtual reality, and 4D CAD. These methods are used mainly to communicate within the design and construction team and between the team and external stakeholders. Use of visualization improves understanding of design intent and project concepts and facilitates effective decision making. However, visualization tools are typically used for presentation only in large-scale urban projects. Visualization is not widely accepted due to a lack of demonstrated engineering benefits for typical agency projects, such as small- and medium-sized projects, rural projects, and projects where external stakeholder communication is not a major issue. Furthermore, there is a perceived high cost of investment of both financial and human capital in adopting visualization tools. The most advanced visualization technique of virtual reality has only been used in academic research settings, and 4D CAD has been used on a very limited basis for highly complicated specialty projects. However, there are a number of less intensive visualization methods available which may provide some benefit to many agency projects. In this paper, we present the results of a feasibility study examining the use of visualization and simulation applications for improving highway planning, design, construction, and safety and mobility.

Reinforcement learning utilizes proxemics: an avatar learns to manipulate the position of people in immersive virtual reality

A reinforcement learning (RL) method was used to train a virtual character to move participants to a specified location. The virtual environment depicted an alleyway displayed through a wide field-of-view head-tracked stereo head-mounted display. Based on proxemics theory, we predicted that when the character approached within a personal or intimate distance to the participants, they would be inclined to move backwards out of the way. We carried out a between-groups experiment with 30 female participants, with 10 assigned arbitrarily to each of the following three groups: In the Intimate condition the character could approach within 0.38m and in the Social condition no nearer than 1.2m. In the Random condition the actions of the virtual character were chosen randomly from among the same set as in the RL method, and the virtual character could approach within 0.38m. The experiment continued in each case until the participant either reached the target or 7 minutes had elapsed. The distributions of the times taken to reach the target showed significant differences between the three groups, with 9 out of 10 in the Intimate condition reaching the target significantly faster than the 6 out of 10 who reached the target in the Social condition. Only 1 out of 10 in the Random condition reached the target. The experiment is an example of applied presence theory: we rely on the many findings that people tend to respond realistically in immersive virtual environments, and use this to get people to achieve a task of which they had been unaware. This method opens up the door for many such applications where the virtual environment adapts to the responses of the human participants with the aim of achieving particular goals.