Learning Two-Person Interaction Models for Responsive Synthetic Humanoids

Imitation learning is a promising approach for generating life-like behaviors of virtual humans and humanoid robots. So far, however, imitation learning has been mostly restricted to single agent settings where observed motions are adapted to new environment conditions but not to the dynamic behavior of interaction partners. In this paper, we introduce a new imitation learning approach that is based on the simultaneous motion capture of two human interaction partners. From the observed interactions, low-dimensional motion models are extracted and a mapping between these motion models is learned. This interaction model allows the real-time generation of agent behaviors that are responsive to the body movements of an interaction partner. The interaction model can be applied both to the animation of virtual characters as well as to the behavior generation for humanoid robots.

Hands-Free Navigation in Immersive Environments for the Evaluation of the Effectiveness of Indoor Navigation Systems

While navigation systems for cars are in widespread use, only recently, indoor navigation systems based on smartphone apps became technically feasible. Hence tools in order to plan and evaluate particular designs of information provision are needed. Since tests in real infrastructures are costly and environmental conditions cannot be held constant, one must resort to virtual infrastructures. This paper presents the development of an environment for the support of the design of indoor navigation systems whose center piece consists in a hands-free navigation method using the Microsoft Kinect in the four-sided Definitely Affordable Virtual Environment (DAVE). Navigation controls using the user's gestures and postures as the input to the controls are designed and implemented. The installation of expensive and bulky hardware like treadmills is avoided while still giving the user a good impression of the distance she has traveled in virtual space. An advantage in comparison to approaches using a head mounted display is that the DAVE allows the users to interact with their smartphone. Thus the effects of different indoor navigation systems can be evaluated already in the planning phase using the resulting system

Collision Detection: Broad Phase Adaptation from Multi-Core to Multi-GPU Architecture

We present in this paper several contributions on the collision detection optimization centered on hardware performance. We focus on the broad phase which is the first step of the collision detection process and propose three new ways of parallelization of the well-known Sweep and Prune algorithm. We first developed a multi-core model takes into account the number of available cores. Multi-core architecture enables us to distribute geometric computations with use of multi-threading. Critical writing section and threads idling have been minimized by introducing new data structures for each thread. Programming with directives, like OpenMP, appears to be a good compromise for code portability. We then proposed a new GPU-based algorithm also based on the "Sweep and Prune" that has been adapted to multi-GPU architectures. Our technique is based on a spatial subdivision method used to distribute computations among GPUs. Results show that significant speed-up can be obtained by passing from 1 to 4 GPUs in a large-scale environment.

Estimating Gesture Accuracy in Motion-Based Health Games

This manuscript details a technique for estimating gesture accuracy within the context of motion-based health video games using the MICROSOFT KINECT. We created a physical therapy game that requires players to imitate clinically significant reference gestures. Player performance is represented by the degree of similarity between the performed and reference gestures and is quantified by collecting the Euler angles of the player's gestures, converting them to a three-dimensional vector, and comparing the magnitude between the vectors. Lower difference values represent greater gestural correspondence and therefore greater player performance. A group of thirty-one subjects was tested. Subjects achieved gestural correspondence sufficient to complete the game's objectives while also improving their ability to perform reference gestures accurately.

A Comparative Evaluation of Three Skin Color Detection Approaches

Skin segmentation is a challenging task due to several influences such as unknown lighting conditions, skin colored background, and camera limitations. A lot of skin segmentation approaches were proposed in the past including adaptive (in the sense of updating the skin color online) and non-adaptive approaches. In this paper, we compare three skin segmentation approaches that are promising to work well for hand tracking, which is our main motivation for this work. Hand tracking can widely be used in VR/AR e.g. navigation and object manipulation. The first skin segmentation approach is a well-known non-adaptive approach. It is based on a simple, pre-computed skin color distribution. Methods two and three adaptively estimate the skin color in each frame utilizing clustering algorithms. The second approach uses a hierarchical clustering for a simultaneous image and color space segmentation, while the third approach is a pure color space clustering, but with a more sophisticated clustering approach. For evaluation, we compared the segmentation results of the approaches against a ground truth dataset. To obtain the ground truth dataset, we labeled about 500 images captured under various conditions.

Influence of Comfort on 3D Selection Task Performance in Immersive Desktop Setups

Immersive virtual environments (IVEs) have the potential to afford natural interaction in the three-dimensional (3D) space around a user. However, interaction performance in 3D mid-air is often reduced and depends on a variety of ergonomics factors, the user's endurance, muscular strength, as well as fitness. In particular, in contrast to traditional desktop-based setups, users often cannot rest their arms in a comfortable pose during the interaction. In this article we analyze the impact of comfort on 3D selection tasks in an immersive desktop setup. First, in a pre-study we identified how comfortable or uncomfortable specific interaction positions and poses are for users who are standing upright. Then, we investigated differences in 3D selection task performance when users interact with their hands in a comfortable or uncomfortable body pose, while sitting on a chair in front of a table while the VE was displayed on a headmounted display (HMD). We conducted a Fitts' Law experiment to evaluate selection performance in different poses. The results suggest that users achieve a significantly higher performance in a comfortable pose when they rest their elbow on the table.

Advanced luminance control and black offset correction for multi-projector display systems

In order to display a homogeneous image using multiple projectors, differences in the projected intensities must be compensated. In this paper, we present novel approaches to combine and extend existing techniques for edge blending and luminance harmonization to achieve a detailed luminance control. Furthermore, we apply techniques for improving the contrast ratio of multi-segmented displays also to the black offset correction. We also present a simple scheme to involve the displayed context in the correction process to dynamically improve the contrast in brighter images. In addition, we present a metric to evaluate the different methods and their influence on the visual quality.

On the comparison of VR-responses, as performance measures in prospective memory, with auditory P300 responses in MCI detection

Patients with amnestic mild cognitive impairment are at high risk for developing Alzheimer's disease. Besides episodic memory dysfunction they show deficits in accessing contextual knowledge that further specifies a general spatial navigation task or an executive function (EF) virtual action planning. There has been only one previous work with virtual reality and the use of a virtual action planning supermarket for the diagnosis of mild cognitive impairment. The authors of that study examined the feasibility and the validity of the virtual action planning supermarket (VAP-S) for the diagnosis of patients with mild cognitive impairment (MCI) and found that the VAP-S is a viable tool to assess EF deficits. In our study we employed the in-house platform of virtual action planning museum (VAP-M) and a sample of 25 MCI and 25 controls, in order to investigate deficits in spatial navigation, prospective memory and executive function. In addition, we used the morphology of late components in event-related potential (ERP) responses, as a marker for cognitive dysfunction. The related measurements were fed to a common classification scheme facilitating the direct comparison of both approaches. Our results indicate that both the VAP-M and ERP averages were able to differentiate between healthy elders and patients with amnestic mild cognitive impairment and agree with the findings of the virtual action planning supermarket (VAP-S). The sensitivity (specificity) was 100% (98%) for the VAP-M data and 87%(90%) for the ERP responses. Considering that ERPs have proven to advance the early detection and diagnosis of "presymptomatic AD", the suggested VAP-M platform appears as an appealing alternative.

New Technologies and Concepts for Rehabilitation in the Acute Phase of Stroke: A Collaborative Matrix

The process of developing a successful stroke rehabilitation methodology requires four key components: a good understanding of the pathophysiological mechanisms underlying this brain disease, clear neuroscientific hypotheses to guide therapy, adequate clinical assessments of its efficacy on multiple timescales, and a systematic approach to the application of modern technologies to assist in the everyday work of therapists. Achieving this goal requires collaboration between neuroscientists, technologists and clinicians to develop well-founded systems and clinical protocols that are able to provide quantitatively validated improvements in patient rehabilitation outcomes. In this article we present three new applications of complementary technologies developed in an interdisciplinary matrix for acute-phase upper limb stroke rehabilitation – functional electrical stimulation, arm robot-assisted therapy and virtual reality-based cognitive therapy. We also outline the neuroscientific basis of our approach, present our detailed clinical assessment protocol and provide preliminary results from patient testing of each of the three systems showing their viability for patient use.

Can a novel computerized cognitive screening test provide additional information for early detection of Alzheimer's disease?

BACKGROUND: Virtual reality testing of everyday activities is a novel type of computerized assessment that measures cognitive, executive, and motor performance as a screening tool for early dementia. This study used a virtual reality day-out task (VR-DOT) environment to evaluate its predictive value in patients with mild cognitive impairment (MCI). METHODS: One hundred thirty-four patients with MCI were selected and compared with 75 healthy control subjects. Participants received an initial assessment that included VR-DOT, a neuropsychological evaluation, magnetic resonance imaging (MRI) scan, and event-related potentials (ERPs). After 12 months, participants were assessed again with MRI, ERP, VR-DOT, and neuropsychological tests. RESULTS: At the end of the study, we differentiated two subgroups of patients with MCI according to their clinical evolution from baseline to follow-up: 56 MCI progressors and 78 MCI nonprogressors. VR-DOT performance profiles correlated strongly with existing predictive biomarkers, especially the ERP and MRI biomarkers of cortical thickness. CONCLUSIONS: Compared with ERP, MRI, or neuropsychological tests alone, the VR-DOT could provide additional predictive information in a low-cost, computerized, and noninvasive way.

Glucocorticoids enhance in vivo exposure-based therapy outcome in spider phobia

BACKGROUND: Preclinical and clinical studies indicate that the administration of glucocorticoids may promote fear extinction processes. In particular, it has been shown that glucocorticoids enhance virtual reality based exposure therapy of fear of heights. Here, we investigate whether glucocorticoids enhance the outcome of in vivo exposure-based group therapy of spider phobia. METHODS: In a double blind, block-randomized, placebo-controlled, between-subject study design, 22 patients with specific phobia of spiders were treated with two sessions of in vivo exposure-based group therapy. Cortisol (20 mg) or placebo was orally administered 1 hr before each therapy session. Patients returned for a follow-up assessment one month after therapy. RESULTS: Exposure-based group therapy led to a significant decrease in phobic symptoms as assessed with the Fear of Spiders Questionnaire (FSQ) from pretreatment to immediate posttreatment and to follow-up. The administration of cortisol to exposure therapy resulted in increased salivary cortisol concentrations and a significantly greater reduction in fear of spiders (FSQ) as compared to placebo at follow-up, but not immediately posttreatment. Furthermore, cortisol-treated patients reported significantly less anxiety during standardized exposure to living spiders at follow-up than placebo-treated subjects. Notably, groups did not differ in phobia-unrelated state-anxiety before and after the exposure sessions and at follow-up. CONCLUSIONS: These findings indicate that adding cortisol to in vivo exposure-based group therapy of spider phobia enhances treatment outcome.

Modulating presence and impulsiveness by external stimulation of the brain

BACKGROUND "The feeling of being there" is one possible way to describe the phenomenon of feeling present in a virtual environment and to act as if this environment is real. One brain area, which is hypothesized to be critically involved in modulating this feeling (also called presence) is the dorso-lateral prefrontal cortex (dlPFC), an area also associated with the control of impulsive behavior. METHODS In our experiment we applied transcranial direct current stimulation (tDCS) to the right dlPFC in order to modulate the experience of presence while watching a virtual roller coaster ride. During the ride we also registered electro-dermal activity. Subjects also performed a test measuring impulsiveness and answered a questionnaire about their presence feeling while they were exposed to the virtual roller coaster scenario. RESULTS Application of cathodal tDCS to the right dlPFC while subjects were exposed to a virtual roller coaster scenario modulates the electrodermal response to the virtual reality stimulus. In addition, measures reflecting impulsiveness were also modulated by application of cathodal tDCS to the right dlPFC. CONCLUSION Modulating the activation with the right dlPFC results in substantial changes in responses of the vegetative nervous system and changed impulsiveness. The effects can be explained by theories discussing the top-down influence of the right dlPFC on the "impulsive system".

Auditive und visuelle Perzeption bei Abseitsentscheidungen im Fussball

Einleitung Die hohe Quote von fehlerhaften Abseitsentscheidungen im Fußball (ca. 26%, Helsen et al., 2006) könnte dadurch erklärt werden, dass Schiedsrichterassistenten zum Zeitpunkt des Passes die Abseitslinie fixieren und den Moment des Passes nur peripher wahr-nehmen (Catteeuw et al., 2009). Diese Annahme wurde geprüft, indem in einem Virtual-Reality-Setting systematisch visuelle (Spielerpositionen) und akustische (Passgeräusch) Informationen manipuliert wurden. Methode In Experiment 1 hatten 28 Teilnehmer die Aufgabe, Abseitsentscheidungen aus der Per-spektive des Schiedsrichterassistenten zu treffen. Dabei wurde das Passgeräusch mani-puliert (Geräusch bei Ballabgabe: 100 ms zu früh, 100 ms zu spät, ohne). Erwartet wurde, dass die Entscheidungsrichtigkeit bei asynchronen Geräuschen sinkt. In Experiment 2 wurde in drei Gruppen die (1) Distanz zum Assistent, (2) Exzentrizität des Passgebers (Winkel zwischen Passgeber und Abseitslinie) oder (3) der zu fixierende Spieler (letzter Verteidiger oder Passgeber) manipuliert. Je Gruppe hatten 20 Teilnehmer die Aufgabe, den Moment des Passes per Knopfdruck anzugeben und Abseitsentscheidungen zu tref-fen. Zu fixierende Orte wurden instruiert und durch Eyetracking geprüft. Prädiziert wurde, dass (1) große Exzentrizitäten und (2) große Entfernungen des Passgebers die Präzision der Passwahrnehmung und die Entscheidungsrichtigkeit senken und (3) der Passmo-ment bei Fixation des Passgebers präziser erkannt wird als bei Fixation des letzten Ver-teidigers. Ergebnisse Die bisher vorliegenden Ergebnisse aus Experiment 1 zeigen, dass die akustische Wahrnehmung des Passes die Entscheidungsrichtigkeit beeinflusst, F(2, 54)= 7.44, p = .01, ηp2 = .22, indem verzögerte Ballgeräusche die Zahl der „Flag-Errors“ (fälschlicher-weise auf Abseits entschieden) erhöht und die der „Non-Flag-Errors“ (Abseits nicht er-kannt) senkt. Experiment 2 wird zeigen, welche Rolle hierbei periphere visuelle Informa-tionen spielen. Diskussion Die vermutete Rückführung von fehlerhaften Abseitsentscheidungen auf periphere (vi-suelle) Wahrnehmungsaspekte konnte bestätigt werden. Der relative Einfluss auditiver und visueller Informationen auf die Fehlerquote gilt es weiter zu untersuchen. Literatur Catteeuw, P., Helsen, W., Gilis, B., Van Roie, E., & Wagemans, J. (2009). Visual scan patterns and decision-making skills of expert assistant referees in offside situations. Journal of Sport & Exercise Psychol-ogy, 31, 786-797. Helsen, W., Gilis, B., & Weston, M. (2006). Errors in judging “offside” in association football: Test of the optical error versus the perceptual flash-lag hypothesis. Journal of Sports Sciences, 24, 521–528.

On the comparison of a novel serious game and electroencephalography biomarkers for early dementia screening

Patients with amnestic mild cognitive impairment are at high risk for developing Alzheimer's disease. Besides episodic memory dysfunction they show deficits in accessing contextual knowledge that further specifies a general spatial navigation task or an executive function (EF) virtual action planning. Virtual reality (VR) environments have already been successfully used in cognitive rehabilitation and show increased potential for use in neuropsychological evaluation allowing for greater ecological validity while being more engaging and user friendly. In our study we employed the in-house platform of virtual action planning museum (VAP-M) and a sample of 25 MCI and 25 controls, in order to investigate deficits in spatial navigation, prospective memory, and executive function. In addition, we used the morphology of late components in event-related potential (ERP) responses, as a marker for cognitive dysfunction. The related measurements were fed to a common classification scheme facilitating the direct comparison of both approaches. Our results indicate that both the VAP-M and ERP averages were able to differentiate between healthy elders and patients with amnestic mild cognitive impairment and agree with the findings of the virtual action planning supermarket (VAP-S). The sensitivity (specificity) was 100% (98%) for the VAP-M data and 87% (90%) for the ERP responses. Considering that ERPs have proven to advance the early detection and diagnosis of "presymptomatic AD," the suggested VAP-M platform appears as an appealing alternative.

Car Design and Road Crossing Behaviour

Humans possess a highly developed sensitivity for facial features. This sensitivity is also deployed to non-human beings and inanimate objects such as cars. In the present study we aimed to investigate whether car design has a bearing on the behaviour of pedestrians. Methods: An immersive virtual reality environment with a zebra crossing was used to determine a) whether the minimum accepted distance for crossing the street is bigger for cars with dominant appearance than for cars with friendly appearance (Block 1) and b) whether the speed of dominant cars are overestimated compared to friendly cars (Block 2). In Block 1, the participant's task was to cross the road in front of an approaching car at the latest moment. The point of time when entering and leaving the street was measured. In Block 2 they were asked to estimate the speed of each passing car. An independent sample rated dominant cars as being more dominant, angry and hostile than friendly cars. Results: None of the predictions regarding the car design was confirmed. Instead, there was an effect of starting position: From the centre island, participants entered the road significantly later (smaller accepted distance) and left the road later than when starting from the pavement. Consistently, the speed of the cars was estimated significantly lower when standing on the centre island compared to the pavement. When entering the visual size of the cars as factor (instead of dominance), we found that participants started to cross the road significantly later in front of small cars compared to big cars and that the speed of smaller cars was overestimated compared to big cars (size-speed bias). Conclusions: Car size and starting position, not car design seem to have an influence on road crossing behaviour.