Learning Two-Person Interaction Models for Responsive Synthetic Humanoids

Imitation learning is a promising approach for generating life-like behaviors of virtual humans and humanoid robots. So far, however, imitation learning has been mostly restricted to single agent settings where observed motions are adapted to new environment conditions but not to the dynamic behavior of interaction partners. In this paper, we introduce a new imitation learning approach that is based on the simultaneous motion capture of two human interaction partners. From the observed interactions, low-dimensional motion models are extracted and a mapping between these motion models is learned. This interaction model allows the real-time generation of agent behaviors that are responsive to the body movements of an interaction partner. The interaction model can be applied both to the animation of virtual characters as well as to the behavior generation for humanoid robots.

A user supported object tracking framework for interactive video production

We present a user supported tracking framework that combines automatic tracking with extended user input to create error free tracking results that are suitable for interactive video production. The goal of our approach is to keep the necessary user input as small as possible. In our framework, the user can select between different tracking algorithms - existing ones and new ones that are described in this paper. Furthermore, the user can automatically fuse the results of different tracking algorithms with our robust fusion approach. The tracked object can be marked in more than one frame, which can significantly improve the tracking result. After tracking, the user can validate the results in an easy way, thanks to the support of a powerful interpolation technique. The tracking results are iteratively improved until the complete track has been found. After the iterative editing process the tracking result of each object is stored in an interactive video file that can be loaded by our player for interactive videos.

Collision Detection: Broad Phase Adaptation from Multi-Core to Multi-GPU Architecture

We present in this paper several contributions on the collision detection optimization centered on hardware performance. We focus on the broad phase which is the first step of the collision detection process and propose three new ways of parallelization of the well-known Sweep and Prune algorithm. We first developed a multi-core model takes into account the number of available cores. Multi-core architecture enables us to distribute geometric computations with use of multi-threading. Critical writing section and threads idling have been minimized by introducing new data structures for each thread. Programming with directives, like OpenMP, appears to be a good compromise for code portability. We then proposed a new GPU-based algorithm also based on the "Sweep and Prune" that has been adapted to multi-GPU architectures. Our technique is based on a spatial subdivision method used to distribute computations among GPUs. Results show that significant speed-up can be obtained by passing from 1 to 4 GPUs in a large-scale environment.

Estimating Gesture Accuracy in Motion-Based Health Games

This manuscript details a technique for estimating gesture accuracy within the context of motion-based health video games using the MICROSOFT KINECT. We created a physical therapy game that requires players to imitate clinically significant reference gestures. Player performance is represented by the degree of similarity between the performed and reference gestures and is quantified by collecting the Euler angles of the player's gestures, converting them to a three-dimensional vector, and comparing the magnitude between the vectors. Lower difference values represent greater gestural correspondence and therefore greater player performance. A group of thirty-one subjects was tested. Subjects achieved gestural correspondence sufficient to complete the game's objectives while also improving their ability to perform reference gestures accurately.

A Comparative Evaluation of Three Skin Color Detection Approaches

Skin segmentation is a challenging task due to several influences such as unknown lighting conditions, skin colored background, and camera limitations. A lot of skin segmentation approaches were proposed in the past including adaptive (in the sense of updating the skin color online) and non-adaptive approaches. In this paper, we compare three skin segmentation approaches that are promising to work well for hand tracking, which is our main motivation for this work. Hand tracking can widely be used in VR/AR e.g. navigation and object manipulation. The first skin segmentation approach is a well-known non-adaptive approach. It is based on a simple, pre-computed skin color distribution. Methods two and three adaptively estimate the skin color in each frame utilizing clustering algorithms. The second approach uses a hierarchical clustering for a simultaneous image and color space segmentation, while the third approach is a pure color space clustering, but with a more sophisticated clustering approach. For evaluation, we compared the segmentation results of the approaches against a ground truth dataset. To obtain the ground truth dataset, we labeled about 500 images captured under various conditions.

Influence of Comfort on 3D Selection Task Performance in Immersive Desktop Setups

Immersive virtual environments (IVEs) have the potential to afford natural interaction in the three-dimensional (3D) space around a user. However, interaction performance in 3D mid-air is often reduced and depends on a variety of ergonomics factors, the user's endurance, muscular strength, as well as fitness. In particular, in contrast to traditional desktop-based setups, users often cannot rest their arms in a comfortable pose during the interaction. In this article we analyze the impact of comfort on 3D selection tasks in an immersive desktop setup. First, in a pre-study we identified how comfortable or uncomfortable specific interaction positions and poses are for users who are standing upright. Then, we investigated differences in 3D selection task performance when users interact with their hands in a comfortable or uncomfortable body pose, while sitting on a chair in front of a table while the VE was displayed on a headmounted display (HMD). We conducted a Fitts' Law experiment to evaluate selection performance in different poses. The results suggest that users achieve a significantly higher performance in a comfortable pose when they rest their elbow on the table.

Advanced luminance control and black offset correction for multi-projector display systems

In order to display a homogeneous image using multiple projectors, differences in the projected intensities must be compensated. In this paper, we present novel approaches to combine and extend existing techniques for edge blending and luminance harmonization to achieve a detailed luminance control. Furthermore, we apply techniques for improving the contrast ratio of multi-segmented displays also to the black offset correction. We also present a simple scheme to involve the displayed context in the correction process to dynamically improve the contrast in brighter images. In addition, we present a metric to evaluate the different methods and their influence on the visual quality.

On the comparison of VR-responses, as performance measures in prospective memory, with auditory P300 responses in MCI detection

Patients with amnestic mild cognitive impairment are at high risk for developing Alzheimer's disease. Besides episodic memory dysfunction they show deficits in accessing contextual knowledge that further specifies a general spatial navigation task or an executive function (EF) virtual action planning. There has been only one previous work with virtual reality and the use of a virtual action planning supermarket for the diagnosis of mild cognitive impairment. The authors of that study examined the feasibility and the validity of the virtual action planning supermarket (VAP-S) for the diagnosis of patients with mild cognitive impairment (MCI) and found that the VAP-S is a viable tool to assess EF deficits. In our study we employed the in-house platform of virtual action planning museum (VAP-M) and a sample of 25 MCI and 25 controls, in order to investigate deficits in spatial navigation, prospective memory and executive function. In addition, we used the morphology of late components in event-related potential (ERP) responses, as a marker for cognitive dysfunction. The related measurements were fed to a common classification scheme facilitating the direct comparison of both approaches. Our results indicate that both the VAP-M and ERP averages were able to differentiate between healthy elders and patients with amnestic mild cognitive impairment and agree with the findings of the virtual action planning supermarket (VAP-S). The sensitivity (specificity) was 100% (98%) for the VAP-M data and 87%(90%) for the ERP responses. Considering that ERPs have proven to advance the early detection and diagnosis of "presymptomatic AD", the suggested VAP-M platform appears as an appealing alternative.

Can a novel computerized cognitive screening test provide additional information for early detection of Alzheimer's disease?

BACKGROUND: Virtual reality testing of everyday activities is a novel type of computerized assessment that measures cognitive, executive, and motor performance as a screening tool for early dementia. This study used a virtual reality day-out task (VR-DOT) environment to evaluate its predictive value in patients with mild cognitive impairment (MCI). METHODS: One hundred thirty-four patients with MCI were selected and compared with 75 healthy control subjects. Participants received an initial assessment that included VR-DOT, a neuropsychological evaluation, magnetic resonance imaging (MRI) scan, and event-related potentials (ERPs). After 12 months, participants were assessed again with MRI, ERP, VR-DOT, and neuropsychological tests. RESULTS: At the end of the study, we differentiated two subgroups of patients with MCI according to their clinical evolution from baseline to follow-up: 56 MCI progressors and 78 MCI nonprogressors. VR-DOT performance profiles correlated strongly with existing predictive biomarkers, especially the ERP and MRI biomarkers of cortical thickness. CONCLUSIONS: Compared with ERP, MRI, or neuropsychological tests alone, the VR-DOT could provide additional predictive information in a low-cost, computerized, and noninvasive way.

Modulating presence and impulsiveness by external stimulation of the brain

BACKGROUND "The feeling of being there" is one possible way to describe the phenomenon of feeling present in a virtual environment and to act as if this environment is real. One brain area, which is hypothesized to be critically involved in modulating this feeling (also called presence) is the dorso-lateral prefrontal cortex (dlPFC), an area also associated with the control of impulsive behavior. METHODS In our experiment we applied transcranial direct current stimulation (tDCS) to the right dlPFC in order to modulate the experience of presence while watching a virtual roller coaster ride. During the ride we also registered electro-dermal activity. Subjects also performed a test measuring impulsiveness and answered a questionnaire about their presence feeling while they were exposed to the virtual roller coaster scenario. RESULTS Application of cathodal tDCS to the right dlPFC while subjects were exposed to a virtual roller coaster scenario modulates the electrodermal response to the virtual reality stimulus. In addition, measures reflecting impulsiveness were also modulated by application of cathodal tDCS to the right dlPFC. CONCLUSION Modulating the activation with the right dlPFC results in substantial changes in responses of the vegetative nervous system and changed impulsiveness. The effects can be explained by theories discussing the top-down influence of the right dlPFC on the "impulsive system".

Auditive und visuelle Perzeption bei Abseitsentscheidungen im Fussball

Einleitung Die hohe Quote von fehlerhaften Abseitsentscheidungen im Fußball (ca. 26%, Helsen et al., 2006) könnte dadurch erklärt werden, dass Schiedsrichterassistenten zum Zeitpunkt des Passes die Abseitslinie fixieren und den Moment des Passes nur peripher wahr-nehmen (Catteeuw et al., 2009). Diese Annahme wurde geprüft, indem in einem Virtual-Reality-Setting systematisch visuelle (Spielerpositionen) und akustische (Passgeräusch) Informationen manipuliert wurden. Methode In Experiment 1 hatten 28 Teilnehmer die Aufgabe, Abseitsentscheidungen aus der Per-spektive des Schiedsrichterassistenten zu treffen. Dabei wurde das Passgeräusch mani-puliert (Geräusch bei Ballabgabe: 100 ms zu früh, 100 ms zu spät, ohne). Erwartet wurde, dass die Entscheidungsrichtigkeit bei asynchronen Geräuschen sinkt. In Experiment 2 wurde in drei Gruppen die (1) Distanz zum Assistent, (2) Exzentrizität des Passgebers (Winkel zwischen Passgeber und Abseitslinie) oder (3) der zu fixierende Spieler (letzter Verteidiger oder Passgeber) manipuliert. Je Gruppe hatten 20 Teilnehmer die Aufgabe, den Moment des Passes per Knopfdruck anzugeben und Abseitsentscheidungen zu tref-fen. Zu fixierende Orte wurden instruiert und durch Eyetracking geprüft. Prädiziert wurde, dass (1) große Exzentrizitäten und (2) große Entfernungen des Passgebers die Präzision der Passwahrnehmung und die Entscheidungsrichtigkeit senken und (3) der Passmo-ment bei Fixation des Passgebers präziser erkannt wird als bei Fixation des letzten Ver-teidigers. Ergebnisse Die bisher vorliegenden Ergebnisse aus Experiment 1 zeigen, dass die akustische Wahrnehmung des Passes die Entscheidungsrichtigkeit beeinflusst, F(2, 54)= 7.44, p = .01, ηp2 = .22, indem verzögerte Ballgeräusche die Zahl der „Flag-Errors“ (fälschlicher-weise auf Abseits entschieden) erhöht und die der „Non-Flag-Errors“ (Abseits nicht er-kannt) senkt. Experiment 2 wird zeigen, welche Rolle hierbei periphere visuelle Informa-tionen spielen. Diskussion Die vermutete Rückführung von fehlerhaften Abseitsentscheidungen auf periphere (vi-suelle) Wahrnehmungsaspekte konnte bestätigt werden. Der relative Einfluss auditiver und visueller Informationen auf die Fehlerquote gilt es weiter zu untersuchen. Literatur Catteeuw, P., Helsen, W., Gilis, B., Van Roie, E., & Wagemans, J. (2009). Visual scan patterns and decision-making skills of expert assistant referees in offside situations. Journal of Sport & Exercise Psychol-ogy, 31, 786-797. Helsen, W., Gilis, B., & Weston, M. (2006). Errors in judging “offside” in association football: Test of the optical error versus the perceptual flash-lag hypothesis. Journal of Sports Sciences, 24, 521–528.

On the comparison of a novel serious game and electroencephalography biomarkers for early dementia screening

Patients with amnestic mild cognitive impairment are at high risk for developing Alzheimer's disease. Besides episodic memory dysfunction they show deficits in accessing contextual knowledge that further specifies a general spatial navigation task or an executive function (EF) virtual action planning. Virtual reality (VR) environments have already been successfully used in cognitive rehabilitation and show increased potential for use in neuropsychological evaluation allowing for greater ecological validity while being more engaging and user friendly. In our study we employed the in-house platform of virtual action planning museum (VAP-M) and a sample of 25 MCI and 25 controls, in order to investigate deficits in spatial navigation, prospective memory, and executive function. In addition, we used the morphology of late components in event-related potential (ERP) responses, as a marker for cognitive dysfunction. The related measurements were fed to a common classification scheme facilitating the direct comparison of both approaches. Our results indicate that both the VAP-M and ERP averages were able to differentiate between healthy elders and patients with amnestic mild cognitive impairment and agree with the findings of the virtual action planning supermarket (VAP-S). The sensitivity (specificity) was 100% (98%) for the VAP-M data and 87% (90%) for the ERP responses. Considering that ERPs have proven to advance the early detection and diagnosis of "presymptomatic AD," the suggested VAP-M platform appears as an appealing alternative.

Establishing the skill of climate field reconstruction techniques for precipitation with pseudoproxy experiments

This study aims at assessing the skill of several climate field reconstruction techniques (CFR) to reconstruct past precipitation over continental Europe and the Mediterranean at seasonal time scales over the last two millennia from proxy records. A number of pseudoproxy experiments are performed within the virtual reality ofa regional paleoclimate simulation at 45 km resolution to analyse different aspects of reconstruction skill. Canonical Correlation Analysis (CCA), two versions of an Analog Method (AM) and Bayesian hierarchical modeling (BHM) are applied to reconstruct precipitation from a synthetic network of pseudoproxies that are contaminated with various types of noise. The skill of the derived reconstructions is assessed through comparison with precipitation simulated by the regional climate model. Unlike BHM, CCA systematically underestimates the variance. The AM can be adjusted to overcome this shortcoming, presenting an intermediate behaviour between the two aforementioned techniques. However, a trade-off between reconstruction-target correlations and reconstructed variance is the drawback of all CFR techniques. CCA (BHM) presents the largest (lowest) skill in preserving the temporal evolution, whereas the AM can be tuned to reproduce better correlation at the expense of losing variance. While BHM has been shown to perform well for temperatures, it relies heavily on prescribed spatial correlation lengths. While this assumption is valid for temperature, it is hardly warranted for precipitation. In general, none of the methods outperforms the other. All experiments agree that a dense and regularly distributed proxy network is required to reconstruct precipitation accurately, reflecting its high spatial and temporal variability. This is especially true in summer, when a specifically short de-correlation distance from the proxy location is caused by localised summertime convective precipitation events.

Real-time computer-based visual feedback improves visual acuity in downbeat nystagmus - a pilot study.

BACKGROUND Patients with downbeat nystagmus syndrome suffer from oscillopsia, which leads to an unstable visual perception and therefore impaired visual acuity. The aim of this study was to use real-time computer-based visual feedback to compensate for the destabilizing slow phase eye movements. METHODS The patients were sitting in front of a computer screen with the head fixed on a chin rest. The eye movements were recorded by an eye tracking system (EyeSeeCam®). We tested the visual acuity with a fixed Landolt C (static) and during real-time feedback driven condition (dynamic) in gaze straight ahead and (20°) sideward gaze. In the dynamic condition, the Landolt C moved according to the slow phase eye velocity of the downbeat nystagmus. The Shapiro-Wilk test was used to test for normal distribution and one-way ANOVA for comparison. RESULTS Ten patients with downbeat nystagmus were included in the study. Median age was 76 years and the median duration of symptoms was 6.3 years (SD +/- 3.1y). The mean slow phase velocity was moderate during gaze straight ahead (1.44°/s, SD +/- 1.18°/s) and increased significantly in sideward gaze (mean left 3.36°/s; right 3.58°/s). In gaze straight ahead, we found no difference between the static and feedback driven condition. In sideward gaze, visual acuity improved in five out of ten subjects during the feedback-driven condition (p = 0.043). CONCLUSIONS This study provides proof of concept that non-invasive real-time computer-based visual feedback compensates for the SPV in DBN. Therefore, real-time visual feedback may be a promising aid for patients suffering from oscillopsia and impaired text reading on screen. Recent technological advances in the area of virtual reality displays might soon render this approach feasible in fully mobile settings.

Study on the response of IFMIF fission chambers to mixed neutron-gamma fields: PH-2 experimental tests

The engineering design of fissionchambers as on-line radiation detectors for IFMIF is being performed in the framework of the IFMIF-EVEDA works. In this paper the results of the experiments performed in the BR2 reactor during the phase-2 of the foreseen validation activities are addressed. Two detectors have been tested in a mixedneutron-gamma field with high neutron fluence and gamma absorbed dose rates, comparable with the expected values in the HFTM in IFMIF. Since the neutron spectra in all BR2 channels are dominated by the thermal neutron component, the detectors have been surrounded by a cylindrical gadolinium screen to cut the thermal neutron component, in order to get a more representative test for IFMIF conditions. The integrated gamma absorbed dose was about 4 × 1010 Gy and the fast neutron fluence (E > 0.1 MeV) 4 × 1020 n/cm2. The fissionchambers were calibrated in three BR2 channels with different neutron-to-gamma ratio, and the long-term evolution of the signals was studied and compared with theoretical calculations