High Resolution Image Correspondences for Video Post-Production

We present an algorithm for estimating dense image correspondences. Our versatile approach lends itself to various tasks typical for video post-processing, including image morphing, optical flow estimation, stereo rectification, disparity/depth reconstruction, and baseline adjustment. We incorporate recent advances in feature matching, energy minimization, stereo vision, and data clustering into our approach. At the core of our correspondence estimation we use Efficient Belief Propagation for energy minimization. While state-of-the-art algorithms only work on thumbnail-sized images, our novel feature downsampling scheme in combination with a simple, yet efficient data term compression, can cope with high-resolution data. The incorporation of SIFT (Scale-Invariant Feature Transform) features into data term computation further resolves matching ambiguities, making long-range correspondence estimation possible. We detect occluded areas by evaluating the correspondence symmetry, we further apply Geodesic matting to automatically determine plausible values in these regions.

Sharpness Matching in Stereo Images

When stereo images are captured under less than ideal conditions, there may be inconsistencies between the two images in brightness, contrast, blurring, etc. When stereo matching is performed between the images, these variations can greatly reduce the quality of the resulting depth map. In this paper we propose a method for correcting sharpness variations in stereo image pairs which is performed as a pre-processing step to stereo matching. Our method is based on scaling the 2D discrete cosine transform (DCT) coefficients of both images so that the two images have the same amount of energy in each of a set of frequency bands. Experiments show that applying the proposed correction method can greatly improve the disparity map quality when one image in a stereo pair is more blurred than the other.

Bitmap Movement Detection: HDR for Dynamic Scenes

Exposure Fusion and other HDR techniques generate well-exposed images from a bracketed image sequence while reproducing a large dynamic range that far exceeds the dynamic range of a single exposure. Common to all these techniques is the problem that the smallest movements in the captured images generate artefacts (ghosting) that dramatically affect the quality of the final images. This limits the use of HDR and Exposure Fusion techniques because common scenes of interest are usually dynamic. We present a method that adapts Exposure Fusion, as well as standard HDR techniques, to allow for dynamic scene without introducing artefacts. Our method detects clusters of moving pixels within a bracketed exposure sequence with simple binary operations. We show that the proposed technique is able to deal with a large amount of movement in the scene and different movement configurations. The result is a ghost-free and highly detailed exposure fused image at a low computational cost.

A Video Database for the Development of Stereo-3D Post-Production Algorithms

This paper introduces a database of freely available stereo-3D content designed to facilitate research in stereo post-production. It describes the structure and content of the database and provides some details about how the material was gathered. The database includes examples of many of the scenarios characteristic to broadcast footage. Material was gathered at different locations including a studio with controlled lighting and both indoor and outdoor on-location sites with more restricted lighting control. The database also includes video sequences with accompanying 3D audio data recorded in an Ambisonics format. An intended consequence of gathering the material is that the database contains examples of degradations that would be commonly present in real-world scenarios. This paper describes one such artefact caused by uneven exposure in the stereo views, causing saturation in the over-exposed view. An algorithm for the restoration of this artefact is proposed in order to highlight the usefuiness of the database.

Learning Two-Person Interaction Models for Responsive Synthetic Humanoids

Imitation learning is a promising approach for generating life-like behaviors of virtual humans and humanoid robots. So far, however, imitation learning has been mostly restricted to single agent settings where observed motions are adapted to new environment conditions but not to the dynamic behavior of interaction partners. In this paper, we introduce a new imitation learning approach that is based on the simultaneous motion capture of two human interaction partners. From the observed interactions, low-dimensional motion models are extracted and a mapping between these motion models is learned. This interaction model allows the real-time generation of agent behaviors that are responsive to the body movements of an interaction partner. The interaction model can be applied both to the animation of virtual characters as well as to the behavior generation for humanoid robots.

A user supported object tracking framework for interactive video production

We present a user supported tracking framework that combines automatic tracking with extended user input to create error free tracking results that are suitable for interactive video production. The goal of our approach is to keep the necessary user input as small as possible. In our framework, the user can select between different tracking algorithms - existing ones and new ones that are described in this paper. Furthermore, the user can automatically fuse the results of different tracking algorithms with our robust fusion approach. The tracked object can be marked in more than one frame, which can significantly improve the tracking result. After tracking, the user can validate the results in an easy way, thanks to the support of a powerful interpolation technique. The tracking results are iteratively improved until the complete track has been found. After the iterative editing process the tracking result of each object is stored in an interactive video file that can be loaded by our player for interactive videos.

Collision Detection: Broad Phase Adaptation from Multi-Core to Multi-GPU Architecture

We present in this paper several contributions on the collision detection optimization centered on hardware performance. We focus on the broad phase which is the first step of the collision detection process and propose three new ways of parallelization of the well-known Sweep and Prune algorithm. We first developed a multi-core model takes into account the number of available cores. Multi-core architecture enables us to distribute geometric computations with use of multi-threading. Critical writing section and threads idling have been minimized by introducing new data structures for each thread. Programming with directives, like OpenMP, appears to be a good compromise for code portability. We then proposed a new GPU-based algorithm also based on the "Sweep and Prune" that has been adapted to multi-GPU architectures. Our technique is based on a spatial subdivision method used to distribute computations among GPUs. Results show that significant speed-up can be obtained by passing from 1 to 4 GPUs in a large-scale environment.

Estimating Gesture Accuracy in Motion-Based Health Games

This manuscript details a technique for estimating gesture accuracy within the context of motion-based health video games using the MICROSOFT KINECT. We created a physical therapy game that requires players to imitate clinically significant reference gestures. Player performance is represented by the degree of similarity between the performed and reference gestures and is quantified by collecting the Euler angles of the player's gestures, converting them to a three-dimensional vector, and comparing the magnitude between the vectors. Lower difference values represent greater gestural correspondence and therefore greater player performance. A group of thirty-one subjects was tested. Subjects achieved gestural correspondence sufficient to complete the game's objectives while also improving their ability to perform reference gestures accurately.

A Comparative Evaluation of Three Skin Color Detection Approaches

Skin segmentation is a challenging task due to several influences such as unknown lighting conditions, skin colored background, and camera limitations. A lot of skin segmentation approaches were proposed in the past including adaptive (in the sense of updating the skin color online) and non-adaptive approaches. In this paper, we compare three skin segmentation approaches that are promising to work well for hand tracking, which is our main motivation for this work. Hand tracking can widely be used in VR/AR e.g. navigation and object manipulation. The first skin segmentation approach is a well-known non-adaptive approach. It is based on a simple, pre-computed skin color distribution. Methods two and three adaptively estimate the skin color in each frame utilizing clustering algorithms. The second approach uses a hierarchical clustering for a simultaneous image and color space segmentation, while the third approach is a pure color space clustering, but with a more sophisticated clustering approach. For evaluation, we compared the segmentation results of the approaches against a ground truth dataset. To obtain the ground truth dataset, we labeled about 500 images captured under various conditions.

Influence of Comfort on 3D Selection Task Performance in Immersive Desktop Setups

Immersive virtual environments (IVEs) have the potential to afford natural interaction in the three-dimensional (3D) space around a user. However, interaction performance in 3D mid-air is often reduced and depends on a variety of ergonomics factors, the user's endurance, muscular strength, as well as fitness. In particular, in contrast to traditional desktop-based setups, users often cannot rest their arms in a comfortable pose during the interaction. In this article we analyze the impact of comfort on 3D selection tasks in an immersive desktop setup. First, in a pre-study we identified how comfortable or uncomfortable specific interaction positions and poses are for users who are standing upright. Then, we investigated differences in 3D selection task performance when users interact with their hands in a comfortable or uncomfortable body pose, while sitting on a chair in front of a table while the VE was displayed on a headmounted display (HMD). We conducted a Fitts' Law experiment to evaluate selection performance in different poses. The results suggest that users achieve a significantly higher performance in a comfortable pose when they rest their elbow on the table.

On the comparison of VR-responses, as performance measures in prospective memory, with auditory P300 responses in MCI detection

Patients with amnestic mild cognitive impairment are at high risk for developing Alzheimer's disease. Besides episodic memory dysfunction they show deficits in accessing contextual knowledge that further specifies a general spatial navigation task or an executive function (EF) virtual action planning. There has been only one previous work with virtual reality and the use of a virtual action planning supermarket for the diagnosis of mild cognitive impairment. The authors of that study examined the feasibility and the validity of the virtual action planning supermarket (VAP-S) for the diagnosis of patients with mild cognitive impairment (MCI) and found that the VAP-S is a viable tool to assess EF deficits. In our study we employed the in-house platform of virtual action planning museum (VAP-M) and a sample of 25 MCI and 25 controls, in order to investigate deficits in spatial navigation, prospective memory and executive function. In addition, we used the morphology of late components in event-related potential (ERP) responses, as a marker for cognitive dysfunction. The related measurements were fed to a common classification scheme facilitating the direct comparison of both approaches. Our results indicate that both the VAP-M and ERP averages were able to differentiate between healthy elders and patients with amnestic mild cognitive impairment and agree with the findings of the virtual action planning supermarket (VAP-S). The sensitivity (specificity) was 100% (98%) for the VAP-M data and 87%(90%) for the ERP responses. Considering that ERPs have proven to advance the early detection and diagnosis of "presymptomatic AD", the suggested VAP-M platform appears as an appealing alternative.

New Technologies and Concepts for Rehabilitation in the Acute Phase of Stroke: A Collaborative Matrix

The process of developing a successful stroke rehabilitation methodology requires four key components: a good understanding of the pathophysiological mechanisms underlying this brain disease, clear neuroscientific hypotheses to guide therapy, adequate clinical assessments of its efficacy on multiple timescales, and a systematic approach to the application of modern technologies to assist in the everyday work of therapists. Achieving this goal requires collaboration between neuroscientists, technologists and clinicians to develop well-founded systems and clinical protocols that are able to provide quantitatively validated improvements in patient rehabilitation outcomes. In this article we present three new applications of complementary technologies developed in an interdisciplinary matrix for acute-phase upper limb stroke rehabilitation – functional electrical stimulation, arm robot-assisted therapy and virtual reality-based cognitive therapy. We also outline the neuroscientific basis of our approach, present our detailed clinical assessment protocol and provide preliminary results from patient testing of each of the three systems showing their viability for patient use.

Can a novel computerized cognitive screening test provide additional information for early detection of Alzheimer's disease?

BACKGROUND: Virtual reality testing of everyday activities is a novel type of computerized assessment that measures cognitive, executive, and motor performance as a screening tool for early dementia. This study used a virtual reality day-out task (VR-DOT) environment to evaluate its predictive value in patients with mild cognitive impairment (MCI). METHODS: One hundred thirty-four patients with MCI were selected and compared with 75 healthy control subjects. Participants received an initial assessment that included VR-DOT, a neuropsychological evaluation, magnetic resonance imaging (MRI) scan, and event-related potentials (ERPs). After 12 months, participants were assessed again with MRI, ERP, VR-DOT, and neuropsychological tests. RESULTS: At the end of the study, we differentiated two subgroups of patients with MCI according to their clinical evolution from baseline to follow-up: 56 MCI progressors and 78 MCI nonprogressors. VR-DOT performance profiles correlated strongly with existing predictive biomarkers, especially the ERP and MRI biomarkers of cortical thickness. CONCLUSIONS: Compared with ERP, MRI, or neuropsychological tests alone, the VR-DOT could provide additional predictive information in a low-cost, computerized, and noninvasive way.

Glucocorticoids enhance in vivo exposure-based therapy outcome in spider phobia

BACKGROUND: Preclinical and clinical studies indicate that the administration of glucocorticoids may promote fear extinction processes. In particular, it has been shown that glucocorticoids enhance virtual reality based exposure therapy of fear of heights. Here, we investigate whether glucocorticoids enhance the outcome of in vivo exposure-based group therapy of spider phobia. METHODS: In a double blind, block-randomized, placebo-controlled, between-subject study design, 22 patients with specific phobia of spiders were treated with two sessions of in vivo exposure-based group therapy. Cortisol (20 mg) or placebo was orally administered 1 hr before each therapy session. Patients returned for a follow-up assessment one month after therapy. RESULTS: Exposure-based group therapy led to a significant decrease in phobic symptoms as assessed with the Fear of Spiders Questionnaire (FSQ) from pretreatment to immediate posttreatment and to follow-up. The administration of cortisol to exposure therapy resulted in increased salivary cortisol concentrations and a significantly greater reduction in fear of spiders (FSQ) as compared to placebo at follow-up, but not immediately posttreatment. Furthermore, cortisol-treated patients reported significantly less anxiety during standardized exposure to living spiders at follow-up than placebo-treated subjects. Notably, groups did not differ in phobia-unrelated state-anxiety before and after the exposure sessions and at follow-up. CONCLUSIONS: These findings indicate that adding cortisol to in vivo exposure-based group therapy of spider phobia enhances treatment outcome.

Auditive und visuelle Perzeption bei Abseitsentscheidungen im Fussball

Einleitung Die hohe Quote von fehlerhaften Abseitsentscheidungen im Fußball (ca. 26%, Helsen et al., 2006) könnte dadurch erklärt werden, dass Schiedsrichterassistenten zum Zeitpunkt des Passes die Abseitslinie fixieren und den Moment des Passes nur peripher wahr-nehmen (Catteeuw et al., 2009). Diese Annahme wurde geprüft, indem in einem Virtual-Reality-Setting systematisch visuelle (Spielerpositionen) und akustische (Passgeräusch) Informationen manipuliert wurden. Methode In Experiment 1 hatten 28 Teilnehmer die Aufgabe, Abseitsentscheidungen aus der Per-spektive des Schiedsrichterassistenten zu treffen. Dabei wurde das Passgeräusch mani-puliert (Geräusch bei Ballabgabe: 100 ms zu früh, 100 ms zu spät, ohne). Erwartet wurde, dass die Entscheidungsrichtigkeit bei asynchronen Geräuschen sinkt. In Experiment 2 wurde in drei Gruppen die (1) Distanz zum Assistent, (2) Exzentrizität des Passgebers (Winkel zwischen Passgeber und Abseitslinie) oder (3) der zu fixierende Spieler (letzter Verteidiger oder Passgeber) manipuliert. Je Gruppe hatten 20 Teilnehmer die Aufgabe, den Moment des Passes per Knopfdruck anzugeben und Abseitsentscheidungen zu tref-fen. Zu fixierende Orte wurden instruiert und durch Eyetracking geprüft. Prädiziert wurde, dass (1) große Exzentrizitäten und (2) große Entfernungen des Passgebers die Präzision der Passwahrnehmung und die Entscheidungsrichtigkeit senken und (3) der Passmo-ment bei Fixation des Passgebers präziser erkannt wird als bei Fixation des letzten Ver-teidigers. Ergebnisse Die bisher vorliegenden Ergebnisse aus Experiment 1 zeigen, dass die akustische Wahrnehmung des Passes die Entscheidungsrichtigkeit beeinflusst, F(2, 54)= 7.44, p = .01, ηp2 = .22, indem verzögerte Ballgeräusche die Zahl der „Flag-Errors“ (fälschlicher-weise auf Abseits entschieden) erhöht und die der „Non-Flag-Errors“ (Abseits nicht er-kannt) senkt. Experiment 2 wird zeigen, welche Rolle hierbei periphere visuelle Informa-tionen spielen. Diskussion Die vermutete Rückführung von fehlerhaften Abseitsentscheidungen auf periphere (vi-suelle) Wahrnehmungsaspekte konnte bestätigt werden. Der relative Einfluss auditiver und visueller Informationen auf die Fehlerquote gilt es weiter zu untersuchen. Literatur Catteeuw, P., Helsen, W., Gilis, B., Van Roie, E., & Wagemans, J. (2009). Visual scan patterns and decision-making skills of expert assistant referees in offside situations. Journal of Sport & Exercise Psychol-ogy, 31, 786-797. Helsen, W., Gilis, B., & Weston, M. (2006). Errors in judging “offside” in association football: Test of the optical error versus the perceptual flash-lag hypothesis. Journal of Sports Sciences, 24, 521–528.