A Video Database for the Development of Stereo-3D Post-Production Algorithms

This paper introduces a database of freely available stereo-3D content designed to facilitate research in stereo post-production. It describes the structure and content of the database and provides some details about how the material was gathered. The database includes examples of many of the scenarios characteristic to broadcast footage. Material was gathered at different locations including a studio with controlled lighting and both indoor and outdoor on-location sites with more restricted lighting control. The database also includes video sequences with accompanying 3D audio data recorded in an Ambisonics format. An intended consequence of gathering the material is that the database contains examples of degradations that would be commonly present in real-world scenarios. This paper describes one such artefact caused by uneven exposure in the stereo views, causing saturation in the over-exposed view. An algorithm for the restoration of this artefact is proposed in order to highlight the usefuiness of the database.

Learning Two-Person Interaction Models for Responsive Synthetic Humanoids

Imitation learning is a promising approach for generating life-like behaviors of virtual humans and humanoid robots. So far, however, imitation learning has been mostly restricted to single agent settings where observed motions are adapted to new environment conditions but not to the dynamic behavior of interaction partners. In this paper, we introduce a new imitation learning approach that is based on the simultaneous motion capture of two human interaction partners. From the observed interactions, low-dimensional motion models are extracted and a mapping between these motion models is learned. This interaction model allows the real-time generation of agent behaviors that are responsive to the body movements of an interaction partner. The interaction model can be applied both to the animation of virtual characters as well as to the behavior generation for humanoid robots.

Collision Detection: Broad Phase Adaptation from Multi-Core to Multi-GPU Architecture

We present in this paper several contributions on the collision detection optimization centered on hardware performance. We focus on the broad phase which is the first step of the collision detection process and propose three new ways of parallelization of the well-known Sweep and Prune algorithm. We first developed a multi-core model takes into account the number of available cores. Multi-core architecture enables us to distribute geometric computations with use of multi-threading. Critical writing section and threads idling have been minimized by introducing new data structures for each thread. Programming with directives, like OpenMP, appears to be a good compromise for code portability. We then proposed a new GPU-based algorithm also based on the "Sweep and Prune" that has been adapted to multi-GPU architectures. Our technique is based on a spatial subdivision method used to distribute computations among GPUs. Results show that significant speed-up can be obtained by passing from 1 to 4 GPUs in a large-scale environment.

Comparison of 2D and 3D GUI Widgets for Stereoscopic Multitouch Setups

Recent developments in the area of interactive entertainment have suggested to combine stereoscopic visualization with multi-touch displays, which has the potential to open up new vistas for natural interaction with interactive three-dimensional (3D) applications. However, the question arises how the user interfaces for system control in such 3D setups should be designed in order to provide an effective user experience. In this article we introduce 3D GUI widgets for interaction with stereoscopic touch displays. The design of the widgets was inspired to skeuomorphism and affordances in such a way that the user should be able to operate the virtual objects in the same way as their real-world equivalents. We evaluate the developed widgets and compared them with their 2D counterparts in the scope of an example application in order to analyze the usability of and user behavior with the widgets. The results reveal differences in user behavior with and without stereoscopic display during touch interaction, and show that the developed 2D as well as 3D GUI widgets can be used effectively in different applications.

Estimating Gesture Accuracy in Motion-Based Health Games

This manuscript details a technique for estimating gesture accuracy within the context of motion-based health video games using the MICROSOFT KINECT. We created a physical therapy game that requires players to imitate clinically significant reference gestures. Player performance is represented by the degree of similarity between the performed and reference gestures and is quantified by collecting the Euler angles of the player's gestures, converting them to a three-dimensional vector, and comparing the magnitude between the vectors. Lower difference values represent greater gestural correspondence and therefore greater player performance. A group of thirty-one subjects was tested. Subjects achieved gestural correspondence sufficient to complete the game's objectives while also improving their ability to perform reference gestures accurately.

A Comparative Evaluation of Three Skin Color Detection Approaches

Skin segmentation is a challenging task due to several influences such as unknown lighting conditions, skin colored background, and camera limitations. A lot of skin segmentation approaches were proposed in the past including adaptive (in the sense of updating the skin color online) and non-adaptive approaches. In this paper, we compare three skin segmentation approaches that are promising to work well for hand tracking, which is our main motivation for this work. Hand tracking can widely be used in VR/AR e.g. navigation and object manipulation. The first skin segmentation approach is a well-known non-adaptive approach. It is based on a simple, pre-computed skin color distribution. Methods two and three adaptively estimate the skin color in each frame utilizing clustering algorithms. The second approach uses a hierarchical clustering for a simultaneous image and color space segmentation, while the third approach is a pure color space clustering, but with a more sophisticated clustering approach. For evaluation, we compared the segmentation results of the approaches against a ground truth dataset. To obtain the ground truth dataset, we labeled about 500 images captured under various conditions.

Influence of Comfort on 3D Selection Task Performance in Immersive Desktop Setups

Immersive virtual environments (IVEs) have the potential to afford natural interaction in the three-dimensional (3D) space around a user. However, interaction performance in 3D mid-air is often reduced and depends on a variety of ergonomics factors, the user's endurance, muscular strength, as well as fitness. In particular, in contrast to traditional desktop-based setups, users often cannot rest their arms in a comfortable pose during the interaction. In this article we analyze the impact of comfort on 3D selection tasks in an immersive desktop setup. First, in a pre-study we identified how comfortable or uncomfortable specific interaction positions and poses are for users who are standing upright. Then, we investigated differences in 3D selection task performance when users interact with their hands in a comfortable or uncomfortable body pose, while sitting on a chair in front of a table while the VE was displayed on a headmounted display (HMD). We conducted a Fitts' Law experiment to evaluate selection performance in different poses. The results suggest that users achieve a significantly higher performance in a comfortable pose when they rest their elbow on the table.

Advanced luminance control and black offset correction for multi-projector display systems

In order to display a homogeneous image using multiple projectors, differences in the projected intensities must be compensated. In this paper, we present novel approaches to combine and extend existing techniques for edge blending and luminance harmonization to achieve a detailed luminance control. Furthermore, we apply techniques for improving the contrast ratio of multi-segmented displays also to the black offset correction. We also present a simple scheme to involve the displayed context in the correction process to dynamically improve the contrast in brighter images. In addition, we present a metric to evaluate the different methods and their influence on the visual quality.

Learning in Virtual Worlds: Research and Applications

Three-dimensional (3D) immersive virtual worlds have been touted as being capable of facilitating highly interactive, engaging, multimodal learning experiences. Much of the evidence gathered to support these claims has been anecdotal but the potential that these environments hold to solve traditional problems in online and technology-mediated education—primarily learner isolation and student disengagement—has resulted in considerable investments in virtual world platforms like Second Life, OpenSimulator, and Open Wonderland by both professors and institutions. To justify this ongoing and sustained investment, institutions and proponents of simulated learning environments must assemble a robust body of evidence that illustrates the most effective use of this powerful learning tool. In this authoritative collection, a team of international experts outline the emerging trends and developments in the use of 3D virtual worlds for teaching and learning. They explore aspects of learner interaction with virtual worlds, such as user wayfinding in Second Life, communication modes and perceived presence, and accessibility issues for elderly or disabled learners. They also examine advanced technologies that hold potential for the enhancement of learner immersion and discuss best practices in the design and implementation of virtual world-based learning interventions and tasks. By evaluating and documenting different methods, approaches, and strategies, the contributors to Learning in Virtual Worlds offer important information and insight to both scholars and practitioners in the field. AU Press is an open access publisher and the book is available for free in PDF format as well as for purchase on our website: http://bit.ly/1W4yTRA

Tecnología de registro y reproducción sonora para entornos de realidad virtual

Este proyecto de fin de carrera tiene como objetivo obtener una visión detallada de los sistemas y tecnologías de grabación y reproducción utilizadas para aplicaciones de audio 3D y entornos de realidad virtual, analizando las diferentes alternativas existentes, su funcionamiento, características, detalles técnicos y sus ámbitos de aplicación. Como punto de partida se estudiará la teoría psicoacústica y la localización de fuentes sonoras en el espacio, base para el estudio de los sistemas de audio 3D. Se estudiará tanto la espacialización sonora en un espacio real y la espacialización virtual (simulación mediante procesado de información de la localización de fuentes sonoras), en los que intervienen algunos fenómenos acústicos y psicoacústicos como ITD, o diferencia de tiempo que existe entre una señal acústica que llega a los pabellones auditivos, la ILD, o diferencia de intensidad o amplitud que hay entre la señal que llega a los pabellones auditivos y la localización espacial mediante otra serie de mecanismos biaurales. Tras una visión general de la teoría psicoacústica y la espacialización sonora, se analizarán con detalle los elementos de grabación y reproducción existentes para audio 3D. Concretamente, a lo largo del proyecto se profundizará en el funcionamiento del sistema estéreo, caracterizado por el posicionamiento sonoro mediante la utilización de dos canales; del sistema biaural, caracterizado por reconstruir campos sonoros mediante el uso de las HRTF; de los sistemas multicanal, detallando gran parte de las alternativas y configuraciones existentes; del sistema Ambiophonics, caracterizado por implementar filtros de cruce; del sistema Ambisonics, y sus diferentes formatos y técnicas de codificación y decodificación; y del sistema Wavefield Synthesis, caracterizado por recrear ambientes sonoros en grandes espacios. ABSTRACT This project aims to get a detailed view of recording and reproducing systems and technologies used to 3D audio applications and virtual reality environments, analyzing the different alternatives available, their functioning, features, technical details and their different scopes of applications. As a starting point, will be studied the psychoacoustic theory and the localization of sound sources in space, basis for the 3D audio study. Will be studied both the spacialization of sound sources in real space as virtual spatialization of sound sources (simulation by information processing of localization of sound sources), in which involves some acoustic and psychoacoustic phenomena like ITD (or the Interaural time difference), the ILD, (or the Interaural Level Difference) and spatial localization by another set of binaural mechanisms. After a general overview of the psychoacoustics theory and the sound spatialization, will be analyzed in detail existing methods of recording and reproducing for 3D audio. Specifically, during the project will analyze the characteristics of the stereo systems, characterized by sound positioning using two channels; the binaural systems, characterized by reconstructing sound fields by using the HRTF; the multichannel systems, detailing many of the existing alternatives and configurations; the Ambiophonics system, which is characterized by implementing crosstalk elimination techniques; the Ambiosonics system, and its various formats and encoding and decoding techniques; and the Wavefield Synthesis system, characterized by recreate soundscapes in large spaces.

Evaluación y tratamiento basados en realidad virtual de la negligencia espacial unilateral: análisis del estado de la cuestión y demostración de enfoques alternativos

En el siguiente trabajo se presenta en primer lugar de forma detallada la enfermedad denominada negligencia espacial unilateral (síntomas, tipos, causas, evaluación y tratamientos) para proporcionar una mejor comprensión del principal objetivo del estudio, que es el análisis de las soluciones virtuales, existentes en la literatura, aplicadas al tratamiento de esta enfermedad, incluyéndose una amplia descripción de cada estudio encontrado sobre el tema. A continuación, se han realizado tres implementaciones en realidad virtual de tres técnicas clásicas de rehabilitación llevadas a cabo en un entorno virtual, que son la estimulación optocinética, eye patching, y adaptación prismática y se ha desarrollado una aplicación 3D para evaluar el grado y tipo de negligencia sufrida por los pacientes. Que de forma conjunta, constituyen un primer paso hacia un enfoque alternativo para el tratamiento de la enfermedad, más personalizado y eficaz. Por último, en las conclusiones, se analizan las principales ventajas y desventajas encontradas en el uso de estas tecnologías aplicadas a la enfermedad y los trabajos futuros que pueden derivar de este trabajo.---ABSTRACT---The following work starts by presenting in detail a disease called unilateral spatial neglect (symptoms, types, causes, assessment and treatment) to provide the background for this study's main objective, which is the analysis of the virtual solutions existing in the literature for the treatment of this disease. The document includes an extensive description of the previous work found in this topic. Afterwards, three implementations of three classical rehabilitation techniques were performed in virtual reality: optokinetic stimulation, eye patching and prism adaptation, as a proof-of-concept, and a 3D application was implemented to assess the degree and type of negligence suffered by patients. Altogether, they constitute a first step towards an alternative approach for the treatment of disease, more personalized and effective. Finally, the conclusions analyze the main advantages and disadvantages encountered in the use of these technologies when applied to this disease and suggest future work.

Restauración virtual del patrimonio sonoro. Aplicación al antiguo rito hispánico

Una de las aplicaciones más interesantes de las tecnologías de Realidad Acústica Virtual es la posibilidad de recuperar el patrimonio cultural del sonido de los recintos históricos que se han perdido o cuyas características se han modificado a lo largo de los siglos. En este trabajo, las tecnologías de realidad acústica virtual se utilizan para tratar de reconstruir virtualmente el sonido producido en las actividades litúrgicas del antiguo rito hispánico. Bajo este nombre se conoce a la liturgia que celebraban los cristianos de la península ibérica hasta su prohibición, en favor del culto romano, a mediados del siglo XI. El rito hispánico -también conocido como rito visigótico-mozárabe- es una de las manifestaciones culturales más interesantes de la Alta Edad Media en occidente. Se trata de un patrimonio cultural desaparecido, tanto desde el punto de vista de las señales sonoras que lo integraban -ya que la mayoría de las melodías que conformaban el rito se han perdido- como desde el punto de vista de los espacios en los que se desarrollaba, debido a que las iglesias conservadas de la época han experimentado modificaciones a lo largo de los siglos que alteran sus condiciones acústicas con respecto a las que tenían en el periodo de vigencia de esta liturgia. Para llevar a cabo este proyecto, se han realizado modelos acústicos digitales de un grupo representativo de iglesias prerrománicas de la Península Ibérica en su estado primitivo. Se ha procurado que las iglesias seleccionadas representen la variedad de comportamientos acústicos esperables en este tipo de edificios. Con este objetivo, se han elegido cinco iglesias prerrománicas que presentan diferencias sustanciales en los parámetros que, a priori, van a influir en mayor manera a su comportamiento acústico: el volumen del recinto, la forma de la planta y el tipo de cubierta. El proceso de creación de los modelos acústicos digitales de las iglesias se ha dividido en dos fases: en la primera se han creado modelos de los edificios en su estado actual, que se han validado a partir de los datos obtenidos en mediciones acústicas realizadas in situ; a partir de los modelos validados de las iglesias en su estado actual, en la segunda fase se han generado los modelos acústicos digitales correspondientes al estado primitivo de las mismas, modificando los modelos anteriores de acuerdo con las hipótesis de reconstrucción propuestas en las investigaciones arqueológicas más recientes. Se han realizado grabaciones en cámara anecoica de una serie de piezas del repertorio original del canto mozárabe. Las grabaciones se han llevado a cabo con un array esférico compuesto por 32 micrófonos, con objeto de obtener información sobre la directividad de la emisión sonora de los cantantes, que se aplicará en las auralizaciones. Finalmente, se han realizado diversas auralizaciones, teniendo en cuenta diferentes configuraciones litúrgicas que eran usuales en este rito. ABSTRACT One of the most interesting applications of the Acoustic Virtual Reality technologies is the possibility to recover the cultural heritage of the sound of the historical sites that have been lost or whose characteristics have been modified through time. In this work, Acoustic Virtual Reality technologies are used to try to reconstruct virtually the sound produced in the liturgical activities of the Hispanic Rite. This is the name given to the liturgy celebrated by Christians of the Iberian Peninsula prior to the introduction of the Roman cult in the mid-eleventh century. The Hispanic Rite (also known as Visigothic or Mozarabic rite) is one of the most interesting cultural manifestations of the Middle Ages. It is a lost cultural heritage, both from the point of view of the sound signals that were used since the majority of the melodies that conformed the rite have been lost, and from the point of view of the spaces in which this liturgy was celebrated, because the churches preserved from that era have changed over the centuries altering its acoustic conditions respect to which they were in the period in which this liturgy was in effect. To carry out this project, acoustic models of a representative group of the pre-Romanesque churches in the Iberian Peninsula have been made in his primitive state. To select the sample of buildings to study, efforts have been made to ensure that the selected churches were representative of the range of expected acoustic behaviors in this type of buildings. Five churches have been selected, showing differences in the parameters that have the greatest influence on their acoustic behavior: the enclosure volume, the shape of the floor plan and the type of roof. The process of creating digital acoustic models of the churches has been divided into two phases. In the first phase acoustic models of the churches in its current state have been created. These models have been validated with the data obtained from in situ acoustic measurements. From the validated models of churches in its current state, in the second phase, changes in the acoustic models have been performed in order to represent the primitive state of the churches, according to the historical reconstruction hypothesis that have been proposed in the most recent archaeological investigations. Anechoic recordings of a series of pieces of the original Mozarabic Chant repertoire have been recorded. The recordings were made with a spherical array composed of 32 microphones, in order to obtain information on the directivity of the sound emission of the singers. These directivity data will be used to simulate the directional radiation of the sound sources in the auralizations. Finally, auralizations were produced corresponding to different liturgical configurations which were common in this rite.

Modelling ecological cognitive rehabilitation therapies for building virtual environments in brain injury

Brain Injury (BI) has become one of the most common causes of neurological disability in developed countries. Cognitive disorders result in a loss of independence and patients? quality of life. Cognitive rehabilitation aims to promote patients? skills to achieve their highest degree of personal autonomy. New technologies such as virtual reality or interactive video allow developing rehabilitation therapies based on reproducible Activities of Daily Living (ADLs), increasing the ecological validity of the therapy. However, the lack of frameworks to formalize and represent the definition of this kind of therapies can be a barrier for widespread use of interactive virtual environments in clinical routine.

Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome

This paper presents the use of immersive virtual reality systems in the educational intervention with Asperger students. The starting points of this study are features of these students' cognitive style that requires an explicit teaching style supported by visual aids and highly structured environments. The proposed immersive virtual reality system, not only to assess the student's behavior and progress, but also is able to adapt itself to the student's specific needs. Additionally, the immersive reality system is equipped with sensors that can determine certain behaviors of the students. This paper determines the possible inclusion of immersive virtual reality as a support tool and learning strategy in these particular students' intervention. With this objective two task protocols have been defined with which the behavior and interaction situations performed by participant students are recorded. The conclusions from this study talks in favor of the inclusion of these virtual immersive environments as a support tool in the educational intervention of Asperger syndrome students as their social competences and executive functions have improved.