A multimodal tablet-based interface for designing and reviewing 3D engineering models

The ability to view and interact with 3D models has been happening for a long time. However, vision-based 3D modeling has only seen limited success in applications, as it faces many technical challenges. Hand-held mobile devices have changed the way we interact with virtual reality environments. Their high mobility and technical features, such as inertial sensors, cameras and fast processors, are especially attractive for advancing the state of the art in virtual reality systems. Also, their ubiquity and fast Internet connection open a path to distributed and collaborative development. However, such path has not been fully explored in many domains. VR systems for real world engineering contexts are still difficult to use, especially when geographically dispersed engineering teams need to collaboratively visualize and review 3D CAD models. Another challenge is the ability to rendering these environments at the required interactive rates and with high fidelity. In this document it is presented a virtual reality system mobile for visualization, navigation and reviewing large scale 3D CAD models, held under the CEDAR (Collaborative Engineering Design and Review) project. It’s focused on interaction using different navigation modes. The system uses the mobile device's inertial sensors and camera to allow users to navigate through large scale models. IT professionals, architects, civil engineers and oil industry experts were involved in a qualitative assessment of the CEDAR system, in the form of direct user interaction with the prototypes and audio-recorded interviews about the prototypes. The lessons learned are valuable and are presented on this document. Subsequently it was prepared a quantitative study on the different navigation modes to analyze the best mode to use it in a given situation.

Tecnología de registro y reproducción sonora para entornos de realidad virtual

Este proyecto de fin de carrera tiene como objetivo obtener una visión detallada de los sistemas y tecnologías de grabación y reproducción utilizadas para aplicaciones de audio 3D y entornos de realidad virtual, analizando las diferentes alternativas existentes, su funcionamiento, características, detalles técnicos y sus ámbitos de aplicación. Como punto de partida se estudiará la teoría psicoacústica y la localización de fuentes sonoras en el espacio, base para el estudio de los sistemas de audio 3D. Se estudiará tanto la espacialización sonora en un espacio real y la espacialización virtual (simulación mediante procesado de información de la localización de fuentes sonoras), en los que intervienen algunos fenómenos acústicos y psicoacústicos como ITD, o diferencia de tiempo que existe entre una señal acústica que llega a los pabellones auditivos, la ILD, o diferencia de intensidad o amplitud que hay entre la señal que llega a los pabellones auditivos y la localización espacial mediante otra serie de mecanismos biaurales. Tras una visión general de la teoría psicoacústica y la espacialización sonora, se analizarán con detalle los elementos de grabación y reproducción existentes para audio 3D. Concretamente, a lo largo del proyecto se profundizará en el funcionamiento del sistema estéreo, caracterizado por el posicionamiento sonoro mediante la utilización de dos canales; del sistema biaural, caracterizado por reconstruir campos sonoros mediante el uso de las HRTF; de los sistemas multicanal, detallando gran parte de las alternativas y configuraciones existentes; del sistema Ambiophonics, caracterizado por implementar filtros de cruce; del sistema Ambisonics, y sus diferentes formatos y técnicas de codificación y decodificación; y del sistema Wavefield Synthesis, caracterizado por recrear ambientes sonoros en grandes espacios. ABSTRACT This project aims to get a detailed view of recording and reproducing systems and technologies used to 3D audio applications and virtual reality environments, analyzing the different alternatives available, their functioning, features, technical details and their different scopes of applications. As a starting point, will be studied the psychoacoustic theory and the localization of sound sources in space, basis for the 3D audio study. Will be studied both the spacialization of sound sources in real space as virtual spatialization of sound sources (simulation by information processing of localization of sound sources), in which involves some acoustic and psychoacoustic phenomena like ITD (or the Interaural time difference), the ILD, (or the Interaural Level Difference) and spatial localization by another set of binaural mechanisms. After a general overview of the psychoacoustics theory and the sound spatialization, will be analyzed in detail existing methods of recording and reproducing for 3D audio. Specifically, during the project will analyze the characteristics of the stereo systems, characterized by sound positioning using two channels; the binaural systems, characterized by reconstructing sound fields by using the HRTF; the multichannel systems, detailing many of the existing alternatives and configurations; the Ambiophonics system, which is characterized by implementing crosstalk elimination techniques; the Ambiosonics system, and its various formats and encoding and decoding techniques; and the Wavefield Synthesis system, characterized by recreate soundscapes in large spaces.

Evaluación y tratamiento basados en realidad virtual de la negligencia espacial unilateral: análisis del estado de la cuestión y demostración de enfoques alternativos

En el siguiente trabajo se presenta en primer lugar de forma detallada la enfermedad denominada negligencia espacial unilateral (síntomas, tipos, causas, evaluación y tratamientos) para proporcionar una mejor comprensión del principal objetivo del estudio, que es el análisis de las soluciones virtuales, existentes en la literatura, aplicadas al tratamiento de esta enfermedad, incluyéndose una amplia descripción de cada estudio encontrado sobre el tema. A continuación, se han realizado tres implementaciones en realidad virtual de tres técnicas clásicas de rehabilitación llevadas a cabo en un entorno virtual, que son la estimulación optocinética, eye patching, y adaptación prismática y se ha desarrollado una aplicación 3D para evaluar el grado y tipo de negligencia sufrida por los pacientes. Que de forma conjunta, constituyen un primer paso hacia un enfoque alternativo para el tratamiento de la enfermedad, más personalizado y eficaz. Por último, en las conclusiones, se analizan las principales ventajas y desventajas encontradas en el uso de estas tecnologías aplicadas a la enfermedad y los trabajos futuros que pueden derivar de este trabajo.---ABSTRACT---The following work starts by presenting in detail a disease called unilateral spatial neglect (symptoms, types, causes, assessment and treatment) to provide the background for this study's main objective, which is the analysis of the virtual solutions existing in the literature for the treatment of this disease. The document includes an extensive description of the previous work found in this topic. Afterwards, three implementations of three classical rehabilitation techniques were performed in virtual reality: optokinetic stimulation, eye patching and prism adaptation, as a proof-of-concept, and a 3D application was implemented to assess the degree and type of negligence suffered by patients. Altogether, they constitute a first step towards an alternative approach for the treatment of disease, more personalized and effective. Finally, the conclusions analyze the main advantages and disadvantages encountered in the use of these technologies when applied to this disease and suggest future work.

Restauración virtual del patrimonio sonoro. Aplicación al antiguo rito hispánico

Una de las aplicaciones más interesantes de las tecnologías de Realidad Acústica Virtual es la posibilidad de recuperar el patrimonio cultural del sonido de los recintos históricos que se han perdido o cuyas características se han modificado a lo largo de los siglos. En este trabajo, las tecnologías de realidad acústica virtual se utilizan para tratar de reconstruir virtualmente el sonido producido en las actividades litúrgicas del antiguo rito hispánico. Bajo este nombre se conoce a la liturgia que celebraban los cristianos de la península ibérica hasta su prohibición, en favor del culto romano, a mediados del siglo XI. El rito hispánico -también conocido como rito visigótico-mozárabe- es una de las manifestaciones culturales más interesantes de la Alta Edad Media en occidente. Se trata de un patrimonio cultural desaparecido, tanto desde el punto de vista de las señales sonoras que lo integraban -ya que la mayoría de las melodías que conformaban el rito se han perdido- como desde el punto de vista de los espacios en los que se desarrollaba, debido a que las iglesias conservadas de la época han experimentado modificaciones a lo largo de los siglos que alteran sus condiciones acústicas con respecto a las que tenían en el periodo de vigencia de esta liturgia. Para llevar a cabo este proyecto, se han realizado modelos acústicos digitales de un grupo representativo de iglesias prerrománicas de la Península Ibérica en su estado primitivo. Se ha procurado que las iglesias seleccionadas representen la variedad de comportamientos acústicos esperables en este tipo de edificios. Con este objetivo, se han elegido cinco iglesias prerrománicas que presentan diferencias sustanciales en los parámetros que, a priori, van a influir en mayor manera a su comportamiento acústico: el volumen del recinto, la forma de la planta y el tipo de cubierta. El proceso de creación de los modelos acústicos digitales de las iglesias se ha dividido en dos fases: en la primera se han creado modelos de los edificios en su estado actual, que se han validado a partir de los datos obtenidos en mediciones acústicas realizadas in situ; a partir de los modelos validados de las iglesias en su estado actual, en la segunda fase se han generado los modelos acústicos digitales correspondientes al estado primitivo de las mismas, modificando los modelos anteriores de acuerdo con las hipótesis de reconstrucción propuestas en las investigaciones arqueológicas más recientes. Se han realizado grabaciones en cámara anecoica de una serie de piezas del repertorio original del canto mozárabe. Las grabaciones se han llevado a cabo con un array esférico compuesto por 32 micrófonos, con objeto de obtener información sobre la directividad de la emisión sonora de los cantantes, que se aplicará en las auralizaciones. Finalmente, se han realizado diversas auralizaciones, teniendo en cuenta diferentes configuraciones litúrgicas que eran usuales en este rito. ABSTRACT One of the most interesting applications of the Acoustic Virtual Reality technologies is the possibility to recover the cultural heritage of the sound of the historical sites that have been lost or whose characteristics have been modified through time. In this work, Acoustic Virtual Reality technologies are used to try to reconstruct virtually the sound produced in the liturgical activities of the Hispanic Rite. This is the name given to the liturgy celebrated by Christians of the Iberian Peninsula prior to the introduction of the Roman cult in the mid-eleventh century. The Hispanic Rite (also known as Visigothic or Mozarabic rite) is one of the most interesting cultural manifestations of the Middle Ages. It is a lost cultural heritage, both from the point of view of the sound signals that were used since the majority of the melodies that conformed the rite have been lost, and from the point of view of the spaces in which this liturgy was celebrated, because the churches preserved from that era have changed over the centuries altering its acoustic conditions respect to which they were in the period in which this liturgy was in effect. To carry out this project, acoustic models of a representative group of the pre-Romanesque churches in the Iberian Peninsula have been made in his primitive state. To select the sample of buildings to study, efforts have been made to ensure that the selected churches were representative of the range of expected acoustic behaviors in this type of buildings. Five churches have been selected, showing differences in the parameters that have the greatest influence on their acoustic behavior: the enclosure volume, the shape of the floor plan and the type of roof. The process of creating digital acoustic models of the churches has been divided into two phases. In the first phase acoustic models of the churches in its current state have been created. These models have been validated with the data obtained from in situ acoustic measurements. From the validated models of churches in its current state, in the second phase, changes in the acoustic models have been performed in order to represent the primitive state of the churches, according to the historical reconstruction hypothesis that have been proposed in the most recent archaeological investigations. Anechoic recordings of a series of pieces of the original Mozarabic Chant repertoire have been recorded. The recordings were made with a spherical array composed of 32 microphones, in order to obtain information on the directivity of the sound emission of the singers. These directivity data will be used to simulate the directional radiation of the sound sources in the auralizations. Finally, auralizations were produced corresponding to different liturgical configurations which were common in this rite.

Modelling ecological cognitive rehabilitation therapies for building virtual environments in brain injury

Brain Injury (BI) has become one of the most common causes of neurological disability in developed countries. Cognitive disorders result in a loss of independence and patients? quality of life. Cognitive rehabilitation aims to promote patients? skills to achieve their highest degree of personal autonomy. New technologies such as virtual reality or interactive video allow developing rehabilitation therapies based on reproducible Activities of Daily Living (ADLs), increasing the ecological validity of the therapy. However, the lack of frameworks to formalize and represent the definition of this kind of therapies can be a barrier for widespread use of interactive virtual environments in clinical routine.

Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome

This paper presents the use of immersive virtual reality systems in the educational intervention with Asperger students. The starting points of this study are features of these students' cognitive style that requires an explicit teaching style supported by visual aids and highly structured environments. The proposed immersive virtual reality system, not only to assess the student's behavior and progress, but also is able to adapt itself to the student's specific needs. Additionally, the immersive reality system is equipped with sensors that can determine certain behaviors of the students. This paper determines the possible inclusion of immersive virtual reality as a support tool and learning strategy in these particular students' intervention. With this objective two task protocols have been defined with which the behavior and interaction situations performed by participant students are recorded. The conclusions from this study talks in favor of the inclusion of these virtual immersive environments as a support tool in the educational intervention of Asperger syndrome students as their social competences and executive functions have improved.

A multimodal tablet-based interface for designing and reviewing 3D engineering models

The ability to view and interact with 3D models has been happening for a long time. However, vision-based 3D modeling has only seen limited success in applications, as it faces many technical challenges. Hand-held mobile devices have changed the way we interact with virtual reality environments. Their high mobility and technical features, such as inertial sensors, cameras and fast processors, are especially attractive for advancing the state of the art in virtual reality systems. Also, their ubiquity and fast Internet connection open a path to distributed and collaborative development. However, such path has not been fully explored in many domains. VR systems for real world engineering contexts are still difficult to use, especially when geographically dispersed engineering teams need to collaboratively visualize and review 3D CAD models. Another challenge is the ability to rendering these environments at the required interactive rates and with high fidelity. In this document it is presented a virtual reality system mobile for visualization, navigation and reviewing large scale 3D CAD models, held under the CEDAR (Collaborative Engineering Design and Review) project. It’s focused on interaction using different navigation modes. The system uses the mobile device's inertial sensors and camera to allow users to navigate through large scale models. IT professionals, architects, civil engineers and oil industry experts were involved in a qualitative assessment of the CEDAR system, in the form of direct user interaction with the prototypes and audio-recorded interviews about the prototypes. The lessons learned are valuable and are presented on this document. Subsequently it was prepared a quantitative study on the different navigation modes to analyze the best mode to use it in a given situation.

Virtual training environments to improve train driver's crisis decision making

This paper reports on a current research project in which virtual reality simulators are being investigated as a means of simulating hazardous Rail work conditions in order to allow train drivers to practice decision-making under stress. When working under high stress conditions train drivers need to move beyond procedural responses into a response activated through their own problem-solving and decision-making skills. This study focuses on the use of stress inoculation training which aims to build driver’s confidence in the use of new decision-making skills by being repeatedly required to respond to hazardous driving conditions. In particular, the study makes use of a train cab driving simulator to reproduce potentially stress inducing real-world scenarios. Initial pilot research has been undertaken in which drivers have experienced the training simulation and subsequently completed surveys on the level of immersion experienced. Concurrently drivers have also participated in a velocity perception experiment designed to objectively measure the fidelity of the virtual training environment. Baseline data, against which decision-making skills post training will be measured, is being gathered via cognitive task analysis designed to identify primary decision requirements for specific rail events. While considerable efforts have been invested in improving Virtual Reality technology, little is known about how to best use this technology for training personnel to respond to workplace conditions in the Rail Industry. To enable the best use of simulators for training in the Rail context the project aims to identify those factors within virtual reality that support required learning outcomes and use this information to design training simulations that reliably and safely train staff in required workplace accident response skills.

Knowledge-based Virtual Reconstruction of Museum Artifacts

Within the framework of heritage preservation, 3D scanning and modeling for heritage documentation has increased significantly in recent years, mainly due to the evolution of laser and image-based techniques, modeling software, powerful computers and virtual reality. 3D laser acquisition constitutes a real development opportunity for 3D modeling based previously on theoretical data. The representation of the object information rely on the knowledge of its historic and theoretical frame to reconstitute a posteriori its previous states. This project proposes an approach dealing with data extraction based on architectural knowledge and Laser statement informing measurements, the whole leading to 3D reconstruction. The experimented Khmer objects are exposed at Guimet museum in Paris. The purpose of this digital modeling meets the need of exploitable models for simulation projects, prototyping, exhibitions, promoting cultural tourism and particularly for archiving against any likely disaster and as an aided tool for the formulation of virtual museum concept.