Haptics and virtual reality in modern manufacturing

Product assembly is one of the most studied processes in modern manufacturing. In recent years a number of computer-based virtual reality systems have been proposed, developed and adopted by the manufacturing industries. Such systems have major advantages over conventional training practices for product assembly. Significant cost savings can be realized due to shorter training-scenario development times and reuse of existing engineering math models. In addition, the time span from the product design to full production can be shortened due to non-reliance on actual components and subsystems for training. Such training systems are effective if the knowledge required to be transferred is just process sequence such as assembly sequence. However, knowledge transfer for procedural and cognitive learning as well as skills development is very limited, due to the lack of user interactivity and immersion.

This talk will focus on a research technology platform where haptics and virtual reality are integrated to create an effective environment for production assembly operators’ training. In this system virtual reality provides the grounds for realistic visualization, as well as immersion, whereas haptics enforces physical constraints within the virtual world generating the feelings of realistic interaction, making it accessible for formal learning and better understanding during task performance.

The developed research technology platform imitates real physical training scenarios by providing comprehensive user interaction, constrained within the physical limitations of the real world. Through the utilization of a haptics device, providing realistic force feedback, users are able to engage in product assembly training with a stronger sense of ‘reality’.

Realistic autonomous fish for virtual reality

We create realistic autonomous fish for Virtual Reality systems. The fish are realistic in appearance, movement and behaviour: the swimming behaviour being non-scripted, within real time rendering.The form of the fish is procedurally created. The size and shape of the form are controlled by a number of variables which are stored in a simple ASCII file. This allows efficient creation of different fish at run time.The behaviour is obtained by implementing a flocking algorithm.

3D sparse feature model using short baseline stereo and multiple view registration

This paper outlines a methodology to generate a distinctive object representation offline, using short-baseline stereo fundamentals to triangulate highly descriptive object features in multiple pairs of stereo images. A group of sparse 2.5D perspective views are built and the multiple views are then fused into a single sparse 3D model using a common 3D shape registration technique. Having prior knowledge, such as the proposed sparse feature model, is useful when detecting an object and estimating its pose for real-time systems like augmented reality.

Augmented optometry training simulator with multi-point haptics

Training of optometrists is traditionally achieved under close supervision of peers and superiors. With the rapid advancement in technology, medical procedures are performed more efficiently and effectively, resulting in faster recovery times and less trauma to the patient. However, application of this technology has made it difficult to effectively demonstrate and teach these manual skills as the education is now a combination of not only the medical procedure but also the use of the technology. In this paper we propose to increase the training capabilities of optometry students through haptically-enabled single-point and multi-point training tools as well as augmented reality techniques. Haptics technology allows a human to touch and feel virtual computer models as though they are real. Through physical connection to the operator, haptic devices are considered to be personal robots that are capable of improving the human-computer interaction with a virtual environment. These devices have played an increasing role in developing expertise, reducing instances of medical error and reducing training costs. A haptically-enabled virtual training environment, integrated with an optometry slit lamp instrument can be used to teach cognitive and manual skills while the system tracks the performance of each individual. These interactions would ideally replicate every aspect of the real procedure, consequently preparing the trainee for every possible scenario, without risking the health of a real patient.

Towards parameter-less 3D mesh segmentation

This thesis focuses on the 3D mesh segmentation process. The research demonstrated how the process can be done in a parameterless approach which allows full automation with accurate results. Applications of this research include, but not limited to, 3D search engines, 3D character animation, robotics environment recognition, and augmented reality.

Keyboard control method for virtual reality micro-robotic cell injection training

The rapid development of virtual reality offers significant potential for skills training applications. Our ongoing work proposes virtual reality operator training for the micro-robotic cell injection procedure. The interface between the operator and the system can be achieved in many different ways. The computer keyboard is ubiquitous in its use for everyday computing applications and also commonly utilized in virtual reality systems. Based on the premise that most people have experience in using a computer keyboard, as opposed to more sophisticated input devices, this paper considers the feasibility of using a keyboard to control the micro-robot for cell injection. In this study, thirteen participants underwent the experimental evaluation. The participants were asked to perform three simulated trial sessions in a virtual micro-robotic cell injection environment. Each session consisted of ten cell injection trials and relevant data for each trial were recorded and analyzed. Results showed participants' performance improvement after the three sessions. It was also observed that participants intuitively controlled multiple axes of the micro-robot simultaneously despite the absence of instruction on how to do so. This continued throughout the experiments and suggests skills transfer from other keyboard based interactions. Based on the results provided, it is suggested that keyboard control is a feasible, simple and low-cost control method for the virtual micro-robot.

Realidade aumentada aplicada em bibliotecas: multiplicidade no acesso a informação

Conceitua Realidade Aumentada do ponto de vista histórico de vários autores e como surgiu esse segmento através da evolução tecnológica até os dias atuais. Aborda seu funcionamento, como também, seus sistemas e aplicações em diversos campos de pesquisas e estudos científicos. Diferencia Realidade Aumentada de Realidade Virtual, visando melhor esclarecimento entre ambas na intenção de descaracterizá-las com uma única “realidade”. Apresenta a Realidade Aumentada e sua aplicação dentro de um contexto de uma unidade de informação, promovendo uma melhor interação com os usuários e as adaptações pelas quais as bibliotecas terão que passar futuramente para se adequarem a “explosão” tecnológica. Descreve o funcionamento da biblioteca ARToolKIT, baseada em RA e suas principais etapas de funcionamento para visualização de objetos virtuais em 3D. Exemplifica os benefícios que uma unidade de informação, que utiliza Realidade Aumentada, promove aos usuários portadores de deficiência, além de sua inclusão no meio digital e sua inserção no mercado de trabalho

Odometria visual baseada em técnicas de structure from motion

Visual Odometry is the process that estimates camera position and orientation based solely on images and in features (projections of visual landmarks present in the scene) extraced from them. With the increasing advance of Computer Vision algorithms and computer processing power, the subarea known as Structure from Motion (SFM) started to supply mathematical tools composing localization systems for robotics and Augmented Reality applications, in contrast with its initial purpose of being used in inherently offline solutions aiming 3D reconstruction and image based modelling. In that way, this work proposes a pipeline to obtain relative position featuring a previously calibrated camera as positional sensor and based entirely on models and algorithms from SFM. Techniques usually applied in camera localization systems such as Kalman filters and particle filters are not used, making unnecessary additional information like probabilistic models for camera state transition. Experiments assessing both 3D reconstruction quality and camera position estimated by the system were performed, in which image sequences captured in reallistic scenarios were processed and compared to localization data gathered from a mobile robotic platform

The generation of scenes in mixed reality environments using the chromakey technique

A classic problem in the development of Mixed Reality systems is the registration. The correct alignment between virtual objects and the real elements is extremely important for the coherent composition of the resultant scene. Considering this context, this paper describes an approach for the composition of scenes in Mixed Reality environments using the chromakey technique for the extraction of real objects. After that, the scene is mounted in a coherent way related to the depth in OpenGL framebuffer for posterior rendering. ©2007 IEEE.

Uma metodologia para a geração de conteúdos digitais baseado na utilização de estudios virtuais com realidade aumentada

Pós-graduação em Televisão Digital: Informação e Conhecimento - FAAC

Ideazione di sistemi distribuiti collaborativi basati realta aumentata mobile: un caso di studio, con approfondimento relativo al livello delle comunicazioni

Il grande sviluppo a cui stiamo assistendo negli ultimi anni nell'ambito delle tecnologie mobile e del wearable computing apre le porte a scenari innovativi e interessanti per quanto riguarda sistemi distribuiti collaborativi. Le persone possono sempre più facilmente cooperare e scambiarsi informazioni grazie a queste nuove tecnologie e si può pensare allo sviluppo di sistemi che permettano forme molto avanzate di collaborazione facendo leva sull'aspetto hands-free, ovvero sulla possibilità di utilizzare dispositivi che liberino le mani, come i moderni smart-glasses. Per lo sviluppo di tali sistemi è necessario però studiare nuove tecniche e architetture, in quanto gli strumenti ad oggi a disposizione a supporto della realtà aumentata non sembrano essere del tutto adeguati per tale scopo. Infatti piattaforme come Wikitude o Layar, seppure offrano potenti tecniche di riconoscimento di markers e immagini e di rendering, non offrono quella dinamicità fondamentale per un sistema distribuito collaborativo. Questo scritto ha lo scopo di esplorare questi aspetti mediante l'ideazione, l'analisi, la progettazione e la prototipazione di un semplice caso di studio ispirato a sistemi collaborativi distribuiti basati su realtà aumentata. In particolare in questo lavoro si porrà l'attenzione sul livello delle comunicazioni e delle infrastrutture di rete.

Image-guided surgical microscope with mounted minitracker

A new image-guided microscope using augmented reality overlays has been developed. Unlike other systems, the novelty of our design consists in mounting a precise mini and low-cost tracker directly on the microscope to track the motion of the surgical tools and the patient. Correctly scaled cut-views of the pre-operative computed tomography (CT) stack can be displayed on the overlay, orthogonal to the optical view or even including the direction of a clinical tool. Moreover, the system can manage three-dimensional models for tumours or bone structures and allows interaction with them using virtual tools, showing trajectories and distances. The mean error of the overlay was 0.7 mm. Clinical accuracy has shown results of 1.1-1.8 mm.

Digital Illumination for Augmented Studios

Virtual studio technology plays an important role for modern television productions. Blue-screen matting is a common technique for integrating real actors or moderators into computer generated sceneries. Augmented reality offers the possibility to mix real and virtual in a more general context. This article proposes a new technological approach for combining real studio content with computergenerated information. Digital light projection allows a controlled spatial, temporal, chrominance and luminance modulation of illumination – opening new possibilities for TV studios.