Sviluppo di funzionalità di Image Targeting con apps basate su realtà aumentata

Dopo una breve introduzione sulla realtà aumentata (definizione, storia e stato dell’arte) viene effettuata un'analisi delle librerie esistenti per l’implementazione su dispositivi mobile. Considerando compatibilità con i recenti SO, frequenza degli aggiornamenti, costi per le licenze e funzionalità offerte viene scelta la libreria Vuforia, originariamente sviluppata da Qualcomm e poi acquistata da PTC inc. Si conviene poi che le apps basate su realtà aumentata creano il contenuto “aumentato” in due maniere: o tramite riconoscimento di una specifica immagine oppure tramite localizzazione GPS. Di questi due metodi descritti, il primo risulta molto più affidabile e per questo viene sviluppata una app che crea un contenuto in 3D (aumentato) riconoscendo una immagine: funzionalità di Image Targeting. Il progetto considera le seguenti varianti: l’immagine da riconoscere, chiamata “target”, può essere in un database locale oppure cloud mentre il contenuto in 3D aumentato può essere sia statico che animato. Durante la fase di implementazione vengono fornite anche alcuni nozioni di base di Computer Graphic per il rendering del modello 3D. La tesi si conclude con una panoramica di apps presenti sullo store che funzionano secondo questo principio di Image Targeting, i possibili utilizzi in ambito educativo/ludico ed i costi di realizzazione.

Studio e sviluppo di un Framework in ambiente Android per la realizzazione di Interfacce Utente Innovative basate su Smart Glass

La tesi tratta dell'esplorazione di una possibile interfaccia utente per Smart Glass in un contesto di utilizzo hands-free con elementi virtuali appartenenti ad un sistema di riferimento solidale all'utente e non al dispositivo, e la conseguente realizzazione di un Framework per lo sviluppo di applicazioni Andoid rispondenti a tale interfaccia e relativo test.

Studio e Progettazione di Sistemi Hands-free basati su Tecnologie Wearable: Un Caso di Studio in Ambito Healthcare

Il percorso intrapreso per la scrittura della tesi, si snoda essenzialmente in due fasi e nove capitoli. La prima fase ha avuto come scopo l’analisi dal punto di vista tecnologico dell’evoluzione tecnologica avuta nel campo di realtà aumentata e dispositivi wearable in particolare orientandosi verso una tipologia di interazioni hands-free. Questo ha portato ad una ricognizione sullo stato dell’arte permettendo di attenere una base di conoscenza solida per la costruzione del sistema presentato all’interno del caso di studi. I capitoli successivi, in particolare dal quinto, introducono alla seconda fase ed hanno lo scopo di progettare e realizzare il sistema proposto, partendo da un’attenta analisi delle caratteristiche richieste passando per la prototipazione e successiva definizione delle caratteristiche atte alla valutazione del sistema stesso.

Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies

Presenting visual feedback for image-guided surgery on a monitor requires the surgeon to perform time-consuming comparisons and diversion of sight and attention away from the patient. Deficiencies in previously developed augmented reality systems for image-guided surgery have, however, prevented the general acceptance of any one technique as a viable alternative to monitor displays. This work presents an evaluation of the feasibility and versatility of a novel augmented reality approach for the visualisation of surgical planning and navigation data. The approach, which utilises a portable image overlay device, was evaluated during integration into existing surgical navigation systems and during application within simulated navigated surgery scenarios.

Endoscopic Image Overlay for the Targeting of Hidden Anatomy in Laparoscopic Visceral Surgery

Limitations associated with the visual information provided to surgeons during laparoscopic surgery increases the difficulty of procedures and thus, reduces clinical indications and increases training time. This work presents a novel augmented reality visualization approach that aims to improve visual data supplied for the targeting of non visible anatomical structures in laparoscopic visceral surgery. The approach aims to facilitate the localisation of hidden structures with minimal damage to surrounding structures and with minimal training requirements. The proposed augmented reality visualization approach incorporates endoscopic images overlaid with virtual 3D models of underlying critical structures in addition to targeting and depth information pertaining to targeted structures. Image overlay was achieved through the implementation of camera calibration techniques and integration of the optically tracked endoscope into an existing image guidance system for liver surgery. The approach was validated in accuracy, clinical integration and targeting experiments. Accuracy of the overlay was found to have a mean value of 3.5 mm ± 1.9 mm and 92.7% of targets within a liver phantom were successfully located laparoscopically by non trained subjects using the approach.

Image-guided surgical microscope with mounted minitracker

A new image-guided microscope using augmented reality overlays has been developed. Unlike other systems, the novelty of our design consists in mounting a precise mini and low-cost tracker directly on the microscope to track the motion of the surgical tools and the patient. Correctly scaled cut-views of the pre-operative computed tomography (CT) stack can be displayed on the overlay, orthogonal to the optical view or even including the direction of a clinical tool. Moreover, the system can manage three-dimensional models for tumours or bone structures and allows interaction with them using virtual tools, showing trajectories and distances. The mean error of the overlay was 0.7 mm. Clinical accuracy has shown results of 1.1-1.8 mm.

Impact of a self-developed planning and self-constructed navigation system on skull base surgery: 10 years experience

CONCLUSION: Our self-developed planning and navigation system has proven its capacity for accurate surgery on the anterior and lateral skull base. With the incorporation of augmented reality, image-guided surgery will evolve into 'information-guided surgery'. OBJECTIVE: Microscopic or endoscopic skull base surgery is technically demanding and its outcome has a great impact on a patient's quality of life. The goal of the project was aimed at developing and evaluating enabling navigation surgery tools for simulation, planning, training, education, and performance. This clinically applied technological research was complemented by a series of patients (n=406) who were treated by anterior and lateral skull base procedures between 1997 and 2006. MATERIALS AND METHODS: Optical tracking technology was used for positional sensing of instruments. A newly designed dynamic reference base with specific registration techniques using fine needle pointer or ultrasound enables the surgeon to work with a target error of < 1 mm. An automatic registration assessment method, which provides the user with a color-coded fused representation of CT and MR images, indicates to the surgeon the location and extent of registration (in)accuracy. Integration of a small tracker camera mounted directly on the microscope permits an advantageous ergonomic way of working in the operating room. Additionally, guidance information (augmented reality) from multimodal datasets (CT, MRI, angiography) can be overlaid directly onto the surgical microscope view. The virtual simulator as a training tool in endonasal and otological skull base surgery provides an understanding of the anatomy as well as preoperative practice using real patient data. RESULTS: Using our navigation system, no major complications occurred in spite of the fact that the series included difficult skull base procedures. An improved quality in the surgical outcome was identified compared with our control group without navigation and compared with the literature. The surgical time consumption was reduced and more minimally invasive approaches were possible. According to the participants' questionnaires, the educational effect of the virtual simulator in our residency program received a high ranking.

Calibration of a surgical microscope with automated zoom lenses using an active optical tracker

BACKGROUND: In this paper, we present a new method for the calibration of a microscope and its registration using an active optical tracker. METHODS: Practically, both operations are done simultaneously by moving an active optical marker within the field of view of the two devices. The IR LEDs composing the marker are first segmented from the microscope images. By knowing their corresponding three-dimensional (3D) position in the optical tracker reference system, it is possible to find the transformation matrix between the referential of the two devices. Registration and calibration parameters can be extracted directly from that transformation. In addition, since the zoom and focus can be modified by the surgeon during the operation, we propose a spline based method to update the camera model to the new setup. RESULTS: The proposed technique is currently being used in an augmented reality system for image-guided surgery in the fields of ear, nose and throat (ENT) and craniomaxillofacial surgeries. CONCLUSIONS: The results have proved to be accurate and the technique is a fast, dynamic and reliable way to calibrate and register the two devices in an OR environment.

Methods and Applications in Interactive Broadcasting

Interactive TV technology has been addressed in many previous works, but there is sparse research on the topic of interactive content broadcasting and how to support the production process. In this article, the interactive broadcasting process is broadly defined to include studio technology and digital TV applications at consumer set-top boxes. In particular, augmented reality studio technology employs smart-projectors as light sources and blends real scenes with interactive computer graphics that are controlled at end-user terminals. Moreover, TV producer-friendly multimedia authoring tools empower the development of novel TV formats. Finally, the support for user-contributed content raises the potential to revolutionize the hierarchical TV production process, by introducing the viewer as part of content delivery chain.

Tracking of industrial objects by using CAD models

In this paper we present a model-based approach for real-time camera pose estimation in industrial scenarios. The line model which is used for tracking is generated by rendering a polygonal model and extracting contours out of the rendered scene. By un-projecting a point on the contour with the depth value stored in the z-buffer, the 3D coordinates of the contour can be calculated. For establishing 2D/3D correspondences the 3D control points on the contour are projected into the image and a perpendicular search for gradient maxima for every point on the contour is performed. Multiple hypotheses of 2D image points corresponding to a 3D control point make the pose estimation robust against ambiguous edges in the image.

Virtuelle Prototypen – Möglichkeiten und Grenzen

Die Ergebnisse der Konstruktion können so aufbereitet werden, dass sie nach entsprechenden Berechnungen und Simulationen als virtuelle Prototypen zur Verfügen gestellt werden können. Die Möglichkeiten des Einsatzes virtueller Prototypen werden aufgezeigt. Der Unterschied zwischen virtuellen und realen Prototypen in Bezug auf die individuelle Wahrnehmung aufgrund der Sinnesmodalitäten wird erläutert. Die gegenwärtigen Grenzen der virtuellen Prototypen werden aufgezeigt.

Interactive Augmentation of Live Images using a HDR Stereo Camera

Adding virtual objects to real environments plays an important role in todays computer graphics: Typical examples are virtual furniture in a real room and virtual characters in real movies. For a believable appearance, consistent lighting of the virtual objects is required. We present an augmented reality system that displays virtual objects with consistent illumination and shadows in the image of a simple webcam. We use two high dynamic range video cameras with fisheye lenses permanently recording the environment illumination. A sampling algorithm selects a few bright parts in one of the wide angle images and the corresponding points in the second camera image. The 3D position can then be calculated using epipolar geometry. Finally, the selected point lights are used in a multi pass algorithm to draw the virtual object with shadows. To validate our approach, we compare the appearance and shadows of the synthetic objects with real objects.