Добавена реалност (Augmented Reality) – същност и приложение

Доклад, поместен в сборника на Националната конференция "Образованието в информационното общество", Пловдив, май, 2012 г.

An augmented reality pipeline to create scenes with coherent illumination using textured cuboids

Shadows and illumination play an important role when generating a realistic scene in computer graphics. Most of the Augmented Reality (AR) systems track markers placed in a real scene and retrieve their position and orientation to serve as a frame of reference for added computer generated content, thereby producing an augmented scene. Realistic depiction of augmented content with coherent visual cues is a desired goal in many AR applications. However, rendering an augmented scene with realistic illumination is a complex task. Many existent approaches rely on a non automated pre-processing phase to retrieve illumination parameters from the scene. Other techniques rely on specific markers that contain light probes to perform environment lighting estimation. This study aims at designing a method to create AR applications with coherent illumination and shadows, using a textured cuboid marker, that does not require a training phase to provide lighting information. Such marker may be easily found in common environments: most of product packaging satisfies such characteristics. Thus, we propose a way to estimate a directional light configuration using multiple texture tracking to render AR scenes in a realistic fashion. We also propose a novel feature descriptor that is used to perform multiple texture tracking. Our descriptor is an extension of the binary descriptor, named discrete descriptor, and outperforms current state-of-the-art methods in speed, while maintaining their accuracy.

Interfaccia utente per assegnamento missioni e monitoraggio di un trattore automatico

Il presente lavoro consiste nella realizzazione di un'interfaccia utente adibita all'assegnamento di missioni e al monitoraggio remoto di un rover agricolo autonomo. Sfruttando l'informatica per la sua implementazione, tale interfaccia trova invece applicazione nel campo dell'automazione e dell'agricoltura di precisione. L'utilizzatore ha perciò la facoltà di muovere il rover in campo aperto e di demandargli missioni specifiche, ricevendo allo stesso tempo un feedback continuo sul suo operato. L'applicativo software comunica quindi in maniera bidirezionale con il veicolo controllato ed è predisposto per sfruttare diversi canali di comunicazione (antenne seriali, pacchetti udp, socket tcp). La scrittura del codice è stata seguita da una serie di prove di comunicazione con il veicolo, effettuate indoor, e infine da alcuni test completi effettuati outdoor, con il rover in movimento.

AR-park för Android : Augmented Reality park i Sundsvall

Målet med detta projekt är att ta fram en applikationsprototyp för Androidenheter som ska locka användare av applikationen till och tillbaka till broparken i Skönsberg med hjälp av augmented-reality. Applikationen ska känna av om användaren befinner sig inom eller utanför parkens område och visa olika bilder/modeller på specifika GPS-koordinater i det digitala lagret beroende på användarens positionering. Arbetet har genomförts i samarbete med Dohi på uppdrag av Sundsvalls kommun där regelbundna möten hafts med uppdragsgivaren. Utvecklingen av applikationen sker i PhoneGap med Wikitude-plugin. Projektet har resulterat i en applikationsprototyp som använder ActionRanges, som är en typ av GeoFence, för att presentera olika bilder hämtade från en egen server i det digitala lagret beroende på användarens position. Användarna har inom parkens område möjlighet att själv påverka de bilder som visas i det digitala lagret genom att i applikationen ta en bild som laddas upp till servern där bilderna lagras och där bilden som tagits även visar det digitala lagret.

Entwicklung eines Funktionsumfangs für Augmented Reality Führungen in Hochschulbibliotheken

Anwendungen in Bibliotheken auf Basis von Augmented Reality (AR), der Erweiterung der Realität durch zusätzliche, virtuelle Informationen, existieren bislang nur vereinzelt. Das vom Bundesministerium für Wirtschaft und Energie geförderte Forschungsprojekt mylibrARy ermittelt seit Juni 2014 Szenarien für den Einsatz von AR in Bibliotheken und Informationseinrichtungen. Das Projekt zielt auf die Entwicklung einer deutschlandweit einsetzbaren Applikation (App). An diesen Gedanken knüpft die vorliegende Arbeit mit der Entwicklung eines Konzepts für Einführungen in Hochschulbibliotheken mittels AR-Technik. Aus der Analyse aktueller Führungsmodelle von Bibliotheken der Hochschulen im Exzellenzcluster 2012-2017, werden die für die Modellführung benötigten charakteristischen Merkmale einer Bibliotheksführung abgeleitet. Auf die technisch relevanten Grundlagen der AR folgt eine Bestandsaufnahme der bisherigen Augmented-Reality-Projekte im internationalen Bibliothekskontext. Daran schließt die Formulierung der theoretischen Anforderungen und die Ausarbeitung der Funktionen im Einzelnen. Nachfolgend lenkt eine kritische Betrachtung den Fokus auf Mehrwert und Risiken der AR für Bibliothek und Nutzer. Mit einer Zusammenfassung und dem Ausblick auf die zukünftige Entwicklung findet die Arbeit ihren Abschuss.

Applying Augmented Reality to Outdoors Industrial Use

Augmented Reality (AR) is currently gaining popularity in multiple different fields. However, the technology for AR still requires development in both hardware and software when considering industrial use. In order to create immersive AR applications, more accurate pose estimation techniques to define virtual camera location are required. The algorithms for pose estimation often require a lot of processing power, which makes robust pose estimation a difficult task when using mobile devices or designated AR tools. The difficulties are even larger in outdoor scenarios where the environment can vary a lot and is often unprepared for AR. This thesis aims to research different possibilities for creating AR applications for outdoor environments. Both hardware and software solutions are considered, but the focus is more on software. The majority of the thesis focuses on different visual pose estimation and tracking techniques for natural features. During the thesis, multiple different solutions were tested for outdoor AR. One commercial AR SDK was tested, and three different custom software solutions were developed for an Android tablet. The custom software solutions were an algorithm for combining data from magnetometer and a gyroscope, a natural feature tracker and a tracker based on panorama images. The tracker based on panorama images was implemented based on an existing scientific publication, and the presented tracker was further developed by integrating it to Unity 3D and adding a possibility for augmenting content. This thesis concludes that AR is very close to becoming a usable tool for professional use. The commercial solutions currently available are not yet ready for creating tools for professional use, but especially for different visualization tasks some custom solutions are capable of achieving a required robustness. The panorama tracker implemented in this thesis seems like a promising tool for robust pose estimation in unprepared outdoor environments.

Evaluating the use of mobile collaborative augmented reality within field service networks: the case of Océ Italia – Canon Group

The adoption of Augmented Reality (AR) technologies can make the provision of field services to industrial equipment more effective. In these situations, the cost of deploying skilled technicians in geographically dispersed locations must be accurately traded off with the risks of not respecting the service level agreements with the customers. This paper, through the case study of a leading OEM in the production printing industry, presents the challenges that have to be faced in order to favour the adoption of a particular kind of AR named Mobile Collaborative Augmented Reality (MCAR). In particular, this study uses both qualitative and quantitative research. Firstly, a demonstration to show how MCAR can support field service was settled in order to achieve information about the use experience of the people involved. Then, the entire field force of Océ Italia – Canon Group was surveyed in order to investigate quantitatively the technicians’ perceptions about the usefulness and ease of use of MCAR, as well as their intentions to use this technology.

Design and Development of a Research Framework for Prototyping Control Tower Augmented Reality Tools

The purpose of the air traffic management system is to ensure the safe and efficient flow of air traffic. Therefore, while augmenting efficiency, throughput and capacity in airport operations, attention has rightly been placed on doing it in a safe manner. In the control tower, many advances in operational safety have come in the form of visualization tools for tower controllers. However, there is a paradox in developing such systems to increase controllers' situational awareness: by creating additional computer displays, the controller's vision is pulled away from the outside view and the time spent looking down at the monitors is increased. This reduces their situational awareness by forcing them to mentally and physically switch between the head-down equipment and the outside view. This research is based on the idea that augmented reality may be able to address this issue. The augmented reality concept has become increasingly popular over the past decade and is being proficiently used in many fields, such as entertainment, cultural heritage, aviation, military & defense. This know-how could be transferred to air traffic control with a relatively low effort and substantial benefits for controllers’ situation awareness. Research on this topic is consistent with SESAR objectives of increasing air traffic controllers’ situation awareness and enable up to 10 % of additional flights at congested airports while still increasing safety and efficiency. During the Ph.D., a research framework for prototyping augmented reality tools was set up. This framework consists of methodological tools for designing the augmented reality overlays, as well as of hardware and software equipment to test them. Several overlays have been designed and implemented in a simulated tower environment, which is a virtual reconstruction of Bologna airport control tower. The positive impact of such tools was preliminary assessed by means of the proposed methodology.

Application of augmented reality in craniofacial surgery: a feasibility study

Augmented Reality (AR) is a novel promising technology, which is gaining success in the medical field. A number of applications in surgery have been described, but few studies have been focusing on pediatric craniofacial surgery. In this research project, the Authors have been implementing a system for intraoperative surgical navigation by means of HoloLens 2 by Microsoft, applied to pediatric craniofacial surgery. The Authors tested the device in a preclinical setting first, and then moved to patients. The Authors assessed the accuracy of the HoloLens 2 by performing 36 procedures in vitro on a printed 3D model of a patient. In clinical setting, 10 patients were prospectively enrolled in the study. The virtual surgical planning was designed for each patient and uploaded onto the software which allows for the AR interface and the standard neurosurgical navigator. For each patient, the surgeon has been drawing osteotomy lines both under the guidance of HoloLens2 and of the neurosurgical navigator. The Author then checked the accuracy with calibrated CAD CAM cutting guides with different grooves, in order to assess the accuracy of the osteotomies performed. We tested levels of accuracy of ±1.5 mm and ±1mm . In the preclinical setting, the HoloLens 2 performed with levels of accuracy of 1.5 mm, whereas in the real setting, surgeons were able to trace the osteotomy lines under the AR guidance for an amount of 45% (0.4 SD) of the entire line, with an accuracy level of ±1.5 mm. This percentage lowers to 34% (0.4 SD) when assessing accuracy level of ±1 mm. The results of the same tasks for the standard navigator are 36% and 16%, for ±1.5 mm and ± 1 mm accuracy level, respectively. The Authors reported encouraging results both in the preclinical and the clinical setting.

The road to augmented reality: a comprehensive study of hardware and applications over the last ten years

Augmented reality has been growing extensively over the years in all aspects and multiple fields. My aim in this paper is to present a comprehensive study on augmented reality(AR) hardware and its applications from early developments to the possible future trends. Particularly my research is more focused on last 11 years(2012-2022), where I systematically reviewed 30 research papers per year to get a clear knowledge on trends of AR. A total of 330 publications were reviewed and grouped according to their application. The review's main contribution is to show the entire landscape of AR research and to provide a broad view of how it has evolved. Along with various AR glasses history and specifications are presented in detail. In the penultimate chapter I explained my methodology of research following my analysis from the past to the present along with my thoughts for the future. To conclude my study, In the final chapter I made some statements about possible future with AR, VR and XR(extended reality).

Enhanced switchgear connection testing through augmented reality: early developments

Augmented reality is an emerging field of interactive design in which virtual material is seamlessly blended with displays of real world environments. The tremendous potential of augmented reality has begun to be explored with the emergence of personal mobile devices capable of constructing engaging augmented reality experiences. This work is part of a project aiming at using augmented reality goggles to bring advance information to the user interacting with switch-gear during automation cabling. In particular we will be focusing on the recognition and definition of the figures of the component on the AR device. In this part we are using standard camera that allows us to get real images and helps us to localize the gearbox in space through ARUCO marker and we can exploit in order to re-project the actual shape of the component that are currently interested in manipulation by exploiting the data provided by the database. The experiments are carried out using the camera to get the images of the real world switch-gear and re-project those images with the component superimposed on it. Using transforms of the database we did localization to re-project the rendered image of component exactly on the real world component, which can be further integrated in AR goggles to see the component superimposed in real-time.

Augmented Reality for Airport Control Tower: Analysis and Implementation of Safety Nets

With the development of new technologies, Air Traffic Control, in the nearby of the airport, switched from a purely visual control to the use of radar, sensors and so on. As the industry is switching to the so-called Industry 4.0, also in this frame, it would be possible to implement some of the new tools that can facilitate the work of Air Traffic Controllers. The European Union proposed an innovative project to help the digitalization of the European Sky by means of the Single European Sky ATM Research (SESAR) program, which is the foundation on which the Single European Sky (SES) is based, in order to improve the already existing technologies to transform Air Traffic Management in Europe. Within this frame, the Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision (RETINA) project, which saw the light in 2016, studied the possibility to apply new tools within the conventional control tower to reduce the air traffic controller workload, thanks to the improvements in the augmented reality technologies. After the validation of RETINA, the Digital Technologies for Tower (DTT) project was established and the solution proposed by the University of Bologna aimed, among other things, to introduce Safety Nets in a Head-Up visualization. The aim of this thesis is to analyze the Safety Nets in use within the control tower and, by developing a working concept, implement them in a Head-Up view to be tested by Air Traffic Control Operators (ATCOs). The results, coming from the technical test, show that this concept is working and it could be leading to a future implementation in a real environment, as it improves the air traffic controller working conditions also when low visibility conditions apply.

Studio e sviluppo di un framework per il riconoscimento vocale nell'ambito di sistemi Hands-Free

Negli ultimi anni, l'avanzamento incredibilmente rapido della tecnologia ha portato allo sviluppo e alla diffusione di dispositivi elettronici portatili aventi dimensioni estremamente ridotte e, allo stesso tempo, capacità computazionali molto notevoli. Più nello specifico, una particolare categoria di dispositivi, attualmente in forte sviluppo, che ha già fatto la propria comparsa sul mercato mondiale è sicuramente la categoria dei dispositivi Wearable. Come suggerisce il nome, questi sono progettati per essere letteralmente indossati, pensati per fornire continuo supporto, in diversi ambiti, a chi li utilizza. Se per interagire con essi l’utente non deve ricorrere obbligatoriamente all'utilizzo delle mani, allora si parla di dispositivi Wearable Hands Free. Questi sono generalmente in grado di percepire e catture l’input dell'utente seguendo tecniche e metodologie diverse, non basate sul tatto. Una di queste è sicuramente quella che prevede di modellare l’input dell’utente stesso attraverso la sua voce, appoggiandosi alla disciplina dell’ASR (Automatic Speech Recognition), che si occupa della traduzione del linguaggio parlato in testo, mediante l’utilizzo di dispositivi computerizzati. Si giunge quindi all’obiettivo della tesi, che è quello di sviluppare un framework, utilizzabile nell’ambito dei dispositivi Wearable, che fornisca un servizio di riconoscimento vocale appoggiandosi ad uno già esistente, in modo che presenti un certo livello di efficienza e facilità di utilizzo. Più in generale, in questo documento si punta a fornire una descrizione approfondita di quelli che sono i dispositivi Wearable e Wearable Hands-Free, definendone caratteristiche, criticità e ambiti di utilizzo. Inoltre, l’intento è quello di illustrare i principi di funzionamento dell’Automatic Speech Recognition per passare poi ad analisi, progettazione e sviluppo del framework appena citato.

Realtà aumentata per dispositivi android: lo stato dell'arte

Obiettivo di questa tesi è quello di illustrare il mondo della realtà aumentata (AR) ed in particolare delle tecnologie software disponibili per lo sviluppo di applicazioni su dispositivi Android. Si partirà dal darne una definizione e riassumerne i principali fatti storici, all'illustrarne i vari hardware disponibili sul mercato e le tecnologie software per sviluppare progetti. Non verranno tralasciati utilizzi e settori di ricerca, e si presenterà poi il sistema operativo Android. Dopo uno sguardo alla sua architettura e alle sue caratteristiche, nonché al linguaggio di programmazione Java, cardine per lo sviluppo in questo sistema, si presenteranno alcune API dell'SDK nativo che si rivelano utili per lo sviluppo di applicazioni per la realtà aumentata. Infine, verrà presentato un approfondimento sull'SDK Metaio.

Ideazione di sistemi distribuiti collaborativi basati realta aumentata mobile: un caso di studio, con approfondimento relativo al livello delle comunicazioni

Il grande sviluppo a cui stiamo assistendo negli ultimi anni nell'ambito delle tecnologie mobile e del wearable computing apre le porte a scenari innovativi e interessanti per quanto riguarda sistemi distribuiti collaborativi. Le persone possono sempre più facilmente cooperare e scambiarsi informazioni grazie a queste nuove tecnologie e si può pensare allo sviluppo di sistemi che permettano forme molto avanzate di collaborazione facendo leva sull'aspetto hands-free, ovvero sulla possibilità di utilizzare dispositivi che liberino le mani, come i moderni smart-glasses. Per lo sviluppo di tali sistemi è necessario però studiare nuove tecniche e architetture, in quanto gli strumenti ad oggi a disposizione a supporto della realtà aumentata non sembrano essere del tutto adeguati per tale scopo. Infatti piattaforme come Wikitude o Layar, seppure offrano potenti tecniche di riconoscimento di markers e immagini e di rendering, non offrono quella dinamicità fondamentale per un sistema distribuito collaborativo. Questo scritto ha lo scopo di esplorare questi aspetti mediante l'ideazione, l'analisi, la progettazione e la prototipazione di un semplice caso di studio ispirato a sistemi collaborativi distribuiti basati su realtà aumentata. In particolare in questo lavoro si porrà l'attenzione sul livello delle comunicazioni e delle infrastrutture di rete.