Application of Time-Delay Estimation to Mixed Reality Multisensor Tracking

Spatial tracking is one of the most challenging and important parts of Mixed Reality environments. Many applications, especially in the domain of Augmented Reality, rely on the fusion of several tracking systems in order to optimize the overall performance. While the topic of spatial tracking sensor fusion has already seen considerable interest, most results only deal with the integration of carefully arranged setups as opposed to dynamic sensor fusion setups. A crucial prerequisite for correct sensor fusion is the temporal alignment of the tracking data from several sensors. Tracking sensors are typically encountered in Mixed Reality applications, are generally not synchronized. We present a general method to calibrate the temporal offset between different sensors by the Time Delay Estimation method which can be used to perform on-line temporal calibration. By applying Time Delay Estimation on the tracking data, we show that the temporal offset between generic Mixed Reality spatial tracking sensors can be calibrated. To show the correctness and the feasibility of this approach, we have examined different variations of our method and evaluated various combinations of tracking sensors. We furthermore integrated this time synchronization method into our UBITRACK Mixed Reality tracking framework to provide facilities for calibration and real-time data alignment.

Development of the infrastructure for a multi-agent traffic management system simulation

This document contains detailed description of the design and the implementation of a multi-agent application controlling traffic lights in a city together with a system for simulating traffic and testing. The goal of this thesis is to design and build a simplified intelligent and distributed solution to the problem with the traffic in the big cities following different good practices in order to allow future refining of the model of the real world. The problem of the traffic in the big cities is still a problem that cannot be solved. Not only is the increasing number of cars a reason for the traffic jams, but also the way the traffic is organized. Usually, the intersections with traffic lights are replaced by roundabouts or interchanges to increase the number of cars that can cross the intersection in certain time. But still there are places where the infrastructure cannot be changed and the traffic light semaphores are the only way to control the car flows. In real life, the traffic lights have a predefined plan for change or they receive information from a centralized system when and how they have to change. But what if the traffic lights can cooperate and decide on their own when and how to change? Using this problem, the purpose of the thesis is to explore different agent-based software engineering approaches to design and build a non-conventional distributed system. From the software engineering point of view, the goal of the thesis is to apply the knowledge and use the skills, acquired during the various courses of the master program in Software Engineering, while solving a practical and complex problem such as the traffic in the cities.

System failure prediction through rare-events elastic-net logistic regression

Predicting failures in a distributed system based on previous events through logistic regression is a standard approach in literature. This technique is not reliable, though, in two situations: in the prediction of rare events, which do not appear in enough proportion for the algorithm to capture, and in environments where there are too many variables, as logistic regression tends to overfit on this situations; while manually selecting a subset of variables to create the model is error- prone. On this paper, we solve an industrial research case that presented this situation with a combination of elastic net logistic regression, a method that allows us to automatically select useful variables, a process of cross-validation on top of it and the application of a rare events prediction technique to reduce computation time. This process provides two layers of cross- validation that automatically obtain the optimal model complexity and the optimal mode l parameters values, while ensuring even rare events will be correctly predicted with a low amount of training instances. We tested this method against real industrial data, obtaining a total of 60 out of 80 possible models with a 90% average model accuracy.

A Ground-Truth Video Dataset for the Development and Evaluation of Vision-based Sense-and-Avoid systems

The importance of vision-based systems for Sense-and-Avoid is increasing nowadays as remotely piloted and autonomous UAVs become part of the non-segregated airspace. The development and evaluation of these systems demand flight scenario images which are expensive and risky to obtain. Currently Augmented Reality techniques allow the compositing of real flight scenario images with 3D aircraft models to produce useful realistic images for system development and benchmarking purposes at a much lower cost and risk. With the techniques presented in this paper, 3D aircraft models are positioned firstly in a simulated 3D scene with controlled illumination and rendering parameters. Realistic simulated images are then obtained using an image processing algorithm which fuses the images obtained from the 3D scene with images from real UAV flights taking into account on board camera vibrations. Since the intruder and camera poses are user-defined, ground truth data is available. These ground truth annotations allow to develop and quantitatively evaluate aircraft detection and tracking algorithms. This paper presents the software developed to create a public dataset of 24 videos together with their annotations and some tracking application results.

Plataforma terapeútica basada en Kinect para niños con parálisis cerebral infantil

El sistema SONRIE (Sistema de terapia, basadO en KiNect, paRa nIños con parálisis cErebral), realizado como Proyecto Fin de Grado por Dña. Estefanía Sampedro Sánchez, se desarrolló con el fin de permitir el proceso de rehabilitación de los músculos faciales en niños con Parálisis Cerebral Infantil (PCI). SONRIE se compone de una plataforma de juegos cuyo objetivo es lograr una mejora terapéutica en la musculatura orofacial de niños diagnosticados de PCI con edades comprendidas entre los 4 y los 12 años. El escenario de aplicación del sistema SONRIE son las escuelas de integración que tienen escolarizados alumnos diagnosticados con este trastorno. La posibilidad de rehabilitación de los músculos faciales mediante tratamientos que se apoyan en el uso de sistemas telemáticos, junto con el empleo de tecnologías actuales (Realidad Virtual, Realidad Aumentada y Serious Games) supone una gran innovación en el entorno de la neuro-rehabilitación, entendida como el proceso de terapia que permite optimizar la participación de una persona en la sociedad, alcanzando un grado de bienestar óptimo. El trabajo realizado en este Proyecto Fin de Grado pretende escalar el sistema SONRIE, mediante el análisis, diseño y desarrollo de un Framework encargado de facilitar, ampliar y validar el uso adecuado del sistema SONRIE en entornos escolares a través de la integración de nuevas tecnologías. La plataforma desarrollada en este proyecto, permite dotar de dinamismo y persistencia a la plataforma de juegos, ofreciendo a los usuarios de SONRIE (principalmente fisioterapeutas y rehabilitadores que trabajan en entornos escolares) un sistema de terapia para niños con PCI accesible vía web. En este Proyecto Fin de Grado se describe el conjunto de componentes software desarrollados con el fin de proporcionar un entorno web que escale el sistema SONRIE, convirtiéndolo en un sistema de terapia efectivo, completo y usable. ABSTRACT. The SONRIE system (Sistema de terapia, basadO en KiNect, paRa nIños con parálisis cErebral), performed as a final project by Miss Estefanía Sampedro, was developed in order to allow the rehabilitation process of the facial muscles of children with Cerebral Palsy (CP). SONRIE consists of a gaming platform which aims to achieve a therapeutic improvement in the orofacial musculature on children diagnosed with CP aged between 4 and 12 years. The application scenario of the SONRIE system are the integration schools that have students diagnosed with this disorder. The possibility of rehabilitation of facial muscles through treatments based on the use of telematics systems, together with the use of new technologies (Virtual Reality, Augmented Reality and Serious Games) is a great innovation in the neuro-rehabilitation environment, understood as the therapy process that optimizes the participation of a person in the society, reaching an optimum level of welfare. The work done in this final project aims to scale the SONRIE system, through the analysis, design and development of a framework in charge of facilitating, extending and validating the proper use of the SONRIE system in school environments, through the integration of new technologies. The platform developed in this project, can provide dynamism and persistence to the gaming platform, offering to the SONRIE users (mainly physiotherapists and rehabilitators who work in school settings) a therapy system for children with CP accessible via web. In this final project are described the software components developed in order to provide a web environment that scales the SONRIE system, making it an effective, complete and usable therapy system.

Diseño e implementación de una aplicación de detección como elemento de realidad aumentada en automóviles

En este proyecto se realiza el diseño y la implementación de un sistema que genera realidad aumentada, mediante la detección de vehículos que se encuentran en una trayectoria, y sirve de base para la conducción autónoma en vehículospara ambientes nocturnos ya que se ha observado mediante el estudio del arte que no existen aplicaciones de este tipo en dichas condiciones. La implementación de dicho sistema se realiza mediante una aplicación móvil en el sistema operativo de Android, que se apoya en sus librerías para el uso de sensores y la creación de menús, y las de OpenCV para el tratamiento de las imágenes. Además, se han realizado una serie de pruebas para demostrar la validez y la eficiencia de dicho algoritmo y se presenta al usuario mediante una aplicación de fácil manejo y uso en un dispositivo móvil. ABSTRACT. This project is about the design and implemantation of a system which generates augmented reality by detecting vehicles that stand along a followed trayectory, working out the basis for autonomus driving in night environments, because it was noticed that any other applications exist for this particular purpose, under the given circumstances. Implementation works through an Android mobile application, and learns over this operative system libraries in order to work with sensors, menu configurations, and OpenCV for image processing. A number of tests were run to prove the algorithm right and efficient; and it is introduced to the users via an easy-to-use app on a mobile device.

Diseño e implementación de una aplicación de detección como elemento de realidad aumentada en automóviles

En este proyecto se realiza el diseño y la implementación de un sistema que genera realidad aumentada, mediante la detección de vehículos que se encuentran en una trayectoria, y sirve de base para la conducción autónoma en vehículospara ambientes nocturnos ya que se ha observado mediante el estudio del arte que no existen aplicaciones de este tipo en dichas condiciones. La implementación de dicho sistema se realiza mediante una aplicación móvil en el sistema operativo de Android, que se apoya en sus librerías para el uso de sensores y la creación de menús, y las de OpenCV para el tratamiento de las imágenes. Además, se han realizado una serie de pruebas para demostrar la validez y la eficiencia de dicho algoritmo y se presenta al usuario mediante una aplicación de fácil manejo y uso en un dispositivo móvil. ABSTRACT. This project is about the design and implemantation of a system which generates augmented reality by detecting vehicles that stand along a followed trayectory, working out the basis for autonomus driving in night environments, because it was noticed that any other applications exist for this particular purpose, under the given circumstances. Implementation works through an Android mobile application, and learns over this operative system libraries in order to work with sensors, menu configurations, and OpenCV for image processing. A number of tests were run to prove the algorithm right and efficient; and it is introduced to the users via an easy-to-use app on a mobile device.

Simbologia em realidade aumentada móvel

Tese de mestrado, Engenharia Informática (Sistemas de Informação), Universidade de Lisboa, Faculdade de Ciências, 2016

A distributed systems approach to secure Internet mail

One of the obstacles to improved security of the Internet is ad hoc development of technologies with different design goals and different security goals. This paper proposes reconceptualizing the Internet as a secure distributed system, focusing specifically on the application layer. The notion is to redesign specific functionality, based on principles discovered in research on distributed systems in the decades since the initial development of the Internet. Because of the problems in retrofitting new technology across millions of clients and servers, any options with prospects of success must support backward compatibility. This paper outlines a possible new architecture for internet-based mail which would replace existing protocols by a more secure framework. To maintain backward compatibility, initial implementation could offer a web browser-based front end but the longer-term approach would be to implement the system using appropriate models of replication. (C) 2005 Elsevier Ltd. All rights reserved.

Middleware for distributed context-aware systems

Context-aware systems represent extremely complex and heterogeneous distributed systems, composed of sensors, actuators, application components, and a variety of context processing components that manage the flow of context information between the sensors/actuators and applications. The need for middleware to seamlessly bind these components together is well recognised. Numerous attempts to build middleware or infrastructure for context-aware systems have been made, but these have provided only partial solutions; for instance, most have not adequately addressed issues such as mobility, fault tolerance or privacy. One of the goals of this paper is to provide an analysis of the requirements of a middleware for context-aware systems, drawing from both traditional distributed system goals and our experiences with developing context-aware applications. The paper also provides a critical review of several middleware solutions, followed by a comprehensive discussion of our own PACE middleware. Finally, it provides a comparison of our solution with the previous work, highlighting both the advantages of our middleware and important topics for future research.

Filtering duplicate items over distributed data streams

In recent years many real time applications need to handle data streams. We consider the distributed environments in which remote data sources keep on collecting data from real world or from other data sources, and continuously push the data to a central stream processor. In these kinds of environments, significant communication is induced by the transmitting of rapid, high-volume and time-varying data streams. At the same time, the computing overhead at the central processor is also incurred. In this paper, we develop a novel filter approach, called DTFilter approach, for evaluating the windowed distinct queries in such a distributed system. DTFilter approach is based on the searching algorithm using a data structure of two height-balanced trees, and it avoids transmitting duplicate items in data streams, thus lots of network resources are saved. In addition, theoretical analysis of the time spent in performing the search, and of the amount of memory needed is provided. Extensive experiments also show that DTFilter approach owns high performance.

The design of fault tolerant software for loosely coupled distributed systems

Requirements for systems to continue to operate satisfactorily in the presence of faults has led to the development of techniques for the construction of fault tolerant software. This thesis addresses the problem of error detection and recovery in distributed systems which consist of a set of communicating sequential processes. A method is presented for the `a priori' design of conversations for this class of distributed system. Petri nets are used to represent the state and to solve state reachability problems for concurrent systems. The dynamic behaviour of the system can be characterised by a state-change table derived from the state reachability tree. Systematic conversation generation is possible by defining a closed boundary on any branch of the state-change table. By relating the state-change table to process attributes it ensures all necessary processes are included in the conversation. The method also ensures properly nested conversations. An implementation of the conversation scheme using the concurrent language occam is proposed. The structure of the conversation is defined using the special features of occam. The proposed implementation gives a structure which is independent of the application and is independent of the number of processes involved. Finally, the integrity of inter-process communications is investigated. The basic communication primitives used in message passing systems are seen to have deficiencies when applied to systems with safety implications. Using a Petri net model a boundary for a time-out mechanism is proposed which will increase the integrity of a system which involves inter-process communications.

An indexing structure and application model for vehicles moving on road networks

Moving objects database systems are the most challenging sub-category among Spatio-Temporal database systems. A database system that updates in real-time the location information of GPS-equipped moving vehicles has to meet even stricter requirements. Currently existing data storage models and indexing mechanisms work well only when the number of moving objects in the system is relatively small. This dissertation research aimed at the real-time tracking and history retrieval of massive numbers of vehicles moving on road networks. A total solution has been provided for the real-time update of the vehicles' location and motion information, range queries on current and history data, and prediction of vehicles' movement in the near future. ^ To achieve these goals, a new approach called Segmented Time Associated to Partitioned Space (STAPS) was first proposed in this dissertation for building and manipulating the indexing structures for moving objects databases. ^ Applying the STAPS approach, an indexing structure of associating a time interval tree to each road segment was developed for real-time database systems of vehicles moving on road networks. The indexing structure uses affordable storage to support real-time data updates and efficient query processing. The data update and query processing performance it provides is consistent without restrictions such as a time window or assuming linear moving trajectories. ^ An application system design based on distributed system architecture with centralized organization was developed to maximally support the proposed data and indexing structures. The suggested system architecture is highly scalable and flexible. Finally, based on a real-world application model of vehicles moving in region-wide, main issues on the implementation of such a system were addressed. ^

An Indexing Structure and Application Model for Vehicles Moving on Road Networks

Moving objects database systems are the most challenging sub-category among Spatio-Temporal database systems. A database system that updates in real-time the location information of GPS-equipped moving vehicles has to meet even stricter requirements. Currently existing data storage models and indexing mechanisms work well only when the number of moving objects in the system is relatively small. This dissertation research aimed at the real-time tracking and history retrieval of massive numbers of vehicles moving on road networks. A total solution has been provided for the real-time update of the vehicles’ location and motion information, range queries on current and history data, and prediction of vehicles’ movement in the near future. To achieve these goals, a new approach called Segmented Time Associated to Partitioned Space (STAPS) was first proposed in this dissertation for building and manipulating the indexing structures for moving objects databases. Applying the STAPS approach, an indexing structure of associating a time interval tree to each road segment was developed for real-time database systems of vehicles moving on road networks. The indexing structure uses affordable storage to support real-time data updates and efficient query processing. The data update and query processing performance it provides is consistent without restrictions such as a time window or assuming linear moving trajectories. An application system design based on distributed system architecture with centralized organization was developed to maximally support the proposed data and indexing structures. The suggested system architecture is highly scalable and flexible. Finally, based on a real-world application model of vehicles moving in region-wide, main issues on the implementation of such a system were addressed.

Studio del modello Augmented Worlds per l'ingegneria di sistemi software basati su realtà aumentata: esplorazioni basate su Unity e Android

Negli ultimi anni il crescere della capacità di calcolo dei dispositivi e il diminuire delle loro dimensioni ha permesso di far nascere idee innovative e di esplorare più in dettaglio alcuni settori. Uno di questi è sicuramente quello della realtà aumentata (Augmented reality), infatti, la discussione su questo argomento nasce già negli anni 40 del novecento, ma, per mancanza di mezzi tecnologici adeguati, solo ora si iniziano a realizzare le prime applicazioni che si basano su questa idea e il grande pubblico inizia ad interessarsi all'argomento. La costruzione di applicazioni di realtà aumentata, al momento, è basata sull'utilizzo di alcuni framework che mettono a disposizione dello sviluppatore alcune funzioni molto comuni in questi software, come il tracking di marker e l'utilizzo di bottoni virtuali. Questi strumenti, seppur comodi, non garantiscono sempre la buona progettazione dell'applicazione e tendono a unire insieme parti di logica applicativa e di grafica. Per questo motivo, anche nella ricerca, si stanno cercando di studiare dei metodi in grado di permettere una divisione ottimale dei compiti in modo da ottenere un software riusabile e facilmente mantenibile, ma che permetta anche di sfruttare appieno le potenzialità dell'AR attraverso, per esempio, sistemi distribuiti. Un framework concettuale che rientra in questa categoria è sicuramente quello degli Augmented Worlds, mondi virtuali collegati a quello fisico che ne incrementano le caratteristiche e le possibilità tramite la presenza di entità aumentate. La tesi, quindi, si propone di sviluppare un prototipo di un framework con le caratteristiche sopra citate di estendibilità, utilizzando le piattaforme in questo momento a disposizione e ispirandosi alla visione degli Augmented Worlds.