Ricostruzione 3d da immagini rgb-d su piattaforma mobile

Viene proposto un porting su piattaforma mobile Android di un sistema SLAM (Simultaneous Localization And Mapping) chiamato SlamDunk. Il porting affronta problematiche di prestazioni e qualità delle ricostruzioni 3D ottenute, proponendo poi la soluzione ritenuta ottimale.

An improvement of a SLAM RGB-D method with movement prediction derived from a study of visual features

This paper presents a method for the fast calculation of a robot’s egomotion using visual features. The method is part of a complete system for automatic map building and Simultaneous Location and Mapping (SLAM). The method uses optical flow to determine whether the robot has undergone a movement. If so, some visual features that do not satisfy several criteria are deleted, and then egomotion is calculated. Thus, the proposed method improves the efficiency of the whole process because not all the data is processed. We use a state-of-the-art algorithm (TORO) to rectify the map and solve the SLAM problem. Additionally, a study of different visual detectors and descriptors has been conducted to identify which of them are more suitable for the SLAM problem. Finally, a navigation method is described using the map obtained from the SLAM solution.

Towards a real-time 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors

In this project, we propose the implementation of a 3D object recognition system which will be optimized to operate under demanding time constraints. The system must be robust so that objects can be recognized properly in poor light conditions and cluttered scenes with significant levels of occlusion. An important requirement must be met: the system must exhibit a reasonable performance running on a low power consumption mobile GPU computing platform (NVIDIA Jetson TK1) so that it can be integrated in mobile robotics systems, ambient intelligence or ambient assisted living applications. The acquisition system is based on the use of color and depth (RGB-D) data streams provided by low-cost 3D sensors like Microsoft Kinect or PrimeSense Carmine. The range of algorithms and applications to be implemented and integrated will be quite broad, ranging from the acquisition, outlier removal or filtering of the input data and the segmentation or characterization of regions of interest in the scene to the very object recognition and pose estimation. Furthermore, in order to validate the proposed system, we will create a 3D object dataset. It will be composed by a set of 3D models, reconstructed from common household objects, as well as a handful of test scenes in which those objects appear. The scenes will be characterized by different levels of occlusion, diverse distances from the elements to the sensor and variations on the pose of the target objects. The creation of this dataset implies the additional development of 3D data acquisition and 3D object reconstruction applications. The resulting system has many possible applications, ranging from mobile robot navigation and semantic scene labeling to human-computer interaction (HCI) systems based on visual information.

Scene Reconstruction And Understanding By RGB-D Sensors

This thesis investigates interactive scene reconstruction and understanding using RGB-D data only. Indeed, we believe that depth cameras will still be in the near future a cheap and low-power 3D sensing alternative suitable for mobile devices too. Therefore, our contributions build on top of state-of-the-art approaches to achieve advances in three main challenging scenarios, namely mobile mapping, large scale surface reconstruction and semantic modeling. First, we will describe an effective approach dealing with Simultaneous Localization And Mapping (SLAM) on platforms with limited resources, such as a tablet device. Unlike previous methods, dense reconstruction is achieved by reprojection of RGB-D frames, while local consistency is maintained by deploying relative bundle adjustment principles. We will show quantitative results comparing our technique to the state-of-the-art as well as detailed reconstruction of various environments ranging from rooms to small apartments. Then, we will address large scale surface modeling from depth maps exploiting parallel GPU computing. We will develop a real-time camera tracking method based on the popular KinectFusion system and an online surface alignment technique capable of counteracting drift errors and closing small loops. We will show very high quality meshes outperforming existing methods on publicly available datasets as well as on data recorded with our RGB-D camera even in complete darkness. Finally, we will move to our Semantic Bundle Adjustment framework to effectively combine object detection and SLAM in a unified system. Though the mathematical framework we will describe does not restrict to a particular sensing technology, in the experimental section we will refer, again, only to RGB-D sensing. We will discuss successful implementations of our algorithm showing the benefit of a joint object detection, camera tracking and environment mapping.

Using a RGB-D camera for 6DoF SLAM

This paper presents a method for fast calculation of the egomotion done by a robot using visual features. The method is part of a complete system for automatic map building and Simultaneous Localization and Mapping (SLAM). The method uses optical flow in order to determine if the robot has done a movement. If so, some visual features which do not accomplish several criteria (like intersection, unicity, etc,) are deleted, and then the egomotion is calculated. We use a state-of-the-art algorithm (TORO) in order to rectify the map and solve the SLAM problem. The proposed method provides better efficiency that other current methods.

Real-time Image-based RGB-D Camera Motion Tracking and Environment Mapping

In this work, image based estimation methods, also known as direct methods, are studied which avoid feature extraction and matching completely. Cost functions use raw pixels as measurements and the goal is to produce precise 3D pose and structure estimates. The cost functions presented minimize the sensor error, because measurements are not transformed or modified. In photometric camera pose estimation, 3D rotation and translation parameters are estimated by minimizing a sequence of image based cost functions, which are non-linear due to perspective projection and lens distortion. In image based structure refinement, on the other hand, 3D structure is refined using a number of additional views and an image based cost metric. Image based estimation methods are particularly useful in conditions where the Lambertian assumption holds, and the 3D points have constant color despite viewing angle. The goal is to improve image based estimation methods, and to produce computationally efficient methods which can be accomodated into real-time applications. The developed image-based 3D pose and structure estimation methods are finally demonstrated in practise in indoor 3D reconstruction use, and in a live augmented reality application.

Registro global de nuvens de pontos RGB-D em tempo real usando fluxo óptico e marcadores

Registration of point clouds captured by depth sensors is an important task in 3D reconstruction applications based on computer vision. In many applications with strict performance requirements, the registration should be executed not only with precision, but also in the same frequency as data is acquired by the sensor. This thesis proposes theuse of the pyramidal sparse optical flow algorithm to incrementally register point clouds captured by RGB-D sensors (e.g. Microsoft Kinect) in real time. The accumulated errorinherent to the process is posteriorly minimized by utilizing a marker and pose graph optimization. Experimental results gathered by processing several RGB-D datasets validatethe system proposed by this thesis in visual odometry and simultaneous localization and mapping (SLAM) applications.

Detección de Lugares con Camaras RGB-D. Aplicación a Cierre de Bucles en SLAM

En este trabajo se propone un método que combina descriptores de imágenes de intensidad y de profundidad para detectar de manera robusta el problema de cierre de bucle en SLAM. La robustez del método, proporcionada por el empleo conjunto de información de diversa naturaleza, permite detectar lugares revisitados en situaciones donde m´etodos basados solo en intensidad o en profundidad presentan dificultades (p.e. condiciones de iluminación deficientes, o falta de geometría). Además, se ha diseñado el métod cuenta su eficiencia, recurriendo para ello al detector FAST para extraer las características de las observaciones y al descriptor binario BRIEF. La detección de bucle se completa con una Bolsa de Palabras binarias. El rendimiento del método propuesto se ha evaluado en condiciones reales, obteniéndose resultados muy satisfactorios.

Análisis comparativo de métodos de calibrado para sensores RGB-D y su influencia en el registro de múltiples vistas

Los sensores de propósito general RGB-D son dispositivos capaces de proporcionar información de color y de profundidad de la escena. Debido al amplio rango de aplicación que tienen estos sensores, despiertan gran interés en múltiples áreas, provocando que en algunos casos funcionen al límite de sensibilidad. Los métodos de calibración resultan más importantes, si cabe, para este tipo de sensores para mejorar la precisión de los datos adquiridos. Por esta razón, resulta de enorme transcendencia analizar y estudiar el calibrado de estos sensores RGBD de propósito general. En este trabajo se ha realizado un estudio de las diferentes tecnologías empleadas para determinar la profundidad, siendo la luz estructurada y el tiempo de vuelo las más comunes. Además, se ha analizado y estudiado aquellos parámetros del sensor que influyen en la obtención de los datos con precisión adecuada dependiendo del problema a tratar. El calibrado determina, como primer elemento del proceso de visión, los parámetros característicos que definen un sistema de visión artificial, en este caso, aquellos que permiten mejorar la exactitud y precisión de los datos aportados. En este trabajo se han analizado tres algoritmos de calibración, tanto de propósito general como de propósito específico, para llevar a cabo el proceso de calibrado de tres sensores ampliamente utilizados: Microsoft Kinect, PrimeSense Carmine 1.09 y Microsoft Kinect v2. Los dos primeros utilizan la tecnología de luz estructurada para determinar la profundidad, mientras que el tercero utiliza tiempo de vuelo. La experimentación realizada permite determinar de manera cuantitativa la exactitud y la precisión de los sensores y su mejora durante el proceso de calibrado, aportando los mejores resultados para cada caso. Finalmente, y con el objetivo de mostrar el proceso de calibrado en un sistema de registro global, diferentes pruebas han sido realizadas con el método de registro µ-MAR. Se ha utilizado inspección visual para determinar el comportamiento de los datos de captura corregidos según los resultados de los diferentes algoritmos de calibrado. Este hecho permite observar la importancia de disponer de datos exactos para ciertas aplicaciones como el registro 3D de una escena.

Juguem amb la Wii!: telemesura d'acceleracions en 3D : desenvolupament d'un projecte informàtic de codi lliure

Treball de recerca realitzat per un alumne d'ensenyament secundari i guardonat amb un Premi CIRIT per fomentar l'esperit científic del Jovent l'any 2009. Aquest treball de recerca és un projecte sobre el disseny i la creació d’un programa informàtic de codi obert amb l’objectiu de mesurar acceleracions en tres dimensions utilitzant el comandament de la wii, també conegut com a wiimote. Per tant, s'ha creat un programa que es connecta amb el wiimote, en rep les dades, les guarda i les representa per analitzar posteriorment diversos tipus de moviments i les seves acceleracions. Per tal de fer això es va aprofitar una biblioteca de funcions de codi obert ja existent que aporta les funcions principals per a la comunicació i control del comandament. El codi obert és un concepte que s’utilitza per als projectes informàtics, el codi dels quals està a la disposició de qui el necessiti. La biblioteca utilitzada està escrita en llenguatge C i per a plataforma Linux, i per tal d’aprofitar-la es va haver d’aprendre a utilitzar tant el llenguatge com la plataforma ja que no s'hi havia treballat mai abans. Gràcies a aquest projecte s'ha tingut la possibilitat de veure el funcionament d’algunes tecnologies alternatives i veure’n els avantatges sobre les convencionals o propietàries. Així doncs, des del punt de vista de l'autor, ha estat útil i enriquidor el fet de realitzar-lo.

Desenvolupament d'un videojoc 3D basat en OpenGL

En aquest treball es presenta una aplicació gràfica per a desenvolupar un videojoc de tema lliure, amb premisses senzilles però fermes i amb la utilització d'unes eines el funcionament de les qual s'ha assolit durant la carrera i s?ha aprofundit durant el temps que ha durat la confecció d'aquest projecte.

Disseny i construcció d'un motor 3D en temps real per la Gameboy Advance

El projecte desenvolupat ha tractat l’estudi i disseny d’un motor 3D interactiu a la consolaGame Boy Advance (GBA). La GBA disposa d’un processador ARM7TDMI a 16’78Mhz i no disposa de operacions 3D per-hardware, és una consola lenta en comparació lesque podem trobar al mercat d’avui en dia. Aquest treball, va partir de la construcció d’un prototipus ray-casting per-columna. Després,vàrem adaptar-lo a una estructura de portals i sectors. Més tard,es va introduir el mapeig de sostre/terra i de paisatges. Per últim,vàrem introduir efectes a la renderització per donar més realismeal recorregut del món, com il·luminació, objectes, etc.Tot i que es va estudiar l’arquitectura d’un motor eficient, no estenia prou per arribar a tenir un motor interactiu. Una de lestasques més difícils va ser la part de optimització. Peraconseguir-ho s’ha hagut de substituir operacions a temps realcostoses a temps de execució, replantejar parts de l’algorisme per fer-lo més eficient, entre altres

Probabilistic position and orientation estimation of rigid objects using an RGB-D sensor

The recent emergence of low-cost RGB-D sensors has brought new opportunities for robotics by providing affordable devices that can provide synchronized images with both color and depth information. In this thesis, recent work on pose estimation utilizing RGBD sensors is reviewed. Also, a pose recognition system for rigid objects using RGB-D data is implemented. The implementation uses half-edge primitives extracted from the RGB-D images for pose estimation. The system is based on the probabilistic object representation framework by Detry et al., which utilizes Nonparametric Belief Propagation for pose inference. Experiments are performed on household objects to evaluate the performance and robustness of the system.

Construction d'un Atlas 3D numérique de la cornée humaine par recalage d'images

Nous proposons de construire un atlas numérique 3D contenant les caractéristiques moyennes et les variabilités de la morphologie d’un organe. Nos travaux seront appliqués particulièrement à la construction d'un atlas numérique 3D de la totalité de la cornée humaine incluant la surface antérieure et postérieure à partir des cartes topographiques fournies par le topographe Orbscan II. Nous procédons tout d'abord par normalisation de toute une population de cornées. Dans cette étape, nous nous sommes basés sur l'algorithme de recalage ICP (iterative closest point) pour aligner simultanément les surfaces antérieures et postérieures d'une population de cornée vers les surfaces antérieure et postérieure d'une cornée de référence. En effet, nous avons élaboré une variante de l'algorithme ICP adapté aux images (cartes) de cornées qui tient compte de changement d'échelle pendant le recalage et qui se base sur la recherche par voisinage via la distance euclidienne pour établir la correspondance entre les points. Après, nous avons procédé pour la construction de l'atlas cornéen par le calcul des moyennes des élévations de surfaces antérieures et postérieures recalées et leurs écarts-types associés. Une population de 100 cornées saines a été utilisée pour construire l'atlas cornéen normal. Pour visualiser l’atlas, on a eu recours à des cartes topographiques couleurs similairement à ce qu’offrent déjà les systèmes topographiques actuels. Enfin, des observations ont été réalisées sur l'atlas cornéen reflétant sa précision et permettant de développer une meilleure connaissance de l’anatomie cornéenne.