Automatic Orientation of Multi-Scale Terrestrial Images for 3D Reconstruction

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Diseño, optimización y análisis de sistemas basados en técnicas láser, para el modelado geométrico, registro y documentación, aplicados a entidades de interés patrimonial.

El objetivo de la presente tesis doctoral es el desarrollo e implementación de un sistema para mejorar la metodología de extracción de la información geométrica necesaria asociada a los procesos de documentación de entidades de interés patrimonial, a partir de la información proporcionada por el empleo de sensores láser, tanto aéreos como terrestres. Para ello, inicialmente se realiza una presentación y justificación de los antecedentes y la problemática en el registro de información geométrica para el patrimonio, detallando todos aquellos sistemas de registro y análisis de la información geométrica utilizados en la actualidad. Este análisis permitirá realizar la comparación con los sistemas de registro basados en técnicas láser, aportando sugerencias de utilización para cada caso concreto. Posteriormente, se detallan los sistemas de registro basados en técnicas láser, comenzando por los sensores aerotransportados y concluyendo con el análisis pormenorizado de los sensores terrestres, tanto en su aplicación en modo estático como móvil. Se exponen las características técnicas y funcionamiento de cada uno de ellos, así como los ámbitos de aplicación y productos generados. Se analizan las fuentes de error que determinan la precisión que puede alcanzar el sistema. Tras la exposición de las características de los sistemas LiDAR, se detallan los procesos a realizar con los datos extraídos para poder generar la información necesaria para los diferentes tipos de objetos analizados. En esta exposición, se hace hincapié en los posibles riesgos que pueden ocurrir en algunas fases delicadas y se analizarán los diferentes algoritmos de filtrado y clasificación de los puntos, fundamentales en el procesamiento de la información LiDAR. Seguidamente, se propone una alternativa para optimizar los modelos de procesamiento existentes, basándose en el desarrollo de algoritmos nuevos y herramientas informáticas que mejoran el rendimiento en la gestión de la información LiDAR. En la implementación, se han tenido en cuenta características y necesidades particulares de la documentación de entidades de interés patrimonial, así como los diferentes ámbitos de utilización del LiDAR, tanto aéreo como terrestre. El resultado es un organigrama de las tareas a realizar desde la nube de puntos LiDAR hasta el cálculo de los modelos digitales del terreno y de superficies. Para llevar a cabo esta propuesta, se han desarrollado hasta 19 algoritmos diferentes que comprenden implementaciones para el modelado en 2.5D y 3D, visualización, edición, filtrado y clasificación de datos LiDAR, incorporación de información de sensores pasivos y cálculo de mapas derivados, tanto raster como vectoriales, como pueden ser mapas de curvas de nivel y ortofotos. Finalmente, para dar validez y consistencia a los desarrollos propuestos, se han realizado ensayos en diferentes escenarios posibles en un proceso de documentación del patrimonio y que abarcan desde proyectos con sensores aerotransportados, proyectos con sensores terrestres estáticos a media y corta distancia, así como un proyecto con un sensor terrestre móvil. Estos ensayos han permitido definir los diferentes parámetros necesarios para el adecuado funcionamiento de los algoritmos propuestos. Asimismo, se han realizado pruebas objetivas expuestas por la ISPRS para la evaluación y comparación del funcionamiento de algoritmos de clasificación LiDAR. Estas pruebas han permitido extraer datos de rendimiento y efectividad del algoritmo de clasificación presentado, permitiendo su comparación con otros algoritmos de prestigio existentes. Los resultados obtenidos han constatado el funcionamiento satisfactorio de la herramienta. Esta tesis está enmarcada dentro del proyecto Consolider-Ingenio 2010: “Programa de investigación en tecnologías para la valoración y conservación del patrimonio cultural” (ref. CSD2007-00058) realizado por el Consejo Superior de Investigaciones Científicas y la Universidad Politécnica de Madrid. ABSTRACT: The goal of this thesis is the design, development and implementation of a system to improve the extraction of useful geometric information in Heritage documentation processes. This system is based on information provided by laser sensors, both aerial and terrestrial. Firstly, a presentation of recording geometric information for Heritage processes is done. Then, a justification of the background and problems is done too. Here, current systems for recording and analyzing the geometric information are studied. This analysis will perform the comparison with the laser system techniques, providing suggestions of use for each specific case. Next, recording systems based on laser techniques are detailed. This study starts with airborne sensors and ends with terrestrial ones, both in static and mobile application. The technical characteristics and operation of each of them are described, as well as the areas of application and generated products. Error sources are also analyzed in order to know the precision this technology can achieve. Following the presentation of the LiDAR system characteristics, the processes to generate the required information for different types of scanned objects are described; the emphasis is on the potential risks that some steps can produce. Moreover different filtering and classification algorithms are analyzed, because of their main role in LiDAR processing. Then, an alternative to optimize existing processing models is proposed. It is based on the development of new algorithms and tools that improve the performance in LiDAR data management. In this implementation, characteristics and needs of the documentation of Heritage entities have been taken into account. Besides, different areas of use of LiDAR are considered, both air and terrestrial. The result is a flowchart of tasks from the LiDAR point cloud to the calculation of digital terrain models and digital surface models. Up to 19 different algorithms have been developed to implement this proposal. These algorithms include implementations for 2.5D and 3D modeling, viewing, editing, filtering and classification of LiDAR data, incorporating information from passive sensors and calculation of derived maps, both raster and vector, such as contour maps and orthophotos. Finally, in order to validate and give consistency to the proposed developments, tests in different cases have been executed. These tests have been selected to cover different possible scenarios in the Heritage documentation process. They include from projects with airborne sensors, static terrestrial sensors (medium and short distances) to mobile terrestrial sensor projects. These tests have helped to define the different parameters necessary for the appropriate functioning of the proposed algorithms. Furthermore, proposed tests from ISPRS have been tested. These tests have allowed evaluating the LiDAR classification algorithm performance and comparing it to others. Therefore, they have made feasible to obtain performance data and effectiveness of the developed classification algorithm. The results have confirmed the reliability of the tool. This investigation is framed within Consolider-Ingenio 2010 project titled “Programa de investigación en tecnologías para la valoración y conservación del patrimonio cultural” (ref. CSD2007-00058) by Consejo Superior de Investigaciones Científicas and Universidad Politécnica de Madrid.

Overlapping Range Images using Genetics Algorithms

El artículo aborda el problema del encaje de diversas imágenes de una misma escena capturadas por escáner 3d para generar un único modelo tridimensional. Para ello se utilizaron algoritmos genéticos. ABSTRACT: This work introduces a solution based on genetic algorithms to find the overlapping area between two point cloud captures obtained from a three-dimensional scanner. Considering three translation coordinates and three rotation angles, the genetic algorithm evaluates the matching points in the overlapping area between the two captures given that transformation. Genetic simulated annealing is used to improve the accuracy of the results obtained by the genetic algorithm.

Deciduous tree reconstruction algorithm based on cylinder fitting from mobile terrestrial laser scanned point clouds

Vector reconstruction of objects from an unstructured point cloud obtained with a LiDAR-based system (light detection and ranging) is one of the most promising methods to build three dimensional models of orchards. The cylinder fitting method for woody structure reconstruction of leafless trees from point clouds obtained with a mobile terrestrial laser scanner (MTLS) has been analysed. The advantage of this method is that it performs reconstruction in a single step. The most time consuming part of the algorithm is generation of the cylinder direction, which must be recalculated at the inclusion of each point in the cylinder. The tree skeleton is obtained at the same time as the cluster of cylinders is formed. The method does not guarantee a unique convergence and the reconstruction parameter values must be carefully chosen. A balanced processing of clusters has also been defined which has proven to be very efficient in terms of processing time by following the hierarchy of branches, predecessors and successors. The algorithm was applied to simulated MTLS of virtual orchard models and to MTLS data of real orchards. The constraints applied in the method have been reviewed to ensure better convergence and simpler use of parameters. The results obtained show a correct reconstruction of the woody structure of the trees and the algorithm runs in linear logarithmic time

3DCOMET: 3D compression methods test dataset

The use of 3D data in mobile robotics applications provides valuable information about the robot’s environment. However usually the huge amount of 3D information is difficult to manage due to the fact that the robot storage system and computing capabilities are insufficient. Therefore, a data compression method is necessary to store and process this information while preserving as much information as possible. A few methods have been proposed to compress 3D information. Nevertheless, there does not exist a consistent public benchmark for comparing the results (compression level, distance reconstructed error, etc.) obtained with different methods. In this paper, we propose a dataset composed of a set of 3D point clouds with different structure and texture variability to evaluate the results obtained from 3D data compression methods. We also provide useful tools for comparing compression methods, using as a baseline the results obtained by existing relevant compression methods.

Point clouds of measurements in the Dechen Cave near Iserlohn, Germany

Modeling natural phenomena from 3D information enhances our understanding of the environment. Dense 3D point clouds are increasingly used as highly detailed input datasets. In addition to the capturing techniques of point clouds with LiDAR, low-cost sensors have been released in the last few years providing access to new research fields and facilitating 3D data acquisition for a broader range of applications. This letter presents an analysis of different speleothem features using 3D point clouds acquired with the gaming device Microsoft® Kinect. We compare the Kinect sensor with terrestrial LiDAR reference measurements using the KinFu pipeline for capturing complete 3D objects (< 4m**3). The results demonstrate the suitability of the Kinect to capture flowstone walls and to derive morphometric parameters of cave features. Although the chosen capturing strategy (KinFu) reveals a high correlation (R2=0.92) of stalagmite morphometry along the vertical object axis, a systematic overestimation (22% for radii and 44% for volume) is found. The comparison of flowstone wall datasets predominantly shows low differences (mean of 1 mm with 7 mm standard deviation) of the order of the Kinect depth precision. For both objects the major differences occur at strongly varying and curved surface structures (e.g. with fine concave parts).

Modelo para reconstrução 3D de cenas baseado em imagens

3D Reconstruction is the process used to obtain a detailed graphical model in three dimensions that represents some real objectified scene. This process uses sequences of images taken from the scene, so it can automatically extract the information about the depth of feature points. These points are then highlighted using some computational technique on the images that compose the used dataset. Using SURF feature points this work propose a model for obtaining depth information of feature points detected by the system. At the ending, the proposed system extract three important information from the images dataset: the 3D position for feature points; relative rotation and translation matrices between images; the realtion between the baseline for adjacent images and the 3D point accuracy error found.

Creazione di una costmap 2D per la navigazione di un rover terrestre

L’obiettivo di questa tesi è riuscire ad elaborare una point cloud 3D proveniente dal laser scanner per individuare possibili ostacoli e creare con essa, successivamente, una mappa che permetta la navigazione di un rover.

Impact of Curling and Warping on Concrete Pavement, TR-668, 2016

Portland cement concrete (PCC) pavement undergoes repeated environmental load-related deflection resulting from temperature and moisture variations across the pavement depth. This phenomenon, referred to as PCC pavement curling and warping, has been known and studied since the mid-1920s. Slab curvature can be further magnified under repeated traffic loads and may ultimately lead to fatigue failures, including top-down and bottom-up transverse, longitudinal, and corner cracking. It is therefore important to measure the “true” degree of curling and warping in PCC pavements, not only for quality control (QC) and quality assurance (QA) purposes, but also to achieve a better understanding of its relationship to long-term pavement performance. In order to better understand the curling and warping behavior of PCC pavements in Iowa and provide recommendations to mitigate curling and warping deflections, field investigations were performed at six existing sites during the late fall of 2015. These sites included PCC pavements with various ages, slab shapes, mix design aspects, and environmental conditions during construction. A stationary light detection and ranging (LiDAR) device was used to scan the slab surfaces. The degree of curling and warping along the longitudinal, transverse, and diagonal directions was calculated for the selected slabs based on the point clouds acquired using LiDAR. The results and findings are correlated to variations in pavement performance, mix design, pavement design, and construction details at each site. Recommendations regarding how to minimize curling and warping are provided based on a literature review and this field study. Some examples of using point cloud data to build three-dimensional (3D) models of the overall curvature of the slab shape are presented to show the feasibility of using this 3D analysis method for curling and warping analysis.

ViDRILO: The Visual and Depth Robot Indoor Localization with Objects information dataset

In this article we describe a semantic localization dataset for indoor environments named ViDRILO. The dataset provides five sequences of frames acquired with a mobile robot in two similar office buildings under different lighting conditions. Each frame consists of a point cloud representation of the scene and a perspective image. The frames in the dataset are annotated with the semantic category of the scene, but also with the presence or absence of a list of predefined objects appearing in the scene. In addition to the frames and annotations, the dataset is distributed with a set of tools for its use in both place classification and object recognition tasks. The large number of labeled frames in conjunction with the annotation scheme make this dataset different from existing ones. The ViDRILO dataset is released for use as a benchmark for different problems such as multimodal place classification and object recognition, 3D reconstruction or point cloud data compression.

Classification of Airborne LIDAR Intensity Data Using Statistical Analysis and Hough Transform with Application to Power Line Corridors

Light Detection and Ranging (LIDAR) has great potential to assist vegetation management in power line corridors by providing more accurate geometric information of the power line assets and vegetation along the corridors. However, the development of algorithms for the automatic processing of LIDAR point cloud data, in particular for feature extraction and classification of raw point cloud data, is in still in its infancy. In this paper, we take advantage of LIDAR intensity and try to classify ground and non-ground points by statistically analyzing the skewness and kurtosis of the intensity data. Moreover, the Hough transform is employed to detected power lines from the filtered object points. The experimental results show the effectiveness of our methods and indicate that better results were obtained by using LIDAR intensity data than elevation data.

Multi-level information fusion for environment aware robotic navigation

This thesis develops the hardware and software framework for an integrated navigation system. Dynamic data fusion algorithms are used to develop a system with a high level of resistance to the typical problems that affect standard navigation systems.

BDNF gene effects on brain circuitry replicated in 455 twins

Brain-derived neurotrophic factor (BDNF) plays a key role in learning and memory, but its effects on the fiber architecture of the living brain are unknown. We genotyped 455 healthy adult twins and their non-twin siblings (188 males/267 females; age: 23.7 ± 2.1. years, mean ± SD) and scanned them with high angular resolution diffusion tensor imaging (DTI), to assess how the BDNF Val66Met polymorphism affects white matter microstructure. By applying genetic association analysis to every 3D point in the brain images, we found that the Val-BDNF genetic variant was associated with lower white matter integrity in the splenium of the corpus callosum, left optic radiation, inferior fronto-occipital fasciculus, and superior corona radiata. Normal BDNF variation influenced the association between subjects' performance intellectual ability (as measured by Object Assembly subtest) and fiber integrity (as measured by fractional anisotropy; FA) in the callosal splenium, and pons. BDNF gene may affect the intellectual performance by modulating the white matter development. This combination of genetic association analysis and large-scale diffusion imaging directly relates a specific gene to the fiber microstructure of the living brain and to human intelligence.

LiDAR Augmented Optimal Path Planning for Unmanned Aerial Vehicle during Remote Sensing at Low Altitude and Network Sensor Data Acquisition

This technical report describes a Light Detection and Ranging (LiDAR) augmented optimal path planning at low level flight methodology for remote sensing and sampling Unmanned Aerial Vehicles (UAV). The UAV is used to perform remote air sampling and data acquisition from a network of sensors on the ground. The data that contains information on the terrain is in the form of a 3D point clouds maps is processed by the algorithms to find an optimal path. The results show that the method and algorithm are able to use the LiDAR data to avoid obstacles when planning a path from a start to a target point. The report compares the performance of the method as the resolution of the LIDAR map is increased and when a Digital Elevation Model (DEM) is included. From a practical point of view, the optimal path plan is loaded and works seemingly with the UAV ground station and also shows the UAV ground station software augmented with more accurate LIDAR data.

Avaliação de correlações para propriedades físico-químicas de frações de petróleo

O petróleo é uma mistura complexa consistindo em um número muito grande de hidrocarbonetos. A descrição completa de todos os hidrocarbonetos existentes nessas misturas é inviável experimentalmente ou consome tempo excessivo em simulações computacionais. Por esta razão, uma abordagem molecular completa para cálculos de propriedades dessas misturas é substituído por uma abordagem pseudo-componente ou por correlações entre propriedades macroscópicas. Algumas dessas propriedades são utilizadas de acordo com a regulamentação de venda de combustíveis, e.g., para gasolina. Dependendo do esquema de refino e do óleo cru utilizado para produção desse combustível, uma larga variedade de valores é encontrada para as propriedades de correntes de processo que compõe o combustível final. A fim de planejar com precisão adequada a mistura dessas correntes, modelos devem estar disponíveis para o cálculo preciso das propriedades necessárias. Neste trabalho, oito séries de combustíveis brasileiros e duas séries de combustíveis estrangeiros foram analisadas: frações de gasolina, querosene, gasóleo e diesel. As propriedades analisadas para as frações são: número de octano, teor de aromáticos, teor de enxofre, índice de refração, densidade, ponto de fulgor, ponto de fluidez, ponto de congelamento, ponto de névoa, ponto de anilina, pressão de vapor Reid e número de cetano. Diversas correlações foram avaliadas e os melhores desempenhos foram destacados, permitindo uma estimação precisa das propriedades do combustível avaliado. Um processo de re-estimação de parâmetros foi aplicado e novos modelos foram ajustados em comparação com os dados experimentais. Esta estratégia permitiu uma estimativa mais exata das propriedades analisadas, sendo verificada por um aumento considerável no desempenho estatístico dos modelos. Além disso, foi apresentado o melhor modelo para cada propriedade e cada série