Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Mejoras y extensiones en el Front-End de un modelo de predicción de acciones de un estudiante en un laboratorio virtual 3D

Este documento presenta las mejoras y las extensiones introducidas en la herramienta de visualización del modelo predictivo del comportamiento del estudiante o Student Behavior Predictor Viewer (SBPV), implementada en un trabajo anterior. El modelo predictivo del comportamiento del estudiante es parte de un sistema inteligente de tutoría, y se construye a partir de los registros de actividad de los estudiantes en un laboratorio virtual 3D, como el Laboratorio Virtual de Biotecnología Agroforestal, implementado en un trabajo anterior, y cuyos registros de actividad de los estudiantes se han utilizado para validar este trabajo fin de grado. El SBPV es una herramienta para visualizar una representación gráfica 2D del grafo extendido asociado con cualquiera de los clusters del modelo predictivo del estudiante. Además de la visualización del grafo extendido, el SBPV controla la navegación a través del grafo por medio del navegador web. Más concretamente, el SBPV permite al usuario moverse a través del grafo, ampliar o reducir el zoom del gráfico o buscar un determinado estado. Además, el SBPV también permite al usuario modificar el diseño predeterminado del grafo en la pantalla al cambiar la posición de los estados con el ratón. Como parte de este trabajo fin de grado, se han corregido errores existentes en la versión anterior y se han introducido una serie de mejoras en el rendimiento y la usabilidad. En este sentido, se han implementado nuevas funcionalidades, tales como la visualización del modelo de comportamiento de cada estudiante individualmente o la posibilidad de elegir el método de clustering para crear el modelo predictivo del estudiante; así como ha sido necesario rediseñar la interfaz de usuario cambiando el tipo de estructuras gráficas con que se muestran los elementos del modelo y mejorando la visualización del grafo al interaccionar el usuario con él. Todas estas mejoras se explican detenidamente en el presente documento.---ABSTRACT---This document presents the improvements and extensions made to the visualization tool Student Behavior Predictor Viewer (SBPV), implemented in a previous job. The student behavior predictive model is part of an intelligent tutoring system, and is built from the records of students activity in a 3D virtual laboratory, like the “Virtual Laboratory of Agroforestry Biotechnology” implemented in a previous work, and whose records of students activity have been used to validate this final degree work. The SBPV is a tool for visualizing a 2D graphical representation of the extended graph associated with any of the clusters of the student predictive model. Apart from visualizing the extended graph, the SBPV supports the navigation across the graph by means of desktop devices. More precisely, the SBPV allows user to move through the graph, to zoom in/out the graphic or to locate a given state. In addition, the SBPV also allows user to modify the default layout of the graph on the screen by changing the position of the states by means of the mouse. As part of this work, some bugs of the previous version have been fixed and some enhancements have been implemented to improve the performance and the usability. In this sense, we have implemented new features, such as the display of the model behavior of only one student or the possibility of selecting the clustering method to create the student predictive model; as well as it was necessary to redesign the user interface changing the type of graphic structures that show model elements and improving the rendering of the graph when the user interacts with it. All these improvements are explained in detail in the next sections.

Estudo ecocardiográfico da função ventricular esquerda em pacientes com lúpus eritematoso sistêmico juvenil através da técnica de Speckle-Tracking bidimensional

Objetivo: O principal propósito do estudo foi pesquisar a disfunção ventricular esquerda subclínica em pacientes com lúpus eritematoso sistêmico juvenil (LESJ) através da técnica de speckle-tracking bidimensional. Foi investigada ainda uma possível correlação entre o comprometimento da deformação miocárdica e o SLEDAI-2K (Systemic Lupus Erithematosus Disease Activity Index 2000), bem como a presença de fatores de risco cardiovascular, tanto tradicionais como ligados à doença. Métodos: 50 pacientes assintomáticos do ponto de vista cardiovascular e 50 controles saudáveis (14,74 vs. 14,82 anos, p=0.83) foram avaliados pelo ecocardiograma convencional e pelo speckle-tracking bidimensional. Resultados: Apesar da fração de ejeção normal, os pacientes apresentaram redução de todos os parâmetros de deformação miocárdica longitudinal e radial, quando comparados aos controles: strain de pico sistólico longitudinal [-20,3 (-11 a -26) vs. -22 (-17,8 a -30.4) %, p < 0,0001], strain rate de pico sistólico longitudinal [-1,19 ± 0,21 vs. -1,3 ± 0,25 s-1, p=0,0005], strain rate longitudinal na diástole precoce [1,7 (0,99 a 2,95) vs. 2 (1,08 a 3,00) s-1 , p=0,0034], strain de pico sistólico radial [33,09 ± 8,6 vs. 44,36 ± 8,72%, p < 0,0001], strain rate de pico sistólico radial [1,98 ± 0,53 vs. 2,49 ± 0,68 s-1, p < 0,0001] e strain rate radial na diástole precoce [-2,31 ± 0,88 vs. -2,75 ± 0,97 s-1, p=0,02]. O strain de pico sistólico circunferencial [-23,67 ± 3,46 vs. - 24,6 ± 2,86%, p=0,43] e o strain rate circunferencial na diástole precoce [2 (0,88 a 3,4) vs. 1,99 (1,19 a 3,7) s-1, p=0,88] foram semelhantes em pacientes e controles. Apenas o strain rate de pico sistólico circunferencial [-1,5 ± 0,3 vs. -1,6 ± 0,3 s-1, p=0,036] mostrou-se reduzido no LESJ. Uma correlação negativa foi identificada entre o strain de pico sistólico longitudinal e o SLEDAI-2K (r = - 0,52; p < 0,0001) e também o número de fatores de risco cardiovascular por paciente (r = -0,32, p=0,024). Conclusões: Foi evidenciada disfunção sistólica e diastólica subclínica de ventrículo esquerdo no LESJ através da técnica de speckle-tracking bidimensional. A atividade da doença e a exposição aos fatores de risco cardiovascular provavelmente contribuíram para o comprometimento da deformação miocárdica nesses pacientes

Noise and PSRR improvement technique for TPC readout front-end in CMOS. technology.

ALICE is one of four major experiments of particle accelerator LHC installed in the European laboratory CERN. The management committee of the LHC accelerator has just approved a program update for this experiment. Among the upgrades planned for the coming years of the ALICE experiment is to improve the resolution and tracking efficiency maintaining the excellent particles identification ability, and to increase the read-out event rate to 100 KHz. In order to achieve this, it is necessary to update the Time Projection Chamber detector (TPC) and Muon tracking (MCH) detector modifying the read-out electronics, which is not suitable for this migration. To overcome this limitation the design, fabrication and experimental test of new ASIC named SAMPA has been proposed . This ASIC will support both positive and negative polarities, with 32 channels per chip and continuous data readout with smaller power consumption than the previous versions. This work aims to design, fabrication and experimental test of a readout front-end in 130nm CMOS technology with configurable polarity (positive/negative), peaking time and sensitivity. The new SAMPA ASIC can be used in both chambers (TPC and MCH). The proposed front-end is composed of a Charge Sensitive Amplifier (CSA) and a Semi-Gaussian shaper. In order to obtain an ASIC integrating 32 channels per chip, the design of the proposed front-end requires small area and low power consumption, but at the same time requires low noise. In this sense, a new Noise and PSRR (Power Supply Rejection Ratio) improvement technique for the CSA design without power and area impact is proposed in this work. The analysis and equations of the proposed circuit are presented which were verified by electrical simulations and experimental test of a produced chip with 5 channels of the designed front-end. The measured equivalent noise charge was <550e for 30mV/fC of sensitivity at a input capacitance of 18.5pF. The total core area of the front-end was 2300?m × 150?m, and the measured total power consumption was 9.1mW per channel.

Snakes based tracking and texture analysis of microscopic images

Comunicación presentada en el VII Symposium Nacional de Reconocimiento de Formas y Análisis de Imágenes, SNRFAI, Barcelona, abril 1997.

Control of Redundant Joint Structures Using Image Information During the Tracking of Non-Smooth Trajectories

Visual information is increasingly being used in a great number of applications in order to perform the guidance of joint structures. This paper proposes an image-based controller which allows the joint structure guidance when its number of degrees of freedom is greater than the required for the developed task. In this case, the controller solves the redundancy combining two different tasks: the primary task allows the correct guidance using image information, and the secondary task determines the most adequate joint structure posture solving the possible joint redundancy regarding the performed task in the image space. The method proposed to guide the joint structure also employs a smoothing Kalman filter not only to determine the moment when abrupt changes occur in the tracked trajectory, but also to estimate and compensate these changes using the proposed filter. Furthermore, a direct visual control approach is proposed which integrates the visual information provided by this smoothing Kalman filter. This last aspect permits the correct tracking when noisy measurements are obtained. All the contributions are integrated in an application which requires the tracking of the faces of Asperger children.

Targetless image-based method for measuring displacements and strains on concrete surfaces with a consumer camera

Measurement of concrete strain through non-invasive methods is of great importance in civil engineering and structural analysis. Traditional methods use laser speckle and high quality cameras that may result too expensive for many applications. Here we present a method for measuring concrete deformations with a standard reflex camera and image processing for tracking objects in the concretes surface. Two different approaches are presented here. In the first one, on-purpose objects are drawn on the surface, while on the second one we track small defects on the surface due to air bubbles in the hardening process. The method has been tested on a concrete sample under several loading/unloading cycles. A stop-motion sequence of the process has been captured and analyzed. Results have been successfully compared with the values given by a strain gauge. Accuracy of our methods in tracking objects is below 8 μm, in the order of more expensive commercial devices.

In situ pore water oxygen microprofiles from the Polar Front, Southern South Atlantic Ocean

Without doubt, global climate change is directly linked to the anthropogenic release of greenhouse gases such as carbon dioxide (CO2) and methane (UN IPCC-Report 2007). Therefore, research efforts to comprehend the global carbon cycle have increased during the last years. In the context of the observed changes, it is of particular interest to decipher the role of the hydro-, bio- and atmospheres and how the different compartments of the earth system are affected by the increase of atmospheric CO2. Due to its huge carbon inventory, the marine carbon cycle represents the most important component in this respect. Numerous findings suggest that the Southern Ocean plays a key role in terms of oceanic CO2 uptake. However, an exact quantification of such fluxes of material is hard to achieve for large areas, not least on account of the inaccessibility of this remote region. In particular, there exist so far only few accurate data for benthic carbon fluxes. The latter can be derived from high resolution pore water oxygen profiles, as one possible method. However the ex situ flux determinations carried out on sediment cores, tend to suffer from temperature and pressure artefacts. Alternatively, oxygen microprofiles can be measured in situ, i.e. at the seafloor. Until now, no such data have been published for the Southern Ocean. During the Antarctic Expedition ANT-XXI/4, within the framework of this thesis, in situ and ex situ oxygen profiles were measured and used to derive benthic organic carbon fluxes. Having both types of measurements from the same locations, it was possible to establish a depth-related correction function which was applied subsequently to revise published and additional unpublished carbon fluxes to the seafloor. This resulted in a consistent data base of benthic carbon inputs covering many important sub-regions of the Southern Ocean including the Amundsen and Bellingshausen Seas (southern Pacific), Scotia and Weddell Seas (southern South Atlantic) as well as the Crozet Basin (southern Indian Ocean). Including additional locations on the Antarctic Shelf, there are now 134 new and revised measurement locations, covering almost 180° of the Southern Ocean, for which benthic organic carbon fluxes and sedimentary oxygen penetration depth values are available. Further, benthic carbon fluxes were empirically related to dominant diatom distributions in surface sediments as well as to long-term remotely sensed chlorophyll-a estimates. The comparison of these results with benthic carbon fluxes of the entire Atlantic Ocean reveals significantly higher export efficiencies for the Southern Ocean than have previously been assumed, especially for the area of the opal belt.

A compendium of the anatomy of the human body : intended principally for the use of students ... This edition is prefixed with A compendious history of anatomy, and The Ruyschian art and method of making preparations to exhibit the structure of the human body, illustrated with a representation of the quicksilver tray and its appendages, which is not in the London edition.

"A compendious history of anatomy" and "The Ruyschian art and method of making preparations to exhibit the structure of the human body" (32 p. at front of v. 1) are by Robert Hooper, and are reprinted, with slight changes in text, from his The anatomist's vade-mecum, 4th ed., London, 1802.

Method for determining reaction rate of mild steel containers during melting of magnesium-aluminium alloys and effect of aluminium content on directionally solidified microstructures

A magnesium alloy of eutectic composition (33 wt-'%Al) was directionally solidified in mild steel tubes at two growth rates, 32 and 580 mum s(-1,) in a temperature gradient between 10 and 20 K mm(-1). After directional solidification, the composition of each specimen varied dramatically, from 32'%Al in the region that had remained solid to 18%Al (32 mum s(-1) specimen) and 13%Al (580 mum s(-1) specimen) at the plane that had been quenched from the eutectic temperature. As the aluminium content decreased, the microstructure contained an increasing volume fraction of primary magnesium dendrites and the eutectic morphology gradually changed from lamellar to partially divorced. The reduction in aluminium content was caused by the growth of an Al-Fe phase ahead of the Mg-Al growth front. Most of the growth of the Al-Fe phase occurred during the remelting period before directional solidification. The thickness of the Al-Fe phase increased with increased temperature and time of contact with the molten Mg-Al alloy. (C) 2003 Maney Publishing.

Tracking the states of a nonlinear and nonstationary system in the weight-space of artificial neural networks

We propose a novel interpretation and usage of Neural Network (NN) in modeling physiological signals, which are allowed to be nonlinear and/or nonstationary. The method consists of training a NN for the k-step prediction of a physiological signal, and then examining the connection-weight-space (CWS) of the NN to extract information about the signal generator mechanism. We de. ne a novel feature, Normalized Vector Separation (gamma(ij)), to measure the separation of two arbitrary states i and j in the CWS and use it to track the state changes of the generating system. The performance of the method is examined via synthetic signals and clinical EEG. Synthetic data indicates that gamma(ij) can track the system down to a SNR of 3.5 dB. Clinical data obtained from three patients undergoing carotid endarterectomy of the brain showed that EEG could be modeled (within a root-means-squared-error of 0.01) by the proposed method, and the blood perfusion state of the brain could be monitored via gamma(ij), with small NNs having no more than 21 connection weight altogether.

A modified particle filter for simultaneous robot localization and landmark tracking in an indoor environment

This paper presents the implementation of a modified particle filter for vision-based simultaneous localization and mapping of an autonomous robot in a structured indoor environment. Through this method, artificial landmarks such as multi-coloured cylinders can be tracked with a camera mounted on the robot, and the position of the robot can be estimated at the same time. Experimental results in simulation and in real environments show that this approach has advantages over the extended Kalman filter with ambiguous data association and various levels of odometric noise.

Incorporating level set methods in Geographical Information Systems (GIS) for land-surface process modelling

Land-surface processes include a broad class of models that operate at a landscape scale. Current modelling approaches tend to be specialised towards one type of process, yet it is the interaction of processes that is increasing seen as important to obtain a more integrated approach to land management. This paper presents a technique and a tool that may be applied generically to landscape processes. The technique tracks moving interfaces across landscapes for processes such as water flow, biochemical diffusion, and plant dispersal. Its theoretical development applies a Lagrangian approach to motion over a Eulerian grid space by tracking quantities across a landscape as an evolving front. An algorithm for this technique, called level set method, is implemented in a geographical information system (GIS). It fits with a field data model in GIS and is implemented as operators in map algebra. The paper describes an implementation of the level set methods in a map algebra programming language, called MapScript, and gives example program scripts for applications in ecology and hydrology.

Tracking the states of a nonlinear system in the weight-space of a feed-forward neural network

Nonlinear, non-stationary signals are commonly found in a variety of disciplines such as biology, medicine, geology and financial modeling. The complexity (e.g. nonlinearity and non-stationarity) of such signals and their low signal to noise ratios often make it a challenging task to use them in critical applications. In this paper we propose a new neural network based technique to address those problems. We show that a feed forward, multi-layered neural network can conveniently capture the states of a nonlinear system in its connection weight-space, after a process of supervised training. The performance of the proposed method is investigated via computer simulations.

Non-rigid object tracking in complex scenes

A recently proposed colour based tracking algorithm has been established to track objects in real circumstances [Zivkovic, Z., Krose, B. 2004. An EM-like algorithm for color-histogram-based object tracking. In: Proc, IEEE Conf. on Computer Vision and Pattern Recognition, pp. 798-803]. To improve the performance of this technique in complex scenes, in this paper we propose a new algorithm for optimally adapting the ellipse outlining the objects of interest. This paper presents a Lagrangian based method to integrate a regularising component into the covariance matrix to be computed. Technically, we intend to reduce the residuals between the estimated probability distribution and the expected one. We argue that, by doing this, the shape of the ellipse can be properly adapted in the tracking stage. Experimental results show that the proposed method has favourable performance in shape adaption and object localisation.