Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Image-based visual servoing for the super-orbital re-entry of Hayabusa spacecraft

This paper presents an image-based visual servoing system that was used to track the atmospheric Earth re-entry of Hayabusa. The primary aim of this ground based tracking platform was to record the emission spectrum radiating from the superheated gas of the shock layer and the surface of the heat shield during re-entry. To the author's knowledge, this is the first time that a visual servoing system has successfully tracked a super-orbital re-entry of a spacecraft and recorded its pectral signature. Furthermore, we improved the system by including a simplified dynamic model for feed-forward control and demonstrate improved tracking performance on the International Space Station (ISS). We present comparisons between simulation and experimental results on different target trajectories including tracking results from Hayabusa and ISS. The required performance for tracking both spacecraft is demanding when combined with a narrow field of view (FOV). We also briefly discuss the preliminary results obtained from the spectroscopy of the Hayabusa's heat shield during re-entry.

Visible and near infrared spectroscopy of Hayabusa re-entry using semi-autonomous tracking

A ground-based tracking camera and co-aligned slit-less spectrograph were used to measure the spectral signature of visible radiation emitted from the Hayabusa capsule as it entered into the Earth's atmosphere in June 2010. Good quality spectra were obtained that showed the presence of radiation from the heat shield of the vehicle and the shock-heated air in front of the vehicle. An analysis of the black body nature of the radiation concluded that the peak average temperature of the surface was about (3100±100) K.

Evaluation of aerial photography for predicting trends in structural attributes of Australian woodland including comparison with ground-based monitoring data

The accurate assessment of trends in the woody structure of savannas has important implications for greenhouse accounting and land-use industries such as pastoralism. Two recent assessments of live woody biomass change from north-east Australian eucalypt woodland between the 1980s and 1990s present divergent results. The first estimate is derived from a network of permanent monitoring plots and the second from woody cover assessments from aerial photography. The differences between the studies are reviewed and include sample density, spatial scale and design. Further analyses targeting potential biases in the indirect aerial photography technique are conducted including a comparison of basal area estimates derived from 28 permanent monitoring sites with basal area estimates derived by the aerial photography technique. It is concluded that the effect of photo-scale; or the failure to include appropriate back-transformation of biomass estimates in the aerial photography study are not likely to have contributed significantly to the discrepancy. However, temporal changes in the structure of woodlands, for example, woodlands maturing from many smaller trees to fewer larger trees or seasonal changes, which affect the relationship between cover and basal area could impact on the detection of trends using the aerial photography technique. It is also possible that issues concerning photo-quality may bias assessments through time, and that the limited sample of the permanent monitoring network may inadequately represent change at regional scales

Exploiting ground-based measurements of the Global Positioning System for numerical weather prediction

Data assimilation provides an initial atmospheric state, called the analysis, for Numerical Weather Prediction (NWP). This analysis consists of pressure, temperature, wind, and humidity on a three-dimensional NWP model grid. Data assimilation blends meteorological observations with the NWP model in a statistically optimal way. The objective of this thesis is to describe methodological development carried out in order to allow data assimilation of ground-based measurements of the Global Positioning System (GPS) into the High Resolution Limited Area Model (HIRLAM) NWP system. Geodetic processing produces observations of tropospheric delay. These observations can be processed either for vertical columns at each GPS receiver station, or for the individual propagation paths of the microwave signals. These alternative processing methods result in Zenith Total Delay (ZTD) and Slant Delay (SD) observations, respectively. ZTD and SD observations are of use in the analysis of atmospheric humidity. A method is introduced for estimation of the horizontal error covariance of ZTD observations. The method makes use of observation minus model background (OmB) sequences of ZTD and conventional observations. It is demonstrated that the ZTD observation error covariance is relatively large in station separations shorter than 200 km, but non-zero covariances also appear at considerably larger station separations. The relatively low density of radiosonde observing stations limits the ability of the proposed estimation method to resolve the shortest length-scales of error covariance. SD observations are shown to contain a statistically significant signal on the asymmetry of the atmospheric humidity field. However, the asymmetric component of SD is found to be nearly always smaller than the standard deviation of the SD observation error. SD observation modelling is described in detail, and other issues relating to SD data assimilation are also discussed. These include the determination of error statistics, the tuning of observation quality control and allowing the taking into account of local observation error correlation. The experiments made show that the data assimilation system is able to retrieve the asymmetric information content of hypothetical SD observations at a single receiver station. Moreover, the impact of real SD observations on humidity analysis is comparable to that of other observing systems.

Eigen-profiles of spatio-temporal fragments for adaptive region-based tracking

We propose a novel space-time descriptor for region-based tracking which is very concise and efficient. The regions represented by covariance matrices within a temporal fragment, are used to estimate this space-time descriptor which we call the Eigenprofiles(EP). EP so obtained is used in estimating the Covariance Matrix of features over spatio-temporal fragments. The Second Order Statistics of spatio-temporal fragments form our target model which can be adapted for variations across the video. The model being concise also allows the use of multiple spatially overlapping fragments to represent the target. We demonstrate good tracking results on very challenging datasets, shot under insufficient illumination conditions.

Hardware development and ground-based studies for drop shaft experiment on oscillatory convection in half floating-zone

Coordinated measurement of temperature, velocity and free surface oscillation were obtained by using the drop shaft facility for microgravity experiments of half floating zone convection. The ground-based studies gave transition from steady to oscillatory convection for multi-quantities measurement.

Subcooled Pool Boiling at Microgravity: Preliminary Ground-Based Experimental Results

A temperature-controlled pool boiling (TCPB) device was developed to perform pool boiling heat transfer studies at both normal gravity on Earth and microgravity in the drop tower Beijing and aboard a Chinese recovery satellite. Two platinum wires of 60 ?m in diameter were simultaneously used as heaters and thermometers. The lengths were 30 mm and 40 mm, respectively. The ends of wires were soldered with copper poles to provide low resistance paths for the electric current. The heater resistance, and thus the heater temperature, was kept constant by a feedback circuit similar to that used in constant-temperature hot-wire anemometry. The fluid was R113 at 0.1 Mpa and subcooled by 30 ?C nominally for all cases. The results of the experiments at normal gravity were presented. Four modes, namely single-phase convection, nucleate boiling, transition two-mode boiling, and film boiling were observed. A few data obtained from several preliminary experiments at microgravity in the drop tower Beijing were also presented. A slight increase of the heat flux was obtained.

Monitoring of streambank stability and streamside vegetation in a livestock exclosure on the Warm Springs River, Oregon : comparison of ground-based surveys with aerial photographic analysis

The objective of this monitoring project was to determine the baseline condition for a 960-m long stream reach and its associated streamside zone, which terminates at the confluence with the Deschutes River. This stream reach had been damaged heavily in the February 1996 flood and had also received many years of overuse by livestock grazing. The monitoring project was conducted in July 1997 just after installation of riparian exclosure fencing. Future resurvey of the study area will allow determination of progress made in ecological recovery.