Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Energy-entropy-momentum time integration methods for coupled smooth dissipative problems

Esta tesis aborda la formulación, análisis e implementación de métodos numéricos de integración temporal para la solución de sistemas disipativos suaves de dimensión finita o infinita de manera que su estructura continua sea conservada. Se entiende por dichos sistemas aquellos que involucran acoplamiento termo-mecánico y/o efectos disipativos internos modelados por variables internas que siguen leyes continuas, de modo que su evolución es considerada suave. La dinámica de estos sistemas está gobernada por las leyes de la termodinámica y simetrías, las cuales constituyen la estructura que se pretende conservar de forma discreta. Para ello, los sistemas disipativos se describen geométricamente mediante estructuras metriplécticas que identifican claramente las partes reversible e irreversible de la evolución del sistema. Así, usando una de estas estructuras conocida por las siglas (en inglés) de GENERIC, la estructura disipativa de los sistemas es identificada del mismo modo que lo es la Hamiltoniana para sistemas conservativos. Con esto, métodos (EEM) con precisión de segundo orden que conservan la energía, producen entropía y conservan los impulsos lineal y angular son formulados mediante el uso del operador derivada discreta introducido para asegurar la conservación de la Hamiltoniana y las simetrías de sistemas conservativos. Siguiendo estas directrices, se formulan dos tipos de métodos EEM basados en el uso de la temperatura o de la entropía como variable de estado termodinámica, lo que presenta importantes implicaciones que se discuten a lo largo de esta tesis. Entre las cuales cabe destacar que las condiciones de contorno de Dirichlet son naturalmente impuestas con la formulación basada en la temperatura. Por último, se validan dichos métodos y se comprueban sus mejores prestaciones en términos de la estabilidad y robustez en comparación con métodos estándar. This dissertation is concerned with the formulation, analysis and implementation of structure-preserving time integration methods for the solution of the initial(-boundary) value problems describing the dynamics of smooth dissipative systems, either finite- or infinite-dimensional ones. Such systems are understood as those involving thermo-mechanical coupling and/or internal dissipative effects modeled by internal state variables considered to be smooth in the sense that their evolutions follow continuos laws. The dynamics of such systems are ruled by the laws of thermodynamics and symmetries which constitutes the structure meant to be preserved in the numerical setting. For that, dissipative systems are geometrically described by metriplectic structures which clearly identify the reversible and irreversible parts of their dynamical evolution. In particular, the framework known by the acronym GENERIC is used to reveal the systems' dissipative structure in the same way as the Hamiltonian is for conserving systems. Given that, energy-preserving, entropy-producing and momentum-preserving (EEM) second-order accurate methods are formulated using the discrete derivative operator that enabled the formulation of Energy-Momentum methods ensuring the preservation of the Hamiltonian and symmetries for conservative systems. Following these guidelines, two kind of EEM methods are formulated in terms of entropy and temperature as a thermodynamical state variable, involving important implications discussed throughout the dissertation. Remarkably, the formulation in temperature becomes central to accommodate Dirichlet boundary conditions. EEM methods are finally validated and proved to exhibit enhanced numerical stability and robustness properties compared to standard ones.

Finite Element Model of the Innovation Diffusion: An Application to Photovoltaic Systems

This paper presents a Finite Element Model, which has been used for forecasting the diffusion of innovations in time and space. Unlike conventional models used in diffusion literature, the model considers the spatial heterogeneity. The implementation steps of the model are explained by applying it to the case of diffusion of photovoltaic systems in a local region in southern Germany. The applied model is based on a parabolic partial differential equation that describes the diffusion ratio of photovoltaic systems in a given region over time. The results of the application show that the Finite Element Model constitutes a powerful tool to better understand the diffusion of an innovation as a simultaneous space-time process. For future research, model limitations and possible extensions are also discussed.

In vivo tracking of platelets: circulating degranulated platelets rapidly lose surface P-selectin but continue to circulate and function.

To examine the hypothesis that surface P-selectin-positive (degranulated) platelets are rapidly cleared from the circulation, we developed novel methods for tracking of platelets and measurement of platelet function in vivo. Washed platelets prepared from nonhuman primates (baboons) were labeled with PKH2 (a lipophilic fluorescent dye), thrombin-activated, washed, and reinfused into the same baboons. Three-color whole blood flow cytometry was used to simultaneously (i) identify platelets with a mAb directed against glycoprotein (GP)IIb-IIIa (integrin alpha 11b beta 3), (ii) distinguish infused platelets by their PKH2 fluorescence, and (iii) analyze platelet function with mAbs. Two hours after infusion of autologous thrombin-activated platelets (P-selectin-positive, PKH2-labeled), 95 +/- 1% (mean +/- SEM, n = 5) of the circulating PKH2-labeled platelets had become P-selectin-negative. Compared with platelets not activated with thrombin preinfusion, the recovery of these circulating PKH2-labeled, P-selectin-negative platelets was similar 24 h after infusion and only slightly less 48 h after infusion. The loss of platelet surface P-selectin was fully accounted for by a 67.1 +/- 16.7 ng/ml increase in the plasma concentration of soluble P-selectin. The circulating PKH2-labeled, P-selectin-negative platelets were still able to function in vivo, as determined by their (i) participation in platelet aggregates emerging from a bleeding time wound, (ii) binding to Dacron in an arteriovenous shunt, (iii) binding of mAb PAC1 (directed against the fibrinogen binding site on GPIIb-IIIa), and (iv) generation of procoagulant platelet-derived microparticles. In summary, (i) circulating degranulated platelets rapidly lose surface P-selectin to the plasma pool, but continue to circulate and function; and (ii) we have developed novel three-color whole blood flow cytometric methods for tracking of platelets and measurement of platelet function in vivo.

The cutoff phenomenon in finite Markov chains.

Natural mixing processes modeled by Markov chains often show a sharp cutoff in their convergence to long-time behavior. This paper presents problems where the cutoff can be proved (card shuffling, the Ehrenfests' urn). It shows that chains with polynomial growth (drunkard's walk) do not show cutoffs. The best general understanding of such cutoffs (high multiplicity of second eigenvalues due to symmetry) is explored. Examples are given where the symmetry is broken but the cutoff phenomenon persists.

Thermodynamic glass transition in a spin glass without time-reversal symmetry

Spin glasses are a longstanding model for the sluggish dynamics that appear at the glass transition. However, spin glasses differ from structural glasses in a crucial feature: they enjoy a time reversal symmetry. This symmetry can be broken by applying an external magnetic field, but embarrassingly little is known about the critical behavior of a spin glass in a field. In this context, the space dimension is crucial. Simulations are easier to interpret in a large number of dimensions, but one must work below the upper critical dimension (i.e., in d < 6) in order for results to have relevance for experiments. Here we show conclusive evidence for the presence of a phase transition in a four-dimensional spin glass in a field. Two ingredients were crucial for this achievement: massive numerical simulations were carried out on the Janus special-purpose computer, and a new and powerful finite-size scaling method.

Finite temperature dynamics of vortices in the two dimensional anisotropic Heisenberg model

We study the effects of finite temperature on the dynamics of non-planar vortices in the classical, two-dimensional anisotropic Heisenberg model with XY- or easy-plane symmetry. To this end, we analyze a generalized Landau-Lifshitz equation including additive white noise and Gilbert damping. Using a collective variable theory with no adjustable parameters we derive an equation of motion for the vortices with stochastic forces which are shown to represent white noise with an effective diffusion constant linearly dependent on temperature. We solve these stochastic equations of motion by means of a Green's function formalism and obtain the mean vortex trajectory and its variance. We find a non-standard time dependence for the variance of the components perpendicular to the driving force. We compare the analytical results with Langevin dynamics simulations and find a good agreement up to temperatures of the order of 25% of the Kosterlitz-Thouless transition temperature. Finally, we discuss the reasons why our approach is not appropriate for higher temperatures as well as the discreteness effects observed in the numerical simulations.

Elastografia em imagens de ultrassom utilizando elementos de contorno.

Este trabalho apresenta uma nova metodologia para elastografia virtual em imagens simuladas de ultrassom utilizando métodos numéricos e métodos de visão computacional. O objetivo é estimar o módulo de elasticidade de diferentes tecidos tendo como entrada duas imagens da mesma seção transversal obtidas em instantes de tempo e pressões aplicadas diferentes. Esta metodologia consiste em calcular um campo de deslocamento das imagens com um método de fluxo óptico e aplicar um método iterativo para estimar os módulos de elasticidade (análise inversa) utilizando métodos numéricos. Para o cálculo dos deslocamentos, duas formulações são utilizadas para fluxo óptico: Lucas-Kanade e Brox. A análise inversa é realizada utilizando duas técnicas numéricas distintas: o Método dos Elementos Finitos (MEF) e o Método dos Elementos de Contorno (MEC), sendo ambos implementados em Unidades de Processamento Gráfico de uso geral, GpGPUs ( \"General Purpose Graphics Units\" ). Considerando uma quantidade qualquer de materiais a serem determinados, para a implementação do Método dos Elementos de Contorno é empregada a técnica de sub-regiões para acoplar as matrizes de diferentes estruturas identificadas na imagem. O processo de otimização utilizado para determinar as constantes elásticas é realizado de forma semi-analítica utilizando cálculo por variáveis complexas. A metodologia é testada em três etapas distintas, com simulações sem ruído, simulações com adição de ruído branco gaussiano e phantoms matemáticos utilizando rastreamento de ruído speckle. Os resultados das simulações apontam o uso do MEF como mais preciso, porém computacionalmente mais caro, enquanto o MEC apresenta erros toleráveis e maior velocidade no tempo de processamento.

Finite size effects in the specific heat of glass-formers

We report clear finite size effects in the specific heat and in the relaxation times of a model glass former at temperatures considerably smaller than the Mode Coupling transition. A crucial ingredient to reach this result is a new Monte Carlo algorithm which allows us to reduce the relaxation time by two order of magnitudes. These effects signal the existence of a large correlation length in static quantities.

Real time motion estimation using a neural architecture implemented on GPUs

This work describes a neural network based architecture that represents and estimates object motion in videos. This architecture addresses multiple computer vision tasks such as image segmentation, object representation or characterization, motion analysis and tracking. The use of a neural network architecture allows for the simultaneous estimation of global and local motion and the representation of deformable objects. This architecture also avoids the problem of finding corresponding features while tracking moving objects. Due to the parallel nature of neural networks, the architecture has been implemented on GPUs that allows the system to meet a set of requirements such as: time constraints management, robustness, high processing speed and re-configurability. Experiments are presented that demonstrate the validity of our architecture to solve problems of mobile agents tracking and motion analysis.

Difference schemes for time-dependent heat conduction models with delay

Different non-Fourier models of heat conduction, that incorporate time lags in the heat flux and/or the temperature gradient, have been increasingly considered in the last years to model microscale heat transfer problems in engineering. Numerical schemes to obtain approximate solutions of constant coefficients lagging models of heat conduction have already been proposed. In this work, an explicit finite difference scheme for a model with coefficients variable in time is developed, and their properties of convergence and stability are studied. Numerical computations showing examples of applications of the scheme are presented.

Active Visual Features Based on Events to Guide Robot Manipulators in Tracking Tasks

Traditional visual servoing systems do not deal with the topic of moving objects tracking. When these systems are employed to track a moving object, depending on the object velocity, visual features can go out of the image, causing the fail of the tracking task. This occurs specially when the object and the robot are both stopped and then the object starts the movement. In this work, we have employed a retina camera based on Address Event Representation (AER) in order to use events as input in the visual servoing system. The events launched by the camera indicate a pixel movement. Event visual information is processed only at the moment it occurs, reducing the response time of visual servoing systems when they are used to track moving objects.

Regional subsidence modelling in Murcia city (SE Spain) using 1-D vertical finite element analysis and 2-D interpolation of ground surface displacements

Subsidence is a hazard that may have natural or anthropogenic origin causing important economic losses. The area of Murcia city (SE Spain) has been affected by subsidence due to groundwater overexploitation since the year 1992. The main observed historical piezometric level declines occurred in the periods 1982–1984, 1992–1995 and 2004–2008 and showed a close correlation with the temporal evolution of ground displacements. Since 2008, the pressure recovery in the aquifer has led to an uplift of the ground surface that has been detected by the extensometers. In the present work an elastic hydro-mechanical finite element code has been used to compute the subsidence time series for 24 geotechnical boreholes, prescribing the measured groundwater table evolution. The achieved results have been compared with the displacements estimated through an advanced DInSAR technique and measured by the extensometers. These spatio-temporal comparisons have showed that, in spite of the limited geomechanical data available, the model has turned out to satisfactorily reproduce the subsidence phenomenon affecting Murcia City. The model will allow the prediction of future induced deformations and the consequences of any piezometric level variation in the study area.

Practicality of time-optimal two-qubit Hamiltonian simulation

What is the time-optimal way of using a set of control Hamiltonians to obtain a desired interaction? Vidal, Hammerer, and Cirac [Phys. Rev. Lett. 88, 237902 (2002)] have obtained a set of powerful results characterizing the time-optimal simulation of a two-qubit quantum gate using a fixed interaction Hamiltonian and fast local control over the individual qubits. How practically useful are these results? We prove that there are two-qubit Hamiltonians such that time-optimal simulation requires infinitely many steps of evolution, each infinitesimally small, and thus is physically impractical. A procedure is given to determine which two-qubit Hamiltonians have this property, and we show that almost all Hamiltonians do. Finally, we determine some bounds on the penalty that must be paid in the simulation time if the number of steps is fixed at a finite number, and show that the cost in simulation time is not too great.