Development of a visual object tracking system using a compressive sensing framework for modeling the appearance

En el presente trabajo se aborda el problema del seguimiento de objetos, cuyo objetivo es encontrar la trayectoria de un objeto en una secuencia de video. Para ello, se ha desarrollado un método de seguimiento-por-detección que construye un modelo de apariencia en un dominio comprimido usando una nueva e innovadora técnica: “compressive sensing”. La única información necesaria es la situación del objeto a seguir en la primera imagen de la secuencia. El seguimiento de objetos es una aplicación típica del área de visión artificial con un desarrollo de bastantes años. Aun así, sigue siendo una tarea desafiante debido a varios factores: cambios de iluminación, oclusión parcial o total de los objetos y complejidad del fondo de la escena, los cuales deben ser considerados para conseguir un seguimiento robusto. Para lidiar lo más eficazmente posible con estos factores, hemos propuesto un algoritmo de tracking que entrena un clasificador Máquina Vector Soporte (“Support Vector Machine” o SVM en sus siglas en inglés) en modo online para separar los objetos del fondo de la escena. Con este fin, hemos generado nuestro modelo de apariencia por medio de un descriptor de características muy robusto que describe los objetos y el fondo devolviendo un vector de dimensiones muy altas. Por ello, se ha implementado seguidamente un paso para reducir la dimensionalidad de dichos vectores y así poder entrenar nuestro clasificador en un dominio mucho menor, al que denominamos domino comprimido. La reducción de la dimensionalidad de los vectores de características se basa en la teoría de “compressive sensing”, que dice que una señal con poca dispersión (pocos componentes distintos de cero) puede estar bien representada, e incluso puede ser reconstruida, a partir de un conjunto muy pequeño de muestras. La teoría de “compressive sensing” se ha aplicado satisfactoriamente en este trabajo y diferentes técnicas de medida y reconstrucción han sido probadas para evaluar nuestros vectores reducidos, de tal forma que se ha verificado que son capaces de preservar la información de los vectores originales. También incluimos una actualización del modelo de apariencia del objeto a seguir, mediante el reentrenamiento de nuestro clasificador en cada cuadro de la secuencia con muestras positivas y negativas, las cuales han sido obtenidas a partir de la posición predicha por el algoritmo de seguimiento en cada instante temporal. El algoritmo propuesto ha sido evaluado en distintas secuencias y comparado con otros algoritmos del estado del arte de seguimiento, para así demostrar el éxito de nuestro método.

Object tracking using direct methods in RGB-D cameras

En esta tesis se presenta un análisis en profundidad de cómo se deben utilizar dos tipos de métodos directos, Lucas-Kanade e Inverse Compositional, en imágenes RGB-D y se analiza la capacidad y precisión de los mismos en una serie de experimentos sintéticos. Estos simulan imágenes RGB, imágenes de profundidad (D) e imágenes RGB-D para comprobar cómo se comportan en cada una de las combinaciones. Además, se analizan estos métodos sin ninguna técnica adicional que modifique el algoritmo original ni que lo apoye en su tarea de optimización tal y como sucede en la mayoría de los artículos encontrados en la literatura. Esto se hace con el fin de poder entender cuándo y por qué los métodos convergen o divergen para que así en el futuro cualquier interesado pueda aplicar los conocimientos adquiridos en esta tesis de forma práctica. Esta tesis debería ayudar al futuro interesado a decidir qué algoritmo conviene más en una determinada situación y debería también ayudarle a entender qué problemas le pueden dar estos algoritmos para poder poner el remedio más apropiado. Las técnicas adicionales que sirven de remedio para estos problemas quedan fuera de los contenidos que abarca esta tesis, sin embargo, sí se hace una revisión sobre ellas.---ABSTRACT---This thesis presents an in-depth analysis about how direct methods such as Lucas- Kanade and Inverse Compositional can be applied in RGB-D images. The capability and accuracy of these methods is also analyzed employing a series of synthetic experiments. These simulate the efects produced by RGB images, depth images and RGB-D images so that diferent combinations can be evaluated. Moreover, these methods are analyzed without using any additional technique that modifies the original algorithm or that aids the algorithm in its search for a global optima unlike most of the articles found in the literature. Our goal is to understand when and why do these methods converge or diverge so that in the future, the knowledge extracted from the results presented here can efectively help a potential implementer. After reading this thesis, the implementer should be able to decide which algorithm fits best for a particular task and should also know which are the problems that have to be addressed in each algorithm so that an appropriate correction is implemented using additional techniques. These additional techniques are outside the scope of this thesis, however, they are reviewed from the literature.

Numerical issues in Lagrangian tracking and topological evolution of fluid particles in wallbounded turbulent flows

The determination of the local Lagrangian evolution of the flow topology in wall-bounded turbulence, and of the Lagrangian evolution associated with entrainment across the turbulent / non-turbulent interface into a turbulent boundary layer, require accurate tracking of a fluid particle and its local velocity gradients. This paper addresses the implementation of fluid-particle tracking in both a turbulent boundary layer direct numerical simulation and in a fully developed channel flow simulation. Determination of the sub-grid particle velocity is performed using both cubic B-spline, four-point Hermite spline and higher-order Hermite spline interpolation. Both wall-bounded flows show similar oscillations in the Lagrangian tracers of both velocity and velocity gradients, corresponding to the movement of particles across the boundaries of computational cells. While these oscillation in the particle velocity are relatively small and have negligible effect on the particle trajectories for time-steps of the order of CFL = 0.1, they appear to be the cause of significant oscillations in the evolution of the invariants of the velocity gradient tensor.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Robust Real-time Vision-based Aircraft Tracking From Unmanned Aerial Vehicles.

Aircraft tracking plays a key and important role in the Sense-and-Avoid system of Unmanned Aerial Vehicles (UAVs). This paper presents a novel robust visual tracking algorithm for UAVs in the midair to track an arbitrary aircraft at real-time frame rates, together with a unique evaluation system. This visual algorithm mainly consists of adaptive discriminative visual tracking method, Multiple-Instance (MI) learning approach, Multiple-Classifier (MC) voting mechanism and Multiple-Resolution (MR) representation strategy, that is called Adaptive M3 tracker, i.e. AM3. In this tracker, the importance of test sample has been integrated to improve the tracking stability, accuracy and real-time performances. The experimental results show that this algorithm is more robust, efficient and accurate against the existing state-of-art trackers, overcoming the problems generated by the challenging situations such as obvious appearance change, variant surrounding illumination, partial aircraft occlusion, blur motion, rapid pose variation and onboard mechanical vibration, low computation capacity and delayed information communication between UAVs and Ground Station (GS). To our best knowledge, this is the first work to present this tracker for solving online learning and tracking freewill aircraft/intruder in the UAVs.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Real-Time Adaptive Multi-Classifier Multi-Resolution Visual Tracking Framework for Unmanned Aerial Vehicles

This paper presents a novel robust visual tracking framework, based on discriminative method, for Unmanned Aerial Vehicles (UAVs) to track an arbitrary 2D/3D target at real-time frame rates, that is called the Adaptive Multi-Classifier Multi-Resolution (AMCMR) framework. In this framework, adaptive Multiple Classifiers (MC) are updated in the (k-1)th frame-based Multiple Resolutions (MR) structure with compressed positive and negative samples, and then applied them in the kth frame-based Multiple Resolutions (MR) structure to detect the current target. The sample importance has been integrated into this framework to improve the tracking stability and accuracy. The performance of this framework was evaluated with the Ground Truth (GT) in different types of public image databases and real flight-based aerial image datasets firstly, then the framework has been applied in the UAV to inspect the Offshore Floating Platform (OFP). The evaluation and application results show that this framework is more robust, efficient and accurate against the existing state-of-art trackers, overcoming the problems generated by the challenging situations such as obvious appearance change, variant illumination, partial/full target occlusion, blur motion, rapid pose variation and onboard mechanical vibration, among others. To our best knowledge, this is the first work to present this framework for solving the online learning and tracking freewill 2D/3D target problems, and applied it in the UAVs.

Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Design and evaluation of a dehumidifying plaster panel for passive architecture integration

Buildings Indoor Air Quality requires a control in the Relative Humidity parameter. In passive architecture in humid climates relative humidity is even more important for human comfort and difficult to control. Therefore, nowadays, there is a research on dehumidifying systems. The present article shows an innovative dehumidifying panel composed of a plaster and Calcium Chloride salt. Laboratory tests are carried out to establish its viability as an indoor air moister regulator integrated in common plaster building interior coatings. There are two types of tests that have been carried out in two consecutive empirical phases: in the first phase, the tests of characterization of the Calcium Chloride as a desiccant are carried out; in a second phase, the dehumidifying panel as a whole is tested. Finally, both types of empirical tests show the efficiency and viability as an air moisture passive control system.

Chromatin modification by DNA tracking

In general, the transcriptional competence of a chromatin domain is correlated with increased sensitivity to DNase I cleavage. A recent observation that actively transcribing RNA polymerase II piggybacks a histone acetyltranferase activity [Wittschieben, B., Otero, G., de Bizemont, T., Fellows, J., Erdjument-Bromage, H., Ohba, R., Li, Y., Allis, C. D., Tempst, P. & Svejstrup, J. Q. (1999) Mol. Cell 4, 123–128] implies that the state of histone acetylation, and hence the ability of chromatin to fold, can be altered by a processive mechanism. In this article, it is proposed that tracking-mediated chromatin modification could create and/or maintain an open configuration in a complete chromatin domain including both intra- and extragenic regions. This mechanism suggests a putative functional role for the extragenic transcription observed at the β-globin and other loci in vertebrate cells.

Tracking molecular evolution of photosynthesis by characterization of a major photosynthesis gene cluster from Heliobacillus mobilis

A DNA sequence has been obtained for a 35.6-kb genomic segment from Heliobacillus mobilis that contains a major cluster of photosynthesis genes. A total of 30 ORFs were identified, 20 of which encode enzymes for bacteriochlorophyll and carotenoid biosynthesis, reaction-center (RC) apoprotein, and cytochromes for cyclic electron transport. Donor side electron-transfer components to the RC include a putative RC-associated cytochrome c553 and a unique four-large-subunit cytochrome bc complex consisting of Rieske Fe-S protein (encoded by petC), cytochrome b6 (petB), subunit IV (petD), and a diheme cytochrome c (petX). Phylogenetic analysis of various photosynthesis gene products indicates a consistent grouping of oxygenic lineages that are distinct and descendent from anoxygenic lineages. In addition, H. mobilis was placed as the closest relative to cyanobacteria, which form a monophyletic origin to chloroplast-based photosynthetic lineages. The consensus of the photosynthesis gene trees also indicates that purple bacteria are the earliest emerging photosynthetic lineage. Our analysis also indicates that an ancient gene-duplication event giving rise to the paralogous bchI and bchD genes predates the divergence of all photosynthetic groups. In addition, our analysis of gene duplication of the photosystem I and photosystem II core polypeptides supports a “heterologous fusion model” for the origin and evolution of oxygenic photosynthesis.

Neurotransplantation of magnetically labeled oligodendrocyte progenitors: Magnetic resonance tracking of cell migration and myelination

Demyelination is a common pathological finding in human neurological diseases and frequently persists as a result of failure of endogenous repair. Transplanted oligodendrocytes and their precursor cells can (re)myelinate axons, raising the possibility of therapeutic intervention. The migratory capacity of transplanted cells is of key importance in determining the extent of (re)myelination and can, at present, be evaluated only by using invasive and irreversible procedures. We have exploited the transferrin receptor as an efficient intracellular delivery device for magnetic nanoparticles, and transplanted tagged oligodendrocyte progenitor cells into the spinal cord of myelin-deficient rats. Cell migration could be easily detected by using three-dimensional magnetic resonance microscopy, with a close correlation between the areas of contrast enhancement and the achieved extent of myelination. The present results demonstrate that magnetic resonance tracking of transplanted oligodendrocyte progenitors is feasible; this technique has the potential to be easily extended to other neurotransplantation studies involving different precursor cell types.

Therapeutic passive vaccination against chronic Lyme disease in mice

Passive and active immunization against outer surface protein A (OspA) has been successful in protecting laboratory animals against subsequent infection with Borrelia burgdorferi. Antibodies (Abs) to OspA convey full protection, but only when they are present at the time of infection. Abs inactivate spirochetes within the tick and block their transmission to mammals, but do not affect established infection because of the loss of OspA in the vertebrate host. Our initial finding that the presence of high serum titers of anti-OspC Abs (5 to 10 μg/ml) correlates with spontaneous resolution of disease and infection in experimentally challenged immunocompetent mice suggested that therapeutic vaccination with OspC may be feasible. We now show that polyclonal and monospecific mouse immune sera to recombinant OspC, but not to OspA, of B. burgdorferi resolve chronic arthritis and carditis and clear disseminated spirochetes in experimentally infected C.B.-17 severe combined immunodeficient mice in a dose-dependent manner. This was verified by macroscopical and microscopical examination of affected tissues and recultivation of spirochetes from ear biopsies. Complete resolution of disease and infection was achieved, independent of whether OspC-specific immune sera (10 μg OspC-specific Abs) were repeatedly given (4× in 3- to 4-day intervals) before the onset (day 10 postinfection) or at the time of fully established arthritis and carditis (days 19 or 60 postinfection). The results indicate that in mice spirochetes constitutively express OspC and are readily susceptible to protective OspC-specific Abs throughout the infection. Thus, an OspC-based vaccine appears to be a candidate for therapy of Lyme disease.