Real-Time Adaptive Multi-Classifier Multi-Resolution Visual Tracking Framework for Unmanned Aerial Vehicles

This paper presents a novel robust visual tracking framework, based on discriminative method, for Unmanned Aerial Vehicles (UAVs) to track an arbitrary 2D/3D target at real-time frame rates, that is called the Adaptive Multi-Classifier Multi-Resolution (AMCMR) framework. In this framework, adaptive Multiple Classifiers (MC) are updated in the (k-1)th frame-based Multiple Resolutions (MR) structure with compressed positive and negative samples, and then applied them in the kth frame-based Multiple Resolutions (MR) structure to detect the current target. The sample importance has been integrated into this framework to improve the tracking stability and accuracy. The performance of this framework was evaluated with the Ground Truth (GT) in different types of public image databases and real flight-based aerial image datasets firstly, then the framework has been applied in the UAV to inspect the Offshore Floating Platform (OFP). The evaluation and application results show that this framework is more robust, efficient and accurate against the existing state-of-art trackers, overcoming the problems generated by the challenging situations such as obvious appearance change, variant illumination, partial/full target occlusion, blur motion, rapid pose variation and onboard mechanical vibration, among others. To our best knowledge, this is the first work to present this framework for solving the online learning and tracking freewill 2D/3D target problems, and applied it in the UAVs.

Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Chromatin modification by DNA tracking

In general, the transcriptional competence of a chromatin domain is correlated with increased sensitivity to DNase I cleavage. A recent observation that actively transcribing RNA polymerase II piggybacks a histone acetyltranferase activity [Wittschieben, B., Otero, G., de Bizemont, T., Fellows, J., Erdjument-Bromage, H., Ohba, R., Li, Y., Allis, C. D., Tempst, P. & Svejstrup, J. Q. (1999) Mol. Cell 4, 123–128] implies that the state of histone acetylation, and hence the ability of chromatin to fold, can be altered by a processive mechanism. In this article, it is proposed that tracking-mediated chromatin modification could create and/or maintain an open configuration in a complete chromatin domain including both intra- and extragenic regions. This mechanism suggests a putative functional role for the extragenic transcription observed at the β-globin and other loci in vertebrate cells.

Tracking molecular evolution of photosynthesis by characterization of a major photosynthesis gene cluster from Heliobacillus mobilis

A DNA sequence has been obtained for a 35.6-kb genomic segment from Heliobacillus mobilis that contains a major cluster of photosynthesis genes. A total of 30 ORFs were identified, 20 of which encode enzymes for bacteriochlorophyll and carotenoid biosynthesis, reaction-center (RC) apoprotein, and cytochromes for cyclic electron transport. Donor side electron-transfer components to the RC include a putative RC-associated cytochrome c553 and a unique four-large-subunit cytochrome bc complex consisting of Rieske Fe-S protein (encoded by petC), cytochrome b6 (petB), subunit IV (petD), and a diheme cytochrome c (petX). Phylogenetic analysis of various photosynthesis gene products indicates a consistent grouping of oxygenic lineages that are distinct and descendent from anoxygenic lineages. In addition, H. mobilis was placed as the closest relative to cyanobacteria, which form a monophyletic origin to chloroplast-based photosynthetic lineages. The consensus of the photosynthesis gene trees also indicates that purple bacteria are the earliest emerging photosynthetic lineage. Our analysis also indicates that an ancient gene-duplication event giving rise to the paralogous bchI and bchD genes predates the divergence of all photosynthetic groups. In addition, our analysis of gene duplication of the photosystem I and photosystem II core polypeptides supports a “heterologous fusion model” for the origin and evolution of oxygenic photosynthesis.

Neurotransplantation of magnetically labeled oligodendrocyte progenitors: Magnetic resonance tracking of cell migration and myelination

Demyelination is a common pathological finding in human neurological diseases and frequently persists as a result of failure of endogenous repair. Transplanted oligodendrocytes and their precursor cells can (re)myelinate axons, raising the possibility of therapeutic intervention. The migratory capacity of transplanted cells is of key importance in determining the extent of (re)myelination and can, at present, be evaluated only by using invasive and irreversible procedures. We have exploited the transferrin receptor as an efficient intracellular delivery device for magnetic nanoparticles, and transplanted tagged oligodendrocyte progenitor cells into the spinal cord of myelin-deficient rats. Cell migration could be easily detected by using three-dimensional magnetic resonance microscopy, with a close correlation between the areas of contrast enhancement and the achieved extent of myelination. The present results demonstrate that magnetic resonance tracking of transplanted oligodendrocyte progenitors is feasible; this technique has the potential to be easily extended to other neurotransplantation studies involving different precursor cell types.

Fluorescent RNA cytochemistry: tracking gene transcripts in living cells

The advent of jellyfish green fluorescent protein and its spectral variants, together with promising new fluorescent proteins from other classes of the Cnidarian phylum (coral and anemones), has greatly enhanced and promises to further boost the detection and localization of proteins in cell biology. It has been less widely appreciated that highly sensitive methods have also recently been developed for detecting the movement and localization in living cells of the very molecules that precede proteins in the gene expression pathway, i.e. RNAs. These approaches include the microinjection of fluorescent RNAs into living cells, the in vivo hybridization of fluorescent oligonucleotides to endogenous RNAs and the expression in cells of fluorescent RNA-binding proteins. This new field of ‘fluorescent RNA cytochemistry’ is summarized in this article, with emphasis on the biological insights it has already provided. These new techniques are likely to soon collaborate with other emerging approaches to advance the investigation of RNA birth, RNA–protein assembly and ribonucleoprotein particle transport in systems such as oocytes, embryos, neurons and other somatic cells, and may even permit the observation of viral replication and transcription pathways as they proceed in living cells, ushering in a new era of nucleic acids research in vivo.

Tracking memory's trace

There is strong converging evidence that the intermediate and medial part of the hyperstriatum ventrale of the chick brain is a memory store for information acquired through the learning process of imprinting. Neurons in this memory system come, through imprinting, to respond selectively to the imprinting stimulus (IS) neurons and so possess the properties of a memory trace. Therefore, the responses of the intermediate and medial part of the hyperstriatum ventrale neurons to a visual imprinting stimulus were determined before, during, and after training. Of the total recorded population, the proportions of IS neurons shortly after each of two 1-h training sessions were significantly higher (approximately 2 times) than the pretraining proportion. However, ≈4.5 h later this proportion had fallen significantly and did not differ significantly from the pretraining proportion. Nevertheless, ≈21.5 h after the end of training, the proportion of IS neurons was at its highest (approximately 3 times the pretraining level). No significant fluctuations occurred in the proportions of neurons responding to the alternative stimulus. In addition, nonmonotonic changes were found commonly in the activity of 230 of the neurons tracked individually from before training to shortly after the end of training. Thus the pattern of change in responsiveness both at the population level and at the level of individual neurons was highly nonmonotonic. Such a pattern of change is not consistent with simple models of memory based on synaptic strengthening to asymptote. A model is proposed that accounts for the changes in the population responses to the imprinting stimulus in terms of changes in the responses of individual neurons.

Tracking the estrogen receptor in neurons: Implications for estrogen-induced synapse formation

Estrogens (E) and progestins regulate synaptogenesis in the CA1 region of the dorsal hippocampus during the estrous cycle of the female rat, and the functional consequences include changes in neurotransmission and memory. Synapse formation has been demonstrated by using the Golgi technique, dye filling of cells, electron microscopy, and radioimmunocytochemistry. N-methyl-d-aspartate (NMDA) receptor activation is required, and inhibitory interneurons play a pivotal role as they express nuclear estrogen receptor alpha (ERα) and show E-induced decreases of GABAergic activity. Although global decreases in inhibitory tone may be important, a more local role for E in CA1 neurons seems likely. The rat hippocampus expresses both ERα and ERβ mRNA. At the light microscopic level, autoradiography shows cell nuclear [3H]estrogen and [125I]estrogen uptake according to a distribution that primarily reflects the localization of ERα-immunoreactive interneurons in the hippocampus. However, recent ultrastructural studies have revealed extranuclear ERα immunoreactivity (IR) within select dendritic spines on hippocampal principal cells, axon terminals, and glial processes, localizations that would not be detectable by using standard light microscopic methods. Based on recent studies showing that both types of ER are expressed in a form that activates second messenger systems, these findings support a testable model in which local, non-genomic regulation by estrogen participates along with genomic actions of estrogens in the regulation of synapse formation.

Tracking wakes: The nocturnal predatory strategy of piscivorous catfish

Swimming fish leave wakes containing hydrodynamic and chemical traces. These traces mark their swim paths and could guide predators. We now show that nocturnal European catfish (Silurus glanis) locate a piscine prey (guppy, Poecilia reticulata) by accurately tracking its three-dimensional swim path before an attack in the absence of visible light. Wakes that were up to 10 s old were followed over distances up to 55 prey-body lengths in our setup. These results demonstrate that prey wakes remain sufficiently identifiable to guide predators, and to extend considerably the area in which prey is detectable. Moreover, wakes elicit rear attacks, which may be more difficult to detect by prey. Wake tracking may be a common strategy among aquatic predators.

In vivo tracking of platelets: circulating degranulated platelets rapidly lose surface P-selectin but continue to circulate and function.

To examine the hypothesis that surface P-selectin-positive (degranulated) platelets are rapidly cleared from the circulation, we developed novel methods for tracking of platelets and measurement of platelet function in vivo. Washed platelets prepared from nonhuman primates (baboons) were labeled with PKH2 (a lipophilic fluorescent dye), thrombin-activated, washed, and reinfused into the same baboons. Three-color whole blood flow cytometry was used to simultaneously (i) identify platelets with a mAb directed against glycoprotein (GP)IIb-IIIa (integrin alpha 11b beta 3), (ii) distinguish infused platelets by their PKH2 fluorescence, and (iii) analyze platelet function with mAbs. Two hours after infusion of autologous thrombin-activated platelets (P-selectin-positive, PKH2-labeled), 95 +/- 1% (mean +/- SEM, n = 5) of the circulating PKH2-labeled platelets had become P-selectin-negative. Compared with platelets not activated with thrombin preinfusion, the recovery of these circulating PKH2-labeled, P-selectin-negative platelets was similar 24 h after infusion and only slightly less 48 h after infusion. The loss of platelet surface P-selectin was fully accounted for by a 67.1 +/- 16.7 ng/ml increase in the plasma concentration of soluble P-selectin. The circulating PKH2-labeled, P-selectin-negative platelets were still able to function in vivo, as determined by their (i) participation in platelet aggregates emerging from a bleeding time wound, (ii) binding to Dacron in an arteriovenous shunt, (iii) binding of mAb PAC1 (directed against the fibrinogen binding site on GPIIb-IIIa), and (iv) generation of procoagulant platelet-derived microparticles. In summary, (i) circulating degranulated platelets rapidly lose surface P-selectin to the plasma pool, but continue to circulate and function; and (ii) we have developed novel three-color whole blood flow cytometric methods for tracking of platelets and measurement of platelet function in vivo.

Three-dimensional tracking of motile bacteria near a solid planar surface.

Knowing how motile bacteria move near and along a solid surface is crucial to understanding such diverse phenomena as the migration of infectious bacteria along a catheter, biofilm growth, and the movement of bacteria through the pore spaces of saturated soil, a critical step in the in situ bioremediation of contaminated aquifers. In this study, a tracking microscope is used to record the three-dimensional motion of Escherichia coli near a planar glass surface. Data from the tracking microscope are analyzed to quantify the effects of bacteria-surface interactions on the swimming behavior of bacteria. The speed of cells approaching the surface is found to decrease in agreement with the mathematical model of Ramia et al. [Ramia, M., Tullock, D. L. & Phan-Tien, N. (1993) Biophys J. 65,755-778], which represents the bacteria as spheres with a single polar flagellum rotating at a constant rate. The tendency of cells to swim adjacent to the surface is shown in computer-generated reproductions of cell traces. The attractive interaction potential between the cells and the solid surface is offered as one of several possible explanations for this tendency.

Estudo ecocardiográfico da função ventricular esquerda em pacientes com lúpus eritematoso sistêmico juvenil através da técnica de Speckle-Tracking bidimensional

Objetivo: O principal propósito do estudo foi pesquisar a disfunção ventricular esquerda subclínica em pacientes com lúpus eritematoso sistêmico juvenil (LESJ) através da técnica de speckle-tracking bidimensional. Foi investigada ainda uma possível correlação entre o comprometimento da deformação miocárdica e o SLEDAI-2K (Systemic Lupus Erithematosus Disease Activity Index 2000), bem como a presença de fatores de risco cardiovascular, tanto tradicionais como ligados à doença. Métodos: 50 pacientes assintomáticos do ponto de vista cardiovascular e 50 controles saudáveis (14,74 vs. 14,82 anos, p=0.83) foram avaliados pelo ecocardiograma convencional e pelo speckle-tracking bidimensional. Resultados: Apesar da fração de ejeção normal, os pacientes apresentaram redução de todos os parâmetros de deformação miocárdica longitudinal e radial, quando comparados aos controles: strain de pico sistólico longitudinal [-20,3 (-11 a -26) vs. -22 (-17,8 a -30.4) %, p < 0,0001], strain rate de pico sistólico longitudinal [-1,19 ± 0,21 vs. -1,3 ± 0,25 s-1, p=0,0005], strain rate longitudinal na diástole precoce [1,7 (0,99 a 2,95) vs. 2 (1,08 a 3,00) s-1 , p=0,0034], strain de pico sistólico radial [33,09 ± 8,6 vs. 44,36 ± 8,72%, p < 0,0001], strain rate de pico sistólico radial [1,98 ± 0,53 vs. 2,49 ± 0,68 s-1, p < 0,0001] e strain rate radial na diástole precoce [-2,31 ± 0,88 vs. -2,75 ± 0,97 s-1, p=0,02]. O strain de pico sistólico circunferencial [-23,67 ± 3,46 vs. - 24,6 ± 2,86%, p=0,43] e o strain rate circunferencial na diástole precoce [2 (0,88 a 3,4) vs. 1,99 (1,19 a 3,7) s-1, p=0,88] foram semelhantes em pacientes e controles. Apenas o strain rate de pico sistólico circunferencial [-1,5 ± 0,3 vs. -1,6 ± 0,3 s-1, p=0,036] mostrou-se reduzido no LESJ. Uma correlação negativa foi identificada entre o strain de pico sistólico longitudinal e o SLEDAI-2K (r = - 0,52; p < 0,0001) e também o número de fatores de risco cardiovascular por paciente (r = -0,32, p=0,024). Conclusões: Foi evidenciada disfunção sistólica e diastólica subclínica de ventrículo esquerdo no LESJ através da técnica de speckle-tracking bidimensional. A atividade da doença e a exposição aos fatores de risco cardiovascular provavelmente contribuíram para o comprometimento da deformação miocárdica nesses pacientes

An Evaluation of Applicant Tracking Systems for Recruiting Efficiency

The very purpose of a recruiting software program is to help the management of organizations, primarily the HR department to keep track of the job applications. An applicant tracking system can reduce an organization's overall recruitment cost, increase productivity, and raise the level of satisfaction due to faster and better completion of transactions and services. This project analyzes four software providers to discover an applicant tracking system which best suits an organization's recruiting needs. The capstone also highlights that great success an organization can be achieved by significantly improving the delivery of its recruiting services to employees, managers and applicants. The adoption of a well managed applicant tracking system can support this goal.

Snakes based tracking and texture analysis of microscopic images

Comunicación presentada en el VII Symposium Nacional de Reconocimiento de Formas y Análisis de Imágenes, SNRFAI, Barcelona, abril 1997.

Plantillas deformables espacio-temporales para el tracking y reconocimiento gestual

Comunicación presentada en la VII Conferencia de la Asociación Española para la Inteligencia Artificial, CAEPIA, Málaga, 12-14 noviembre, 1997.