Why the hand motion proceeds the target in tracking experiment?

In the present study, to shed light on a role of positional error correction mechanism and prediction mechanism in the proactive control discovered earlier, we carried out a visual tracking experiment, in which the region where target was shown, was regulated in a circular orbit. Main results found in this research were following. Recognition of a time step, obtained from the environmental stimuli, is required for the predictive function. The period of the rhythm in the brain obtained from environmental stimuli is shortened about 10%, when the visual information is cut-off. The shortening of the period of the rhythm in the brain accelerates the motion as soon as the visual information is cut-off, and lets the hand motion precedes the target motion. Although the precedence of the hand in the blind region is reset by the environmental information when the target enters the visible region, the hand precedes in average the target when the predictive mechanism dominates the error-corrective mechanism.

Robust Real-time Vision-based Aircraft Tracking From Unmanned Aerial Vehicles.

Aircraft tracking plays a key and important role in the Sense-and-Avoid system of Unmanned Aerial Vehicles (UAVs). This paper presents a novel robust visual tracking algorithm for UAVs in the midair to track an arbitrary aircraft at real-time frame rates, together with a unique evaluation system. This visual algorithm mainly consists of adaptive discriminative visual tracking method, Multiple-Instance (MI) learning approach, Multiple-Classifier (MC) voting mechanism and Multiple-Resolution (MR) representation strategy, that is called Adaptive M3 tracker, i.e. AM3. In this tracker, the importance of test sample has been integrated to improve the tracking stability, accuracy and real-time performances. The experimental results show that this algorithm is more robust, efficient and accurate against the existing state-of-art trackers, overcoming the problems generated by the challenging situations such as obvious appearance change, variant surrounding illumination, partial aircraft occlusion, blur motion, rapid pose variation and onboard mechanical vibration, low computation capacity and delayed information communication between UAVs and Ground Station (GS). To our best knowledge, this is the first work to present this tracker for solving online learning and tracking freewill aircraft/intruder in the UAVs.

Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

DESEMPENHO EM TESTES DE ATENÇÃO VISUAL E INTERFERÊNCIA DE ESTÍMULOS AUDITIVOS DE MOVIMENTO EM INDIVÍDUOS COM E SEM MEDO DE ALTURA

Desordens da ansiedade, especialmente a agorafobia e a desordem do pânico foram associadas a anormalidades das funções vestibulares. Evidências de que o controle do equilíbrio pode exigir habilidades atencionais também foram relatadas. Utilizando o medo de altura como modelo clínico onde sintomas ansiosos coexistem com anormalidades com a percepção espacial e controle do equilíbrio, este estudo investigou o desempenho em testes de atenção visual em voluntários normais com altos e baixos escores obtidos do Questionário de Acrofobia. O teste de rastreio visual foi realizado em 30 indivíduos (15 em cada grupo) enquanto ouviam dois tipos diferentes de estímulos auditivos. Na condição volume um som de 900 Hz era apresentado em ambos ouvidos durante 2 segundos seguidos de mais 2 segundos de silêncio. Na condição balanço , o mesmo som era apresentado durante 2 segundos ao ouvido direito seguido por 2 segundos ao ouvido esquerdo. Estímulos auditivos de movimento provocaram maior desconforto em ambos os grupos, mas nos indivíduos com maiores escores de acrofobia estes estímulos foram associados a um pior desempenho no teste visual. Embora muito limitado pela amostra experimental, este estudo sugere que o medo de altura pode estar associado à dependência visual para manutenção do equilíbrio e que poderia piorar o desempenho nos testes visuais devido à competição dos recursos neuro-cognitivos. Implicações experimentais e clínicas destes achados preliminares exigem outras pesquisas.

DESEMPENHO EM TESTES DE ATENÇÃO VISUAL E INTERFERÊNCIA DE ESTÍMULOS AUDITIVOS DE MOVIMENTO EM INDIVÍDUOS COM E SEM MEDO DE ALTURA

Desordens da ansiedade, especialmente a agorafobia e a desordem do pânico foram associadas a anormalidades das funções vestibulares. Evidências de que o controle do equilíbrio pode exigir habilidades atencionais também foram relatadas. Utilizando o medo de altura como modelo clínico onde sintomas ansiosos coexistem com anormalidades com a percepção espacial e controle do equilíbrio, este estudo investigou o desempenho em testes de atenção visual em voluntários normais com altos e baixos escores obtidos do Questionário de Acrofobia. O teste de rastreio visual foi realizado em 30 indivíduos (15 em cada grupo) enquanto ouviam dois tipos diferentes de estímulos auditivos. Na condição volume um som de 900 Hz era apresentado em ambos ouvidos durante 2 segundos seguidos de mais 2 segundos de silêncio. Na condição balanço , o mesmo som era apresentado durante 2 segundos ao ouvido direito seguido por 2 segundos ao ouvido esquerdo. Estímulos auditivos de movimento provocaram maior desconforto em ambos os grupos, mas nos indivíduos com maiores escores de acrofobia estes estímulos foram associados a um pior desempenho no teste visual. Embora muito limitado pela amostra experimental, este estudo sugere que o medo de altura pode estar associado à dependência visual para manutenção do equilíbrio e que poderia piorar o desempenho nos testes visuais devido à competição dos recursos neuro-cognitivos. Implicações experimentais e clínicas destes achados preliminares exigem outras pesquisas.

DESEMPENHO EM TESTES DE ATENÇÃO VISUAL E INTERFERÊNCIA DE ESTÍMULOS AUDITIVOS DE MOVIMENTO EM INDIVÍDUOS COM E SEM MEDO DE ALTURA

Desordens da ansiedade, especialmente a agorafobia e a desordem do pânico foram associadas a anormalidades das funções vestibulares. Evidências de que o controle do equilíbrio pode exigir habilidades atencionais também foram relatadas. Utilizando o medo de altura como modelo clínico onde sintomas ansiosos coexistem com anormalidades com a percepção espacial e controle do equilíbrio, este estudo investigou o desempenho em testes de atenção visual em voluntários normais com altos e baixos escores obtidos do Questionário de Acrofobia. O teste de rastreio visual foi realizado em 30 indivíduos (15 em cada grupo) enquanto ouviam dois tipos diferentes de estímulos auditivos. Na condição volume um som de 900 Hz era apresentado em ambos ouvidos durante 2 segundos seguidos de mais 2 segundos de silêncio. Na condição balanço , o mesmo som era apresentado durante 2 segundos ao ouvido direito seguido por 2 segundos ao ouvido esquerdo. Estímulos auditivos de movimento provocaram maior desconforto em ambos os grupos, mas nos indivíduos com maiores escores de acrofobia estes estímulos foram associados a um pior desempenho no teste visual. Embora muito limitado pela amostra experimental, este estudo sugere que o medo de altura pode estar associado à dependência visual para manutenção do equilíbrio e que poderia piorar o desempenho nos testes visuais devido à competição dos recursos neuro-cognitivos. Implicações experimentais e clínicas destes achados preliminares exigem outras pesquisas.

Vision-based tracking for sports performance analysis

With the world of professional sports shifting towards employing better sport analytics, the demand for vision-based performance analysis is growing increasingly in recent years. In addition, the nature of many sports does not allow the use of any kind of sensors or other wearable markers attached to players for monitoring their performances during competitions. This provides a potential application of systematic observations such as tracking information of the players to help coaches to develop their visual skills and perceptual awareness needed to make decisions about team strategy or training plans. My PhD project is part of a bigger ongoing project between sport scientists and computer scientists involving also industry partners and sports organisations. The overall idea is to investigate the contribution technology can make to the analysis of sports performance on the example of team sports such as rugby, football or hockey. A particular focus is on vision-based tracking, so that information about the location and dynamics of the players can be gained without any additional sensors on the players. To start with, prior approaches on visual tracking are extensively reviewed and analysed. In this thesis, methods to deal with the difficulties in visual tracking to handle the target appearance changes caused by intrinsic (e.g. pose variation) and extrinsic factors, such as occlusion, are proposed. This analysis highlights the importance of the proposed visual tracking algorithms, which reflect these challenges and suggest robust and accurate frameworks to estimate the target state in a complex tracking scenario such as a sports scene, thereby facilitating the tracking process. Next, a framework for continuously tracking multiple targets is proposed. Compared to single target tracking, multi-target tracking such as tracking the players on a sports field, poses additional difficulties, namely data association, which needs to be addressed. Here, the aim is to locate all targets of interest, inferring their trajectories and deciding which observation corresponds to which target trajectory is. In this thesis, an efficient framework is proposed to handle this particular problem, especially in sport scenes, where the players of the same team tend to look similar and exhibit complex interactions and unpredictable movements resulting in matching ambiguity between the players. The presented approach is also evaluated on different sports datasets and shows promising results. Finally, information from the proposed tracking system is utilised as the basic input for further higher level performance analysis such as tactics and team formations, which can help coaches to design a better training plan. Due to the continuous nature of many team sports (e.g. soccer, hockey), it is not straightforward to infer the high-level team behaviours, such as players’ interaction. The proposed framework relies on two distinct levels of performance analysis: low-level performance analysis, such as identifying players positions on the play field, as well as a high-level analysis, where the aim is to estimate the density of player locations or detecting their possible interaction group. The related experiments show the proposed approach can effectively explore this high-level information, which has many potential applications.

Model-based vehicle detection and classification using orthographic approximations

This paper reports the current state of work to simplify our previous model-based methods for visual tracking of vehicles for use in a real-time system intended to provide continuous monitoring and classification of traffic from a fixed camera on a busy multi-lane motorway. The main constraints of the system design were: (i) all low level processing to be carried out by low-cost auxiliary hardware, (ii) all 3-D reasoning to be carried out automatically off-line, at set-up time. The system developed uses three main stages: (i) pose and model hypothesis using 1-D templates, (ii) hypothesis tracking, and (iii) hypothesis verification, using 2-D templates. Stages (i) & (iii) have radically different computing performance and computational costs, and need to be carefully balanced for efficiency. Together, they provide an effective way to locate, track and classify vehicles.

Anticipatory engineering: anticipation in sensory-motor systems of human

In visual tracking experiments, distributions of the relative phase be-tween target and tracer showed positive relative phase indicating that the tracer precedes the target position. We found a mode transition from the reactive to anticipatory mode. The proposed integrated model provides a framework to understand the antici-patory behaviour of human, focusing on the integration of visual and soma-tosensory information. The time delays in visual processing and somatosensory feedback are explicitly treated in the simultaneous differential equations. The anticipatory behaviour observed in the visual tracking experiments can be ex-plained by the feedforward term of target velocity, internal dynamics, and time delay in somatosensory feedback.

Eisströme und Schelfeise an der Küste der Amundsen See (West-Antarktis), beobachtet mit ERS-SAR /// Es stehen 2 Dateien zur Auswahl: pdf-Datei Abbildungen nur schwarz-weiss, zip-Datei Abbildungen farbig

In der vorliegenden Diplomarbeit erfolgte die Beobachtung von Eisströmen und Schelfeisen an der Küste der Amundsen See in der West-Antarktis, unter Verwendung von ERS-SAR-Amplitudenbildprodukten. Bestandteile dieser Beobachtung waren die Erstellung eines Gletscherinventares, die Erstellung von Multitemporalbildern, die Auswertung von Veränderungen der Eisfronpositionen und - schwerpunktmäßig - die Bestimmung von Eisfließgeschwindigkeiten und deren räumlicher und zeitlicher Vergleich.

Hardware-Based Particle Filter with Evolutionary Resampling Stage

Autonomous systems require, in most of the cases, reasoning and decision-making capabilities. Moreover, the decision process has to occur in real time. Real-time computing means that every situation or event has to have an answer before a temporal deadline. In complex applications, these deadlines are usually in the order of milliseconds or even microseconds if the application is very demanding. In order to comply with these timing requirements, computing tasks have to be performed as fast as possible. The problem arises when computations are no longer simple, but very time-consuming operations. A good example can be found in autonomous navigation systems with visual-tracking submodules where Kalman filtering is the most extended solution. However, in recent years, some interesting new approaches have been developed. Particle filtering, given its more general problem-solving features, has reached an important position in the field. The aim of this thesis is to design, implement and validate a hardware platform that constitutes itself an embedded intelligent system. The proposed system would combine particle filtering and evolutionary computation algorithms to generate intelligent behavior. Traditional approaches to particle filtering or evolutionary computation have been developed in software platforms, including parallel capabilities to some extent. In this work, an additional goal is fully exploiting hardware implementation advantages. By using the computational resources available in a FPGA device, better performance results in terms of computation time are expected. These hardware resources will be in charge of extensive repetitive computations. With this hardware-based implementation, real-time features are also expected.

A vision-based strategy for autonomous aerial refueling tasks

Autonomous aerial refueling is a key enabling technology for both manned and unmanned aircraft where extended flight duration or range are required. The results presented within this paper offer one potential vision-based sensing solution, together with a unique test environment. A hierarchical visual tracking algorithm based on direct methods is proposed and developed for the purposes of tracking a drogue during the capture stage of autonomous aerial refueling, and of estimating its 3D position. Intended to be applied in real time to a video stream from a single monocular camera mounted on the receiver aircraft, the algorithm is shown to be highly robust, and capable of tracking large, rapid drogue motions within the frame of reference. The proposed strategy has been tested using a complex robotic testbed and with actual flight hardware consisting of a full size probe and drogue. Results show that the vision tracking algorithm can detect and track the drogue at real-time frame rates of more than thirty frames per second, obtaining a robust position estimation even with strong motions and multiple occlusions of the drogue.

Attentional blink reflex modulation in a continuous performance task is modality specific

Four experiments investigated the attentional modulation of acoustic blinks during continuous spatial tracking tasks. Experiment 1 found blink magnitude inhibition in a visual tracking task. Experiment 2 replicated this finding and also found blink latency slowing. Experiment 3 varied the difficulty of the task and found larger blink inhibition in the easy condition. Blink latency slowing did not differ and was significant at both difficulty levels. Experiment 4 employed less difficult visual and acoustic tracking tasks at two levels of task load. Blink magnitude inhibition during the visual and facilitation during the acoustic task was significant during high load in both modality groups. Blink latency was slowed in all visual task conditions and shortened in the difficult acoustic task. These results indicate that attentional blink modulation in a continuous spatial tracking task is modality specific.

Synthesis and characterization of dual-functionalized core-shell fluorescent microspheres for bioconjugation and cellular delivery

The efficient transport of micron-sized beads into cells, via a non-endocytosis mediated mechanism, has only recently been described. As such there is considerable scope for optimization and exploitation of this procedure to enable imaging and sensing applications to be realized. Herein, we report the design, synthesis and characterization of fluorescent microsphere-based cellular delivery agents that can also carry biological cargoes. These core-shell polymer microspheres possess two distinct chemical environments; the core is hydrophobic and can be labeled with fluorescent dye, to permit visual tracking of the microsphere during and after cellular delivery, whilst the outer shell renders the external surfaces of the microspheres hydrophilic, thus facilitating both bioconjugation and cellular compatibility. Cross-linked core particles were prepared in a dispersion polymerization reaction employing styrene, divinylbenzene and a thiol-functionalized co-monomer. These core particles were then shelled in a seeded emulsion polymerization reaction, employing styrene, divinylbenzene and methacrylic acid, to generate orthogonally functionalized core-shell microspheres which were internally labeled via the core thiol moieties through reaction with a thiol reactive dye (DY630-maleimide). Following internal labeling, bioconjugation of green fluorescent protein (GFP) to their carboxyl-functionalized surfaces was successfully accomplished using standard coupling protocols. The resultant dual-labeled microspheres were visualized by both of the fully resolvable fluorescence emissions of their cores (DY630) and shells (GFP). In vitro cellular uptake of these microspheres by HeLa cells was demonstrated conventionally by fluorescence-based flow cytometry, whilst MTT assays demonstrated that 92% of HeLa cells remained viable after uptake. Due to their size and surface functionalities, these far-red-labeled microspheres are ideal candidates for in vitro, cellular delivery of proteins, as described in the accompanying paper.

Expertise sportive et entraînement perceptivo-cognitif de l’athlète

Pour être performant au plus haut niveau, les athlètes doivent posséder une capacité perceptivo-cognitive supérieure à la moyenne. Cette faculté, reflétée sur le terrain par la vision et l’intelligence de jeu des sportifs, permet d’extraire l’information clé de la scène visuelle. La science du sport a depuis longtemps observé l’expertise perceptivo-cognitive au sein de l’environnement sportif propre aux athlètes. Récemment, des études ont rapporté que l’expertise pouvait également se refléter hors de ce contexte, lors d’activités du quotidien par exemple. De plus, les récentes théories entourant la capacité plastique du cerveau ont amené les chercheurs à développer des outils pour entraîner les capacités perceptivo-cognitives des athlètes afin de les rendre plus performants sur le terrain. Ces méthodes sont la plupart du temps contextuelles à la discipline visée. Cependant, un nouvel outil d’entraînement perceptivo-cognitif, nommé 3-Dimensional Multiple Object Tracking (3D-MOT) et dénué de contexte sportif, a récemment vu le jour et a fait l’objet de nos recherches. Un de nos objectifs visait à mettre en évidence l’expertise perceptivo-cognitive spécifique et non-spécifique chez des athlètes lors d’une même étude. Nous avons évalué la perception du mouvement biologique chez des joueurs de soccer et des non-athlètes dans une salle de réalité virtuelle. Les sportifs étaient systématiquement plus performants en termes d’efficacité et de temps de réaction que les novices pour discriminer la direction du mouvement biologique lors d’un exercice spécifique de soccer (tir) mais également lors d’une action issue du quotidien (marche). Ces résultats signifient que les athlètes possèdent une meilleure capacité à percevoir les mouvements biologiques humains effectués par les autres. La pratique du soccer semble donc conférer un avantage fondamental qui va au-delà des fonctions spécifiques à la pratique d’un sport. Ces découvertes sont à mettre en parallèle avec la performance exceptionnelle des athlètes dans le traitement de scènes visuelles dynamiques et également dénuées de contexte sportif. Des joueurs de soccer ont surpassé des novices dans le test de 3D-MOT qui consiste à suivre des cibles en mouvement et stimule les capacités perceptivo-cognitives. Leur vitesse de suivi visuel ainsi que leur faculté d’apprentissage étaient supérieures. Ces résultats confirmaient des données obtenues précédemment chez des sportifs. Le 3D-MOT est un test de poursuite attentionnelle qui stimule le traitement actif de l’information visuelle dynamique. En particulier, l’attention sélective, dynamique et soutenue ainsi que la mémoire de travail. Cet outil peut être utilisé pour entraîner les fonctions perceptivo-cognitives des athlètes. Des joueurs de soccer entraînés au 3D-MOT durant 30 sessions ont montré une amélioration de la prise de décision dans les passes de 15% sur le terrain comparés à des joueurs de groupes contrôles. Ces données démontrent pour la première fois un transfert perceptivo-cognitif du laboratoire au terrain suivant un entraînement perceptivo-cognitif non-contextuel au sport de l’athlète ciblé. Nos recherches aident à comprendre l’expertise des athlètes par l’approche spécifique et non-spécifique et présentent également les outils d’entraînements perceptivo-cognitifs, en particulier le 3D-MOT, pour améliorer la performance dans le sport de haut-niveau.