Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Proprioceptive force estimation and control for teleoperation in radioactive environments

A medida que se incrementa la energía de los aceleradores de partículas o iones pesados como el CERN o GSI, de los reactores de fusión como JET o ITER, u otros experimentos científicos, se va haciendo cada vez más imprescindible el uso de técnicas de manipulación remota para la interacción con el entorno sujeto a la radiación. Hasta ahora la tasa de dosis radioactiva en el CERN podía tomar valores cercanos a algunos mSv para tiempos de enfriamiento de horas, que permitían la intervención humana para tareas de mantenimiento. Durante los primeros ensayos con plasma en JET, se alcanzaban valores cercanos a los 200 μSv después de un tiempo de enfriamiento de 4 meses y ya se hacía extensivo el uso de técnicas de manipulación remota. Hay una clara tendencia al incremento de los niveles de radioactividad en el futuro en este tipo de instalaciones. Un claro ejemplo es ITER, donde se esperan valores de 450 Sv/h en el centro del toroide a los 11 días de enfriamiento o los nuevos niveles energéticos del CERN que harán necesario una apuesta por niveles de mantenimiento remotos. En estas circunstancias se enmarca esta tesis, que estudia un sistema de control bilateral basado en fuerza-posición, tratando de evitar el uso de sensores de fuerza/par, cuyo contenido electrónico los hace especialmente sensitivos en estos ambientes. El contenido de este trabajo se centra en la teleoperación de robots industriales, que debido a su reconocida solvencia y facilidad para ser adaptados a estos entornos, unido al bajo coste y alta disponibilidad, les convierte en una alternativa interesante para tareas de manipulación remota frente a costosas soluciones a medida. En primer lugar se considera el problema cinemático de teleoperación maestro-esclavo de cinemática disimilar y se desarrolla un método general para la solución del problema en el que se incluye el uso de fuerzas asistivas para guiar al operador. A continuación se explican con detalle los experimentos realizados con un robot ABB y que muestran las dificultades encontradas y recomendaciones para solventarlas. Se concluye el estudio cinemático con un método para el encaje de espacios de trabajo entre maestro y esclavo disimilares. Posteriormente se mira hacia la dinámica, estudiándose el modelado de robots con vistas a obtener un método que permita estimar las fuerzas externas que actúan sobre los mismos. Durante la caracterización del modelo dinámico, se realizan varios ensayos para tratar de encontrar un compromiso entre complejidad de cálculo y error de estimación. También se dan las claves para modelar y caracterizar robots con estructura en forma de paralelogramo y se presenta la arquitectura de control deseada. Una vez obtenido el modelo completo del esclavo, se investigan diferentes alternativas que permitan una estimación de fuerzas externas en tiempo real, minimizando las derivadas de la posición para minimizar el ruido. Se comienza utilizando observadores clásicos del estado para ir evolucionando hasta llegar al desarrollo de un observador de tipo Luenberger-Sliding cuya implementación es relativamente sencilla y sus resultados contundentes. También se analiza el uso del observador propuesto durante un control bilateral simulado en el que se compara la realimentación de fuerzas obtenida con las técnicas clásicas basadas en error de posición frente a un control basado en fuerza-posición donde la fuerza es estimada y no medida. Se comprueba como la solución propuesta da resultados comparables con las arquitecturas clásicas y sin embargo introduce una alternativa para la teleoperación de robots industriales cuya teleoperación en entornos radioactivos sería imposible de otra manera. Finalmente se analizan los problemas derivados de la aplicación práctica de la teleoperación en los escenarios mencionados anteriormente. Debido a las condiciones prohibitivas para todo equipo electrónico, los sistemas de control se deben colocar a gran distancia de los manipuladores, dando lugar a longitudes de cable de centenares de metros. En estas condiciones se crean sobretensiones en controladores basados en PWM que pueden ser destructivas para el sistema formado por control, cableado y actuador, y por tanto, han de ser eliminadas. En este trabajo se propone una solución basada en un filtro LC comercial y se prueba de forma extensiva que su inclusión no produce efectos negativos sobre el control del actuador. ABSTRACT As the energy on the particle accelerators or heavy ion accelerators such as CERN or GSI, fusion reactors such as JET or ITER, or other scientific experiments is increased, it is becoming increasingly necessary to use remote handling techniques to interact with the remote and radioactive environment. So far, the dose rate at CERN could present values near several mSv for cooling times on the range of hours, which allowed human intervention for maintenance tasks. At JET, they measured values close to 200 μSv after a cooling time of 4 months and since then, the remote handling techniques became usual. There is a clear tendency to increase the radiation levels in the future. A clear example is ITER, where values of 450 Sv/h are expected in the centre of the torus after 11 days of cooling. Also, the new energetic levels of CERN are expected to lead to a more advanced remote handling means. In these circumstances this thesis is framed, studying a bilateral control system based on force-position, trying to avoid the use of force/torque sensors, whose electronic content makes them very sensitive in these environments. The contents of this work are focused on teleoperating industrial robots, which due its well-known reliability, easiness to be adapted to these environments, cost-effectiveness and high availability, are considered as an interesting alternative to expensive custom-made solutions for remote handling tasks. Firstly, the kinematic problem of teloperating master and slave with dissimilar kinematics is analysed and a new general approach for solving this issue is presented. The solution includes using assistive forces in order to guide the human operator. Coming up next, I explain with detail the experiments accomplished with an ABB robot that show the difficulties encountered and the proposed solutions. This section is concluded with a method to match the master’s and slave’s workspaces when they present dissimilar kinematics. Later on, the research studies the dynamics, with special focus on robot modelling with the purpose of obtaining a method that allows to estimate external forces acting on them. During the characterisation of the model’s parameters, a set of tests are performed in order to get to a compromise between computational complexity and estimation error. Key points for modelling and characterising robots with a parallelogram structure are also given, and the desired control architecture is presented. Once a complete model of the slave is obtained, different alternatives for external force estimation are review to be able to predict forces in real time, minimizing the position differentiation to minimize the estimation noise. The research starts by implementing classic state observers and then it evolves towards the use of Luenberger- Sliding observers whose implementation is relatively easy and the results are convincing. I also analyse the use of proposed observer during a simulated bilateral control on which the force feedback obtained with the classic techniques based on the position error is compared versus a control architecture based on force-position, where the force is estimated instead of measured. I t is checked how the proposed solution gives results comparable with the classical techniques and however introduces an alternative method for teleoperating industrial robots whose teleoperation in radioactive environments would have been impossible in a different way. Finally, the problems originated by the practical application of teleoperation in the before mentioned scenarios are analysed. Due the prohibitive conditions for every electronic equipment, the control systems should be placed far from the manipulators. This provokes that the power cables that fed the slaves devices can present lengths of hundreds of meters. In these circumstances, overvoltage waves are developed when implementing drives based on PWM technique. The occurrence of overvoltage is very dangerous for the system composed by drive, wiring and actuator, and has to be eliminated. During this work, a solution based on commercial LC filters is proposed and it is extensively proved that its inclusion does not introduce adverse effects into the actuator’s control.

Global instability analysis and control of three-dimensional long open cavity flow

The three-dimensional wall-bounded open cavity may be considered as a simplified geometry found in industrial applications such as leading gear or slotted flats on the airplane. Understanding the three-dimensional complex flow structure that surrounds this particular geometry is therefore of major industrial interest. At the light of the remarkable former investigations in this kind of flows, enough evidences suggest that the lateral walls have a great influence on the flow features and hence on their instability modes. Nevertheless, even though there is a large body of literature on cavity flows, most of them are based on the assumption that the flow is two-dimensional and spanwise-periodic. The flow over realistic open cavity should be considered. This thesis presents an investigation of three-dimensional wall-bounded open cavity with geometric ratio 6:2:1. To this aim, three-dimensional Direct Numerical Simulation (DNS) and global linear instability have been performed. Linear instability analysis reveals that the onset of the first instability in this open cavity is around Recr 1080. The three-dimensional shear layer mode with a complex structure is shown to be the most unstable mode. I t is noteworthy that the flow pattern of this high-frequency shear layer mode is similar to the observed unstable oscillations in supercritical unstable case. DNS of the cavity flow carried out at different Reynolds number from steady state until a nonlinear saturated state is obtained. The comparison of time histories of kinetic energy presents a clearly dominant energetic mode which shifts between low-frequency and highfrequency oscillation. A complete flow patterns from subcritical cases to supercritical case has been put in evidence. The flow structure at the supercritical case Re=1100 resembles typical wake-shedding instability oscillations with a lateral motion existed in the subcritical cases. Also, This flow pattern is similar to the observations in experiments. In order to validate the linear instability analysis results, the topology of the composite flow fields reconstructed by linear superposition of a three-dimensional base flow and its leading three-dimensional global eigenmodes has been studied. The instantaneous wall streamlines of those composited flows display distinguish influence region of each eigenmode. Attention has been focused on the leading high-frequency shear layer mode; the composite flow fields have been fully recognized with respect to the downstream wave shedding. The three-dimensional shear layer mode is shown to give rise to a typical wake-shedding instability with a lateral motions occurring downstream which is in good agreement with the experiment results. Moreover, the spanwise-periodic, open cavity with the same length to depth ratio has been also studied. The most unstable linear mode is different from the real three-dimensional cavity flow, because of the existence of the side walls. Structure sensitivity of the unstable global mode is analyzed in the flow control context. The adjoint-based sensitivity analysis has been employed to localized the receptivity region, where the flow is more sensible to momentum forcing and mass injection. Because of the non-normality of the linearized Navier-Stokes equations, the direct and adjoint field has a large spatial separation. The strongest sensitivity region is locate in the upstream lip of the three-dimensional cavity. This numerical finding is in agreement with experimental observations. Finally, a prototype of passive flow control strategy is applied.

Multi-UAV Coordination and Control Interface

The interest in missions with multiple Unmanned Aerial Vehicles (UAVs) has increased significantly in last years. These missions take advantage of the use of fleets instead of single UAVs to ensure the success, reduce the duration or increase the goals of the mission. In addition, they allow performing tasks that require multiple agents and certain coordination (e.g. surveillance of large areas or transport of heavy loads). Nevertheless, these missions suppose a challenge in terms of control and monitoring. In fact, the workload of the operators rises with the utilization of multiple UAVs and payloads, since they have to analyze more information, make more decisions and generate more commands during the mission. This work addresses the operator workload problem in multi-UAV missions by reducing and selecting the information. Two approaches are considered: a first one that selects the information according to the mission state, and a second one that selects it according to the operator preferences. The result is an interface that is able to control the amount of information and show what is relevant for mission and operator at the time.

Ultrafast detection and control of molecular dynamics

Many elementary chemical and physical processes such as the breaking of a chemical bond or the vibrational motion of atoms within a molecule take place on a femtosecond (fs = 10−15 s) or picosecond (ps = 10−12 s) time scale. It is now possible to monitor these events as a function of time with temporal resolution well below 100 fs. This capability is based on the pump-probe technique where one optical pulse triggers a reaction and a second delayed optical pulse probes the changes that ensue. To illustrate this capability, the dynamics of ligand motion within a protein are presented. Moving beyond casual observation of a reaction to active control of its outcome requires additional experimental and theoretical effort. To illustrate the concept of control, the effect of optical pulse duration on the vibrational dynamics of a tri-atomic molecule are discussed. The experimental and theoretical resources currently available are poised to make the dream of reaction control a reality for certain molecular systems.

Human naïve regulatory T-cells feature high steady-state turnover and are maintained by IL-7

Naïve FoxP3-expressing regulatory T-cells (Tregs) are essential to control immune responses via continuous replenishment of the activated-Treg pool with thymus-committed suppressor cells. The mechanisms underlying naïve-Treg maintenance throughout life in face of the age-associated thymic involution remain unclear. We found that in adults thymectomized early in infancy the naïve-Treg pool is remarkably well preserved, in contrast to conventional naïve CD4 T-cells. Naïve-Tregs featured high levels of cycling and pro-survival markers, even in healthy individuals, and contrasted with other circulating naïve/memory CD4 T-cell subsets in terms of their strong γc-cytokine-dependent signaling, particularly in response to IL-7. Accordingly, ex-vivo stimulation of naïve-Tregs with IL-7 induced robust cytokine-dependent signaling, Bcl-2 expression, and phosphatidylinositol 3-kinase (PI3K)-dependent proliferation, whilst preserving naïve phenotype and suppressive capacity. Altogether, our data strongly implicate IL-7 in the thymus-independent long-term survival of functional naïve-Tregs, and highlight the potential of targeting the IL-7 pathway to modulate Tregs in different clinical settings.

State Aid and Public Service Broadcasting How Future-proof is the Remit of Public Broadcasting Organisations? IES WORKING PAPER 1/2009

The European Commission’s interference through state aid rules with the Member States’ support for public service broadcasting is not undisputed. Member States, public broadcasters and numerous academics fear that State aid control might limit the public service remit and, hence, the multi-platform and holistic role of public broadcasters in the converging media industries. This paper assesses to what extent the fear for Commission intervention is, indeed, justified. It starts with the assumption that the transformation from public service broadcasting to public service media is vital for the European democratic society. The paper leads to the observation that, in fact, European State aid policy might contribute to such a necessary and urgent transformation, instead of threatening it. The paper consists of three main parts. Firstly, the legal constraints and margins of the Community’s State aid framework are discussed. Secondly, the application of the rules to a selection of public broadcasting cases is analyzed. Finally, some conclusions are drawn from the analysis.

First National Conference on Chronic Disease Prevention and Control : identifying effective strategies : September 9-11, 1986, Atlanta, Ga. : conference summary /

"March 1987."

An electronic surveillance and control system for traffic management of the Borman expressway. Final report.

Indiana Department of Transportation, Indianapolis

Report of a proposal to establish a tri-state commission to study possible health effects of radiation from nuclear weapons testing.

Mode of access: Internet.

Speed zoning and control. Final report.

Texas State Department of Highways and Public Transportation, Transportation Planning Division, Austin

Correlation of design and control characteristics with accidents at rural multi-lane highway intersections in Indiana. Interim report.

Federal Highway Administration, Washington, D.C.

Development and use of a computer simulation model for the evaluation of design and control alternatives for intersections of minor roads with multi-lane rural highways: field studies and model validation. Interim report.

Federal Highway Administration, Washington, D.C.

Development and use of a computer simulation model for the evaluation of design and control alternatives for intersections of minor roads with multi-lane rural highways: model application. Interim report.

Federal Highway Administration, Washington, D.C.

Development and use of a computer simulation model for the evaluation of design and control alternatives for intersections of minor roads with multi-lane rural highways: selection of the simulation model. Interim report.

Federal Highway Administration, Washington, D.C.