On-board and Ground Visual Pose Estimation Techniques for UAV Control

In this paper, two techniques to control UAVs (Unmanned Aerial Vehicles), based on visual information are presented. The first one is based on the detection and tracking of planar structures from an on-board camera, while the second one is based on the detection and 3D reconstruction of the position of the UAV based on an external camera system. Both strategies are tested with a VTOL (Vertical take-off and landing) UAV, and results show good behavior of the visual systems (precision in the estimation and frame rate) when estimating the helicopter¿s position and using the extracted information to control the UAV.

Towards a ground navigation system based in visual feedback provided by a mini UAV

This paper addresses initial efforts to develop a navigation system for ground vehicles supported by visual feedback from a mini aerial vehicle. A visual-based algorithm computes the ground vehicle pose in the world frame, as well as possible obstacles within the ground vehicle pathway. Relying on that information, a navigation and obstacle avoidance system is used to re-plan the ground vehicle trajectory, ensuring an optimal detour. Finally, some experiments are presented employing a unmanned ground vehicle (UGV) and a low cost mini unmanned aerial vehicle (UAV).

Risk analysis for UAV safe operations: a rationalization for an agricultural environment

The road to the automation of the agricultural processes passes through the safe operation of the autonomous vehicles. This requirement is a fact in ground mobile units, but it still has not well defined for the aerial robots (UAVs) mainly because the normative and legislation are quite diffuse or even inexistent. Therefore, to define a common and global policy is the challenge to tackle. This characterization has to be addressed from the field experience. Accordingly, this paper presents the work done in this direction, based on the analysis of the most common sources of hazards when using UAV's for agricultural tasks. The work, based on the ISO 31000 normative, has been carried out by applying a three-step structure that integrates the identification, assessment and reduction procedures. The present paper exposes how this method has been applied to analyze previous accidents and malfunctions during UAV operations in order to obtain real failure causes. It has allowed highlighting common risks and hazardous sources and proposing specific guards and safety measures for the agricultural context.

New generation of human machine interfaces for controlling UAV through depth based gesture recognition

New forms of natural interactions between human operators and UAVs (Unmanned Aerial Vehicle) are demanded by the military industry to achieve a better balance of the UAV control and the burden of the human operator. In this work, a human machine interface (HMI) based on a novel gesture recognition system using depth imagery is proposed for the control of UAVs. Hand gesture recognition based on depth imagery is a promising approach for HMIs because it is more intuitive, natural, and non-intrusive than other alternatives using complex controllers. The proposed system is based on a Support Vector Machine (SVM) classifier that uses spatio-temporal depth descriptors as input features. The designed descriptor is based on a variation of the Local Binary Pattern (LBP) technique to efficiently work with depth video sequences. Other major consideration is the especial hand sign language used for the UAV control. A tradeoff between the use of natural hand signs and the minimization of the inter-sign interference has been established. Promising results have been achieved in a depth based database of hand gestures especially developed for the validation of the proposed system.

OMNIWORKS: Omnidirectional vision for human-UAV co-working

The OMNIWORKS project objective is to develop an autonomous and modular aerial inspection system for an off-shore meteorological mast up to 90m in length. The UAV was equipped with an omni-directional camera and vertical take-off/landing capabilities that should be simple enough to operate as to not need the interventions of a professional pilot under challenging situations. Therefore the tests included different aspects used to evaluate both the technical performance of the UAV behavior as well as the operators? point of view.

Fotogrametría de UAV de ala fija y comparación con topografía clásica.

El objetivo de este proyecto es realizar un estudio comparativo utilizando dos procedimientos diferentes para la obtención de mapas y modelos digitales del terreno. Por un lado, realizando un levantamiento de una zona determinada mediante GPS y por otro lado mediante fotogrametría aérea utilizando un vehículo aéreo no tripulado. Dicho estudio surge de la necesidad de obtener cartografía de los elementos que nos rodean y en nuestro caso, como primera toma de contacto, en el empleo de una nueva tecnología. Puesto que este requisito ha existido siempre, es evidente que en cada época el avance de las distintas metodologías y tecnologías para la producción cartográfica es mayor. En la actualidad, una de las técnicas más empleadas es la Fotogrametría. La idea de poder sobrevolar una extensión determinada para obtener imágenes y posteriormente poder recomponer el modelo para obtener la geometría y los elementos de dicha extensión hace que esta técnica sea muy eficaz. Hoy en día, las técnicas siguen avanzando y encontramos en el levantamiento mediante la utilización de vehículos aéreos no tripulados una de las mejores alternativas. Se trata de una herramienta muy potente en la realización de trabajos a grandes escalas (1:1000, 1:500). Comparado con métodos tradicionales, la fotogrametría aérea mediante vehículos aéreos no tripulados ofrece una gran cantidad de ventajas: · Mayor precisión: la alta resolución de las fotos tomadas permite obtener modelos digitales del terreno con mayor nivel de detalle. · Ahorro de tiempo: el levantamiento topográfico se puede completar en cuestión de horas, en comparación a métodos de topografía terrestre y fotogrametría tradicional. · Reducción de costos operativos: ya que levantamiento topográfico toma menos tiempo y menos recursos. · Mejoras en seguridad: el uso de una plataforma UAV puede eliminar o reducir la necesidad de acceder a zonas peligrosas. · Manejo de mayor información: fotografías verticales u oblicuas combinadas, levantamientos topográficos precisos, modelos digitales del terreno y análisis volumétrico. · Facilita la toma de decisiones: basada en información relevante y actualizada.

Toward Visual Autonomous Ship Board Landing of a VTOL UAV

In this paper we tackle the problem of landing a helicopter autonomously on a ship deck, using as the main sensor, an on-board colour camera. To create a test-bed, we first adequately simulate the movement of a ship landing platform on the Sea, for different Sea States, for different ships, randomly and realistically enough. We use a commercial parallel robot to get this movement. Once we had this, we developed an accurate and robust computer vision system to measure the pose of the helipad with respect to the on-board camera. To deal with the noise and the possible fails of the computer vision, a state estimator was created. With all of this, we are now able to develop and test a controller that closes the loop and finish the autonomous landing task.

Obtención de una ortoimagen del yacimiento de los Yesares mediante equipos UAV.

El objetivo principal del presente proyecto es la obtención de una Ortoimagen del yacimiento arqueológico de la Guerra Civil Española de Los Yesares en Pinto, Madrid, para su estudio y valoración. El proyecto se realiza en colaboración con el equipo arqueológico cota 667, mediante los datos adquiridos por un dispositivo UAV proporcionado por la empresa Geotronics. Para alcanzar este objetivo principal se plantearon las siguientes tareas: -Creación del proyecto de vuelo. -Distribución en campo de las dianas para los puntos de apoyo y toma de coordenadas de estas mediante equipo GNSS. -Materialización del vuelo. Preparación del dispositivo de despegue, carga de los datos del vuelo en el UAV y lugar aterrizaje. -Procesado de las fotografías y datos obtenidos y obtención del producto final para proporcionar a los arqueólogos información cartográfica del yacimiento. Gracias a la rapidez y precisión de las nuevas herramientas de documentación geométrica tales como los equipos UAV o los equipos laser escáner, cada vez es más habitual su utilización en todo tipo de ámbitos, incluidos la documentación de elementos patrimoniales como es el caso de este proyecto. Por ello es esencial conocer estas nuevas herramientas, su correcta utilización y los diferentes flujos de trabajo para cada tipo de proyecto.

Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Detecting, Localizing and Following Dynamic Objects with a Mini-UAV

This paper presents an approach for the detection, localization and following of dynamic terrestrial objects using a mini-UAV. The development is intended to be used for surveillance of large infrastructures. The detection algorithm is based on finding several pre-defined characteristics of the target, such as color, shape and size. The process used to localize the target, once it is detected, is based on an inversion of the Pinhole camera model. The task of following the Summit XL was designed to keep the target inside the field of view of the camera, and it was implemented in the form of a PID controller. The system has been tested both in simulation and with real robots, showing promising results.

Multi-UAV Coordination and Control Interface

The interest in missions with multiple Unmanned Aerial Vehicles (UAVs) has increased significantly in last years. These missions take advantage of the use of fleets instead of single UAVs to ensure the success, reduce the duration or increase the goals of the mission. In addition, they allow performing tasks that require multiple agents and certain coordination (e.g. surveillance of large areas or transport of heavy loads). Nevertheless, these missions suppose a challenge in terms of control and monitoring. In fact, the workload of the operators rises with the utilization of multiple UAVs and payloads, since they have to analyze more information, make more decisions and generate more commands during the mission. This work addresses the operator workload problem in multi-UAV missions by reducing and selecting the information. Two approaches are considered: a first one that selects the information according to the mission state, and a second one that selects it according to the operator preferences. The result is an interface that is able to control the amount of information and show what is relevant for mission and operator at the time.

A Reliable Open-Source System Architecture for the Fast Designing and Prototyping of Autonomous Multi-UAV Systems: Simulation and Experimentation

During the process of design and development of an autonomous Multi-UAV System, two main problems appear. The first one is the difficulty of designing all the modules and behaviors of the aerial multi-robot system. The second one is the difficulty of having an autonomous prototype of the system for the developers that allows to test the performance of each module even in an early stage of the project. These two problems motivate this paper. A multipurpose system architecture for autonomous multi-UAV platforms is presented. This versatile system architecture can be used by the system designers as a template when developing their own systems. The proposed system architecture is general enough to be used in a wide range of applications, as demonstrated in the paper. This system architecture aims to be a reference for all designers. Additionally, to allow for the fast prototyping of autonomous multi-aerial systems, an Open Source framework based on the previously defined system architecture is introduced. It allows developers to have a flight proven multi-aerial system ready to use, so that they can test their algorithms even in an early stage of the project. The implementation of this framework, introduced in the paper with the name of “CVG Quadrotor Swarm”, which has also the advantages of being modular and compatible with different aerial platforms, can be found at https://​github.​com/​Vision4UAV/​cvg_​quadrotor_​swarm with a consistent catalog of available modules. The good performance of this framework is demonstrated in the paper by choosing a basic instance of it and carrying out simulation and experimental tests whose results are summarized and discussed in this paper.

3D Object following based on visual information for Unmanned Aerial Vehicles

This article presents a novel system and a control strategy for visual following of a 3D moving object by an Unmanned Aerial Vehicle UAV. The presented strategy is based only on the visual information given by an adaptive tracking method based on the color information, which jointly with the dynamics of a camera fixed to a rotary wind UAV are used to develop an Image-based visual servoing IBVS system. This system is focused on continuously following a 3D moving target object, maintaining it with a fixed distance and centered on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Aerodynamic study of airfoils geometric imperfections at low Reynolds number

The interest of this study is based on the observation that some manufacturing processes of various vehicles wings, such as unmanned aerial vehicle (UAV), or blades, such as wind turbine blades, or other devices that use aerodynamic profiles, produce imperfections in the leading edge or open trailing edge with bigger thickness than original airfoil, because, for example, they are manufactured in two parts, top surface and bottom surface and subsequently joined. In this last step might appear a sliding between the top surface and the bottom surface having a small step on the leading edge or a small thickness gain can occur on the trailing edge. Normally these imperfections are corrected through a refill and/or sanding processes using many hours of manual labor. Therefore the initial objective of this research is to determine the level of influence in the aerodynamic characteristics at low Reynolds numbers (Lissaman, 1981, Carmichael, 1981, Nagamatsu and Cuche, 1981, Schmitz, 1957, Cebeci, 1989, Mueller and Batill, 1982) of these imperfections in the manufacture, and determine whether there may be a value for which it would not be necessary to correct them

System Integration in a UAS design. An example of undergraduate student's tasks.

Systems integration is the origin of most major difficulties found in the engineering design of aeronautical vehicles. The whole design team must assure that each subsystem accomplishes its particular goals and that, together with the rest of the systems, they all meet the general aircraft requirements.Design and building of UAS is a field of actuation to which leading Universities, research Centers and Aeronautical designers have dedicated a lot of effort. In recent years, a team of students, lecturers and professors at the Escuela Universitaria de Ingeniería Técnica Aeronáutica (EUITA) have been working on the design and building of a UAS for civil observation. The design of multi-mission Unmanned Aerial Vehicles (UAVs) has seen a rapid progress in the last years. A wide variety of designs and applications, some of them really ingenious, have been proposed. The project, which has been going on as a teamwork experience for the last ten years, consists of the design and building of a UAV, and its peculiarity is that it has been carried out entirely by undergraduate students, as part of their Final Research Project. The students face a challenge that includes all the features and stages of an authentic engineering project. We present the current moment of evolution in the process, together with a description of the main difficulties the project has undergone, as a global experience in engineering design and development.