On-board and Ground Visual Pose Estimation Techniques for UAV Control

In this paper, two techniques to control UAVs (Unmanned Aerial Vehicles), based on visual information are presented. The first one is based on the detection and tracking of planar structures from an on-board camera, while the second one is based on the detection and 3D reconstruction of the position of the UAV based on an external camera system. Both strategies are tested with a VTOL (Vertical take-off and landing) UAV, and results show good behavior of the visual systems (precision in the estimation and frame rate) when estimating the helicopter¿s position and using the extracted information to control the UAV.

Aerial Remote Sensing in Agriculture: A Practical Approach to Area Coverage and Path Planning for Fleets of Mini Aerial Robots

In this paper, a system that allows applying precision agriculture techniques is described. The application is based on the deployment of a team of unmanned aerial vehicles that are able to take georeferenced pictures in order to create a full map by applying mosaicking procedures for postprocessing. The main contribution of this work is practical experimentation with an integrated tool. Contributions in different fields are also reported. Among them is a new one-phase automatic task partitioning manager, which is based on negotiation among the aerial vehicles, considering their state and capabilities. Once the individual tasks are assigned, an optimal path planning algorithm is in charge of determining the best path for each vehicle to follow. Also, a robust flight control based on the use of a control law that improves the maneuverability of the quadrotors has been designed. A set of field tests was performed in order to analyze all the capabilities of the system, from task negotiations to final performance. These experiments also allowed testing control robustness under different weather conditions.

Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

Multi-UAV Coordination and Control Interface

The interest in missions with multiple Unmanned Aerial Vehicles (UAVs) has increased significantly in last years. These missions take advantage of the use of fleets instead of single UAVs to ensure the success, reduce the duration or increase the goals of the mission. In addition, they allow performing tasks that require multiple agents and certain coordination (e.g. surveillance of large areas or transport of heavy loads). Nevertheless, these missions suppose a challenge in terms of control and monitoring. In fact, the workload of the operators rises with the utilization of multiple UAVs and payloads, since they have to analyze more information, make more decisions and generate more commands during the mission. This work addresses the operator workload problem in multi-UAV missions by reducing and selecting the information. Two approaches are considered: a first one that selects the information according to the mission state, and a second one that selects it according to the operator preferences. The result is an interface that is able to control the amount of information and show what is relevant for mission and operator at the time.

Precision Agriculture without borders: Practical issues and improvements in farmland coverage with aerial vehicles.

This work presents a solution for the aerial coverage of a field by using a fleet of aerial vehicles. The use of Unmanned Aerial Vehicles allows to obtain high resolution mosaics to be used in Precision Agriculture techniques. This report is focus on providing a solution for the full simultaneous coverage problem taking into account restrictions as the required spatial resolution and overlap while maintaining similar light conditions and safety operation of the drones. Results obtained from real field tests are finally reported

System Integration in a UAS design. An example of undergraduate student's tasks.

Systems integration is the origin of most major difficulties found in the engineering design of aeronautical vehicles. The whole design team must assure that each subsystem accomplishes its particular goals and that, together with the rest of the systems, they all meet the general aircraft requirements.Design and building of UAS is a field of actuation to which leading Universities, research Centers and Aeronautical designers have dedicated a lot of effort. In recent years, a team of students, lecturers and professors at the Escuela Universitaria de Ingeniería Técnica Aeronáutica (EUITA) have been working on the design and building of a UAS for civil observation. The design of multi-mission Unmanned Aerial Vehicles (UAVs) has seen a rapid progress in the last years. A wide variety of designs and applications, some of them really ingenious, have been proposed. The project, which has been going on as a teamwork experience for the last ten years, consists of the design and building of a UAV, and its peculiarity is that it has been carried out entirely by undergraduate students, as part of their Final Research Project. The students face a challenge that includes all the features and stages of an authentic engineering project. We present the current moment of evolution in the process, together with a description of the main difficulties the project has undergone, as a global experience in engineering design and development.

Aerial Coverage Path Planning applied to Mapping

In the last decade we have seen how small and light weight aerial platforms - aka, Mini Unmanned Aerial Vehicles (MUAV) - shipped with heterogeneous sensors have become a 'most wanted' Remote Sensing (RS) tool. Most of the off-the-shelf aerial systems found in the market provide way-point navigation. However, they do not rely on a tool that compute the aerial trajectories considering all the aspects that allow optimizing the aerial missions. One of the most demanded RS applications of MUAV is image surveying. The images acquired are typically used to build a high-resolution image, i.e., a mosaic of the workspace surface. Although, it may be applied to any other application where a sensor-based map must be computed. This thesis provides a study of this application and a set of solutions and methods to address this kind of aerial mission by using a fleet of MUAVs. In particular, a set of algorithms are proposed for map-based sampling, and aerial coverage path planning (ACPP). Regarding to map-based sampling, the approaches proposed consider workspaces with different shapes and surface characteristics. The workspace is sampled considering the sensor characteristics and a set of mission requirements. The algorithm applies different computational geometry approaches, providing a unique way to deal with workspaces with different shape and surface characteristics in order to be surveyed by one or more MUAVs. This feature introduces a previous optimization step before path planning. After that, the ACPP problem is theorized and a set of ACPP algorithms to compute the MUAVs trajectories are proposed. The problem addressed herein is the problem to coverage a wide area by using MUAVs with limited autonomy. Therefore, the mission must be accomplished in the shortest amount of time. The aerial survey is usually subject to a set of workspace restrictions, such as the take-off and landing positions as well as a safety distance between elements of the fleet. Moreover, it has to avoid forbidden zones to y. Three different algorithms have been studied to address this problem. The approaches studied are based on graph searching, heuristic and meta-heuristic approaches, e.g., mimic, evolutionary. Finally, an extended survey of field experiments applying the previous methods, as well as the materials and methods adopted in outdoor missions is presented. The reported outcomes demonstrate that the findings attained from this thesis improve ACPP mission for mapping purpose in an efficient and safe manner.

Aerial Coverage Path Planning applied to Mapping

In the last decade we have seen how small and light weight aerial platforms - aka, Mini Unmanned Aerial Vehicles (MUAV) - shipped with heterogeneous sensors have become a 'most wanted' Remote Sensing (RS) tool. Most of the off-the-shelf aerial systems found in the market provide way-point navigation. However, they do not rely on a tool that compute the aerial trajectories considering all the aspects that allow optimizing the aerial missions. One of the most demanded RS applications of MUAV is image surveying. The images acquired are typically used to build a high-resolution image, i.e., a mosaic of the workspace surface. Although, it may be applied to any other application where a sensor-based map must be computed. This thesis provides a study of this application and a set of solutions and methods to address this kind of aerial mission by using a fleet of MUAVs. In particular, a set of algorithms are proposed for map-based sampling, and aerial coverage path planning (ACPP). Regarding to map-based sampling, the approaches proposed consider workspaces with different shapes and surface characteristics. The workspace is sampled considering the sensor characteristics and a set of mission requirements. The algorithm applies different computational geometry approaches, providing a unique way to deal with workspaces with different shape and surface characteristics in order to be surveyed by one or more MUAVs. This feature introduces a previous optimization step before path planning. After that, the ACPP problem is theorized and a set of ACPP algorithms to compute the MUAVs trajectories are proposed. The problem addressed herein is the problem to coverage a wide area by using MUAVs with limited autonomy. Therefore, the mission must be accomplished in the shortest amount of time. The aerial survey is usually subject to a set of workspace restrictions, such as the take-off and landing positions as well as a safety distance between elements of the fleet. Moreover, it has to avoid forbidden zones to y. Three different algorithms have been studied to address this problem. The approaches studied are based on graph searching, heuristic and meta-heuristic approaches, e.g., mimic, evolutionary. Finally, an extended survey of field experiments applying the previous methods, as well as the materials and methods adopted in outdoor missions is presented. The reported outcomes demonstrate that the findings attained from this thesis improve ACPP mission for mapping purpose in an efficient and safe manner.

What is the optimal stitching orientation from an aerial survey perspective?

Mosaicing is a technique that allows obtaining a large high resolution image by stitching several images together. These base images are usually acquired from an elevated point of view. Until recently, low-altitude image acquisition has been performed typically by using using airplanes, as well as other manned platforms. However, mini unmanned aerial vehicles (MUAV) endowed with a camera have lately made this task more available for small for cicil applications, for example for small farmers in order to obtain accurate agronomic information about their crop fields. The stitching orientation, or the image acquisition orientation usually coincides with the aircraft heading assuming a downwards orientation of the camera. In this paper, the efect of the image orientation in the eficiency of the aerial coverage path planning is studied. Moreover, an algorithm to compute an optimal stitching orientation angle is proposed and results are numerically compared with classical approaches.

Modelos digitales del terreno mediante fotogrametría aérea realizada con un vehículo aéreo no tripulado

En las últimas décadas, el avance tecnológico ha aumentado a una velocidad vertiginosa. Muchos son los campos beneficiados, entre ellos la ingeniería y por consiguiente el desarrollo de maquinaria, técnicas y herramientas que facilitan el trabajo. Los vehículos aéreos no tripulados (UAVs), más conocidos como drones, se presentan como una alternativa muy interesante para llevar a cabo levantamientos topográficos mediante la técnica fotogramétrica. En este proyecto se desarrollan los Modelos Digitales del Terreno (MDT) de una superficie sobrevolada por un UAV en la localidad de Cubillos del Sil (León). Para ello, se han utilizado dos software: Mission Planner y PhotoModeler. En el primero se ha llevado a cabo la programación de la misión de vuelo del drone, siendo el fin la toma de fotografías aéreas. El segundo es el encargado de realizar la reconstrucción digital en 3D del terreno sobrevolado a partir de esas fotografías. El objetivo final es que los MDT obtenidos puedan ser utilizados en un futuro por cualquier equipo de trabajo que quiera desarrollar un proyecto sobre esos terrenos. ABSTRACT The pace of technology innovation is faster now than it has been for the past few decades. Engineering, and consequently machinery and diverse techniques, are the main beneficiaries. A clear example of this is the development of unmanned aerial vehicles (UAVs), more commonly known as drones. They are a very good alternative that can be used to carry out topographic surveys through photogrammetry. In this project the Digital Terrain Models (DTM) of an overflown area in the municipality of Cubillos del Sil (León) are developed. Two main pieces of software have been used for this purpose; Mission Planner and PhotoModeler. Programming of the drone flight mission was done with the former, allowing it to take aerial pictures, whilst the latter was used to digitally rebuild the overflown area in 3D, using the pictures from the first piece of software. The final aim of the project is to make the obtained DTM available for any future project.

Biomechanics of smart wings in a bat robot: morphing wings using SMA actuators

This paper presents the design of a bat-like micro aerial vehicle with actuated morphing wings. NiTi shape memory alloys (SMAs) acting as artificial biceps and triceps muscles are used for mimicking the morphing wing mechanism of the bat flight apparatus. Our objective is twofold. Firstly, we have implemented a control architecture that allows an accurate and fast SMA actuation. This control makes use of the electrical resistance measurements of SMAs to adjust morphing wing motions. Secondly, the feasibility of using SMA actuation technology is evaluated for the application at hand. To this purpose, experiments are conducted to analyze the control performance in terms of nominal and overloaded operation modes of the SMAs. This analysis includes: (i) inertial forces regarding the stretchable wing membrane and aerodynamic loads, and (ii) uncertainties due to impact of airflow conditions over the resistance–motion relationship of SMAs. With the proposed control, morphing actuation speed can be increased up to 2.5 Hz, being sufficient to generate lift forces at a cruising speed of 5ms−1.

New Technique for Static Target Detection in Dense-Multipath Urban Environments

Synthetic Aperture Radar (SAR) images a target region reflectivity function in the multi-dimensional spatial domain of range and cross-range with a finer azimuth resolution than the one provided by any on-board real antenna. Conventional SAR techniques assume a single reflection of transmitted waveforms from targets. Nevertheless, new uses of Unmanned Aerial Vehicles (UAVs) for civilian-security applications force SAR systems to work in much more complex scenes such as urban environments. Consequently, multiple-bounce returns are additionally superposed to direct-scatter echoes. They are known as ghost images, since they obscure true target image and lead to poor resolution. All this may involve a significant problem in applications related to surveillance and security. In this work, an innovative multipath mitigation technique is presented in which Time Reversal (TR) concept is applied to SAR images when the target is concealed in clutter, leading to TR-SAR technique. This way, the effect of multipath is considerably reduced ?or even removed?, recovering the lost resolution due to multipath propagation. Furthermore, some focusing indicators such as entropy (E), contrast (C) and Rényi entropy (RE) provide us with a good focusing criterion when using TR-SAR.

Experimental investigation on box-wing configuration for UAS

The range for airframe configurations available for UAS is as diverse as those used for manned aircraft and more since the commercial risk in trying unorthodox solutions is less for the UAS manufacturer. This is principally because the UAS airframes are usually much smaller than the manned aircraft and operators are less likely to have a bias against unconventional configurations. One of these unconventional configurations is the box-wing, which is an unconventional solution for the design of the new UAS generation. The existence of two wings separated in different planes that are, however, significantly close together, means that the aerodynamic analysis by theoretical or computational methods is a difficult task, due to the considerable interference existing. Considering the fact that the flight of most UAS takes place at low Reynolds numbers, it is necessary to study the aerodynamics of the box wing configuration by testing different models in a wind tunnel to be able to obtain reasonable results. In the present work, the study is enhanced by varying not only the sweepback angles of the two wings, but also their position along the models’ fuselage. Certain models have shown being more efficient than others, pointing out that certain relative positions of wing exists that can improve the aerodynamics efficiency of the box wing configuration.

Quadcopter see and avoid using a fuzzy controller

Unmanned Aerial Vehicles (UAVs) industry is a fast growing sector. Nowadays, the market offers numerous possibilities for off-the-shelf UAVs such as quadrotors or fixed-wings. Until UAVs demonstrate advance capabilities such as autonomous collision avoidance they will be segregated and restricted to flight in controlled environments. This work presents a visual fuzzy servoing system for obstacle avoidance using UAVs. To accomplish this task we used the visual information from the front camera. Images are processed off-board and the result send to the Fuzzy Logic controller which then send commands to modify the orientation of the aircraft. Results from flight test are presented with a commercial off-the-shelf platform.

UAS See-and-Avoid Strategy using a Fuzzy Logic Controller Optimized by Cross-Entropy in Scaling Factors and Membership Functions

This work aims to develop a novel Cross-Entropy (CE) optimization-based fuzzy controller for Unmanned Aerial Monocular Vision-IMU System (UAMVIS) to solve the seeand- avoid problem using its accurate autonomous localization information. The function of this fuzzy controller is regulating the heading of this system to avoid the obstacle, e.g. wall. In the Matlab Simulink-based training stages, the Scaling Factor (SF) is adjusted according to the specified task firstly, and then the Membership Function (MF) is tuned based on the optimized Scaling Factor to further improve the collison avoidance performance. After obtained the optimal SF and MF, 64% of rules has been reduced (from 125 rules to 45 rules), and a large number of real flight tests with a quadcopter have been done. The experimental results show that this approach precisely navigates the system to avoid the obstacle. To our best knowledge, this is the first work to present the optimized fuzzy controller for UAMVIS using Cross-Entropy method in Scaling Factors and Membership Functions optimization.