Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

New generation of human machine interfaces for controlling UAV through depth based gesture recognition

New forms of natural interactions between human operators and UAVs (Unmanned Aerial Vehicle) are demanded by the military industry to achieve a better balance of the UAV control and the burden of the human operator. In this work, a human machine interface (HMI) based on a novel gesture recognition system using depth imagery is proposed for the control of UAVs. Hand gesture recognition based on depth imagery is a promising approach for HMIs because it is more intuitive, natural, and non-intrusive than other alternatives using complex controllers. The proposed system is based on a Support Vector Machine (SVM) classifier that uses spatio-temporal depth descriptors as input features. The designed descriptor is based on a variation of the Local Binary Pattern (LBP) technique to efficiently work with depth video sequences. Other major consideration is the especial hand sign language used for the UAV control. A tradeoff between the use of natural hand signs and the minimization of the inter-sign interference has been established. Promising results have been achieved in a depth based database of hand gestures especially developed for the validation of the proposed system.

Software tool for simulation of aircraft trajectories

New concepts in air navigation have been introduced recently. Among others, are the concepts of trajectory optimization, 4D trajectories, RBT (Reference Business Trajectory), TBO (trajectory based operations), CDA (Continuous Descent Approach) and ACDA (Advanced CDA), conflict resolution, arrival time (AMAN), introduction of new aircraft (UAVs, UASs) in air space, etc. Although some of these concepts are new, the future Air Traffic Management will maintain the four ATM key performance areas such as Safety, Capacity, Efficiency, and Environmental impact. So much, the performance of the ATM system is directly related to the accuracy with which the future evolution of the traffic can be predicted. In this sense, future air traffic management will require a variety of support tools to provide suitable help to users and engineers involved in the air space management. Most of these tools are based on an appropriate trajectory prediction module as main component. Therefore, the purposes of these tools are related with testing and evaluation of any air navigation concept before they become fully operative. The aim of this paper is to provide an overview to the design of a software tool useful to estimate aircraft trajectories adapted to air navigation concepts. Other usage of the tool, like controller design, vertical navigation assessment, procedures validation and hardware and software in the loop are available in the software tool. The paper will show the process followed to design the tool, the software modules needed to perform accurately and the process followed to validate the output data.

Sistema de soporte al desarrollo de un planificador de misiones en un vehículo aéreo no tripulado

En esta memoria se describe el trabajo de construcción de una arquitectura software diseñada para facilitar el desarrollo un planificador de misión de un vehículo aéreo no tripulado (UAV), con el fin de que éste alcance los objetivos marcados en la competición internacional de robótica IARC (séptima edición). A lo largo de la memoria, se describe en primer lugar, una revisión de técnicas de robótica inteligente aplicadas a la construcción de vehículos aéreos no tripulados, en el que se ven los diferentes paradigmas de programación de la robótica inteligente y la clasificación de dichos robots aéreos, dependiendo de su autonomía. Este descripción finaliza con la presentación del problema correspondiente a la competición IARC. A continuación se describe el diseño realizado para soporte al desarrollo de un planificador de misiones de UAVs, con simulación de comportamiento de vehículos robóticos y visualización 3D con movimiento. Finalmente, se muestran las pruebas que se han realizado para validar la construcción de dicha arquitectura software. ---ABSTRACT---In this report it is presented the construction of a software architecture, designed to facilitate the development of a mission planner for an unmanned aerial vehicle (UAV), so that it reaches the goals set in the International Aerial Robotics Competition - IARC (seventh edition). Throughout this report, it is described first, a review of intelligent robotics techniques applied to the construction of unmanned aerial vehicles, where different paradigms of intelligent robotics are seen, along with a classification of such aerial robots, depending on their autonomy. Description ends with the presentation of the problem corresponding to the IARC competition. Following, it is described the design made to satisfy the support to the development of a mission planner for UAV´s, with a simulation of the robotics vehicles’ behaviours and a 3D display with motion. Finally, we will deal with the tests that have been conducted to validate the construction of the software architecture.

Modelos digitales del terreno mediante fotogrametría aérea realizada con un vehículo aéreo no tripulado

En las últimas décadas, el avance tecnológico ha aumentado a una velocidad vertiginosa. Muchos son los campos beneficiados, entre ellos la ingeniería y por consiguiente el desarrollo de maquinaria, técnicas y herramientas que facilitan el trabajo. Los vehículos aéreos no tripulados (UAVs), más conocidos como drones, se presentan como una alternativa muy interesante para llevar a cabo levantamientos topográficos mediante la técnica fotogramétrica. En este proyecto se desarrollan los Modelos Digitales del Terreno (MDT) de una superficie sobrevolada por un UAV en la localidad de Cubillos del Sil (León). Para ello, se han utilizado dos software: Mission Planner y PhotoModeler. En el primero se ha llevado a cabo la programación de la misión de vuelo del drone, siendo el fin la toma de fotografías aéreas. El segundo es el encargado de realizar la reconstrucción digital en 3D del terreno sobrevolado a partir de esas fotografías. El objetivo final es que los MDT obtenidos puedan ser utilizados en un futuro por cualquier equipo de trabajo que quiera desarrollar un proyecto sobre esos terrenos. ABSTRACT The pace of technology innovation is faster now than it has been for the past few decades. Engineering, and consequently machinery and diverse techniques, are the main beneficiaries. A clear example of this is the development of unmanned aerial vehicles (UAVs), more commonly known as drones. They are a very good alternative that can be used to carry out topographic surveys through photogrammetry. In this project the Digital Terrain Models (DTM) of an overflown area in the municipality of Cubillos del Sil (León) are developed. Two main pieces of software have been used for this purpose; Mission Planner and PhotoModeler. Programming of the drone flight mission was done with the former, allowing it to take aerial pictures, whilst the latter was used to digitally rebuild the overflown area in 3D, using the pictures from the first piece of software. The final aim of the project is to make the obtained DTM available for any future project.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Robust Real-time Vision-based Aircraft Tracking From Unmanned Aerial Vehicles.

Aircraft tracking plays a key and important role in the Sense-and-Avoid system of Unmanned Aerial Vehicles (UAVs). This paper presents a novel robust visual tracking algorithm for UAVs in the midair to track an arbitrary aircraft at real-time frame rates, together with a unique evaluation system. This visual algorithm mainly consists of adaptive discriminative visual tracking method, Multiple-Instance (MI) learning approach, Multiple-Classifier (MC) voting mechanism and Multiple-Resolution (MR) representation strategy, that is called Adaptive M3 tracker, i.e. AM3. In this tracker, the importance of test sample has been integrated to improve the tracking stability, accuracy and real-time performances. The experimental results show that this algorithm is more robust, efficient and accurate against the existing state-of-art trackers, overcoming the problems generated by the challenging situations such as obvious appearance change, variant surrounding illumination, partial aircraft occlusion, blur motion, rapid pose variation and onboard mechanical vibration, low computation capacity and delayed information communication between UAVs and Ground Station (GS). To our best knowledge, this is the first work to present this tracker for solving online learning and tracking freewill aircraft/intruder in the UAVs.

A Ground-Truth Video Dataset for the Development and Evaluation of Vision-based Sense-and-Avoid systems

The importance of vision-based systems for Sense-and-Avoid is increasing nowadays as remotely piloted and autonomous UAVs become part of the non-segregated airspace. The development and evaluation of these systems demand flight scenario images which are expensive and risky to obtain. Currently Augmented Reality techniques allow the compositing of real flight scenario images with 3D aircraft models to produce useful realistic images for system development and benchmarking purposes at a much lower cost and risk. With the techniques presented in this paper, 3D aircraft models are positioned firstly in a simulated 3D scene with controlled illumination and rendering parameters. Realistic simulated images are then obtained using an image processing algorithm which fuses the images obtained from the 3D scene with images from real UAV flights taking into account on board camera vibrations. Since the intruder and camera poses are user-defined, ground truth data is available. These ground truth annotations allow to develop and quantitatively evaluate aircraft detection and tracking algorithms. This paper presents the software developed to create a public dataset of 24 videos together with their annotations and some tracking application results.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Real-Time Adaptive Multi-Classifier Multi-Resolution Visual Tracking Framework for Unmanned Aerial Vehicles

This paper presents a novel robust visual tracking framework, based on discriminative method, for Unmanned Aerial Vehicles (UAVs) to track an arbitrary 2D/3D target at real-time frame rates, that is called the Adaptive Multi-Classifier Multi-Resolution (AMCMR) framework. In this framework, adaptive Multiple Classifiers (MC) are updated in the (k-1)th frame-based Multiple Resolutions (MR) structure with compressed positive and negative samples, and then applied them in the kth frame-based Multiple Resolutions (MR) structure to detect the current target. The sample importance has been integrated into this framework to improve the tracking stability and accuracy. The performance of this framework was evaluated with the Ground Truth (GT) in different types of public image databases and real flight-based aerial image datasets firstly, then the framework has been applied in the UAV to inspect the Offshore Floating Platform (OFP). The evaluation and application results show that this framework is more robust, efficient and accurate against the existing state-of-art trackers, overcoming the problems generated by the challenging situations such as obvious appearance change, variant illumination, partial/full target occlusion, blur motion, rapid pose variation and onboard mechanical vibration, among others. To our best knowledge, this is the first work to present this framework for solving the online learning and tracking freewill 2D/3D target problems, and applied it in the UAVs.

Las Redes de Petri Interpretadas en el Diagnóstico de Fallos de Sistemas Híbridos. Aplicación a un Helicóptero no Tripulado

La seguridad y fiabilidad de los procesos industriales son la principal preocupación de los ingenieros encargados de las plantas industriales. Por lo tanto, desde un punto de vista económico, el objetivo principal es reducir el costo del mantenimiento, el tiempo de inactividad y las pérdidas causadas por los fallos. Por otra parte, la seguridad de los operadores, que afecta a los aspectos sociales y económicos, es el factor más relevante a considerar en cualquier sistema Debido a esto, el diagnóstico de fallos se ha convertido en un foco importante de interés para los investigadores de todo el mundo e ingenieros en la industria. Los principales trabajos enfocados en detección de fallos se basan en modelos de los procesos. Existen diferentes técnicas para el modelado de procesos industriales tales como máquinas de estado, árboles de decisión y Redes de Petri (RdP). Por lo tanto, esta tesis se centra en el modelado de procesos utilizando redes de petri interpretadas. Redes de Petri es una herramienta usada en el modelado gráfico y matemático con la habilidad para describir información de los sistemas de una manera concurrente, paralela, asincrona, distribuida y no determinística o estocástica. RdP son también una herramienta de comunicación visual gráfica útil como lo son las cartas de flujo o diagramas de bloques. Adicionalmente, las marcas de las RdP simulan la dinámica y concurrencia de los sistemas. Finalmente, ellas tienen la capacidad de definir ecuaciones de estado específicas, ecuaciones algebraicas y otros modelos que representan el comportamiento común de los sistemas. Entre los diferentes tipos de redes de petri (Interpretadas, Coloreadas, etc.), este trabajo de investigación trata con redes de petri interpretadas principalmente debido a características tales como sincronización, lugares temporizados, aparte de su capacidad para procesamiento de datos. Esta investigación comienza con el proceso para diseñar y construir el modelo y diagnosticador para detectar fallos definitivos, posteriormente, la dinámica temporal fue adicionada para detectar fallos intermitentes. Dos procesos industriales, concretamente un HVAC (Calefacción, Ventilación y Aire Acondicionado) y un Proceso de Envasado de Líquidos fueron usados como banco de pruebas para implementar la herramienta de diagnóstico de fallos (FD) creada. Finalmente, su capacidad de diagnóstico fue ampliada en orden a detectar fallos en sistemas híbridos. Finalmente, un pequeño helicóptero no tripulado fue elegido como ejemplo de sistema donde la seguridad es un desafío, y las técnicas de detección de fallos desarrolladas en esta tesis llevan a ser una herramienta valorada, desde que los accidentes de las aeronaves no tripuladas (UAVs) envuelven un alto costo económico y son la principal razón para introducir restricciones de volar sobre áreas pobladas. Así, este trabajo introduce un proceso sistemático para construir un Diagnosticador de Fallos del sistema mencionado basado en RdR Esta novedosa herramienta es capaz de detectar fallos definitivos e intermitentes. El trabajo realizado es discutido desde un punto de vista teórico y práctico. El procedimiento comienza con la división del sistema en subsistemas para seguido integrar en una RdP diagnosticadora global que es capaz de monitorear el sistema completo y mostrar las variables críticas al operador en orden a determinar la salud del UAV, para de esta manera prevenir accidentes. Un Sistema de Adquisición de Datos (DAQ) ha sido también diseñado para recoger datos durante los vuelos y alimentar la RdP diagnosticadora. Vuelos reales realizados bajo condiciones normales y de fallo han sido requeridos para llevar a cabo la configuración del diagnosticador y verificar su comportamiento. Vale la pena señalar que un alto riesgo fue asumido en la generación de fallos durante los vuelos, a pesar de eso esto permitió recoger datos básicos para desarrollar el diagnóstico de fallos, técnicas de aislamiento, protocolos de mantenimiento, modelos de comportamiento, etc. Finalmente, un resumen de la validación de resultados obtenidos durante las pruebas de vuelo es también incluido. Un extensivo uso de esta herramienta mejorará los protocolos de mantenimiento para UAVs (especialmente helicópteros) y permite establecer recomendaciones en regulaciones. El uso del diagnosticador usando redes de petri es considerado un novedoso enfoque. ABSTRACT Safety and reliability of industrial processes are the main concern of the engineers in charge of industrial plants. Thus, from an economic point of view, the main goal is to reduce the maintenance downtime cost and the losses caused by failures. Moreover, the safety of the operators, which affects to social and economic aspects, is the most relevant factor to consider in any system. Due to this, fault diagnosis has become a relevant focus of interest for worldwide researchers and engineers in the industry. The main works focused on failure detection are based on models of the processes. There are different techniques for modelling industrial processes such as state machines, decision trees and Petri Nets (PN). Thus, this Thesis is focused on modelling processes by using Interpreted Petri Nets. Petri Nets is a tool used in the graphic and mathematical modelling with ability to describe information of the systems in a concurrent, parallel, asynchronous, distributed and not deterministic or stochastic manner. PNs are also useful graphical visual communication tools as flow chart or block diagram. Additionally, the marks of the PN simulate the dynamics and concurrence of the systems. Finally, they are able to define specific state equations, algebraic equations and other models that represent the common behaviour of systems. Among the different types of PN (Interpreted, Coloured, etc.), this research work deals with the interpreted Petri Nets mainly due to features such as synchronization capabilities, timed places, apart from their capability for processing data. This Research begins with the process for designing and building the model and diagnoser to detect permanent faults, subsequently, the temporal dynamic was added for detecting intermittent faults. Two industrial processes, namely HVAC (Heating, Ventilation and Air Condition) and Liquids Packaging Process were used as testbed for implementing the Fault Diagnosis (FD) tool created. Finally, its diagnostic capability was enhanced in order to detect faults in hybrid systems. Finally, a small unmanned helicopter was chosen as example of system where safety is a challenge and fault detection techniques developed in this Thesis turn out to be a valuable tool since UAVs accidents involve high economic cost and are the main reason for setting restrictions to fly over populated areas. Thus, this work introduces a systematic process for building a Fault Diagnoser of the mentioned system based on Petri Nets. This novel tool is able to detect both intermittent and permanent faults. The work carried out is discussed from theoretical and practical point of view. The procedure begins with a division of the system into subsystems for further integration into a global PN diagnoser that is able to monitor the whole system and show critical variables to the operator in order to determine the UAV health, preventing accidents in this manner. A Data Acquisition System (DAQ) has been also designed for collecting data during the flights and feed PN Diagnoser. Real flights carried out under nominal and failure conditions have been required to perform the diagnoser setup and verify its performance. It is worth noting that a high risk was assumed in the generation of faults during the flights, nevertheless this allowed collecting basic data so as to develop fault diagnosis, isolations techniques, maintenance protocols, behaviour models, etc. Finally, a summary of the validation results obtained during real flight tests is also included. An extensive use of this tool will improve preventive maintenance protocols for UAVs (especially helicopters) and allow establishing recommendations in regulations. The use of the diagnoser by using Petri Nets is considered as novel approach.

Herramienta de interacción persona-ordenador para la operación de vehículos aéreos no tripulados

En la presente memoria se describe el trabajo de diseño de una herramienta de interacción persona-ordenador (HMI) para la operación y supervisión de vehículos aéreos no tripulados (UAV). En primer lugar se hace una introducción a los tipos de UAVs y aplicaciones más comunes, describiendo sus características técnicas y los componentes que integra en el sistema. Mediante la revisión y análisis de los diferentes niveles de autonomía y las diferentes soluciones de presentación existentes en el mercado, se identifican los modos de operación y componentes principales de la interfaz. A continuación se describe el diseño final del software de la interfaz y el proceso de desarrollo de la misma, para ello se hace un análisis previo del software robótico sobre el que opera el sistema abordo del UAV y se establecen los enlaces de comunicación entre cada uno de los componentes y los requisitos de integración con el sistema. Finalmente, se muestran las pruebas que se han realizado para validar la construcción de la herramienta. This report outlines the design and construction of a human-machine interface (HMI), designed to facilitate the supervision and operation with unmanned aerial vehicles (UAV). First, it is described an introduction to UAVs classification and application fields, reviewing the hardware features and software integration components. In order to define the basic components and operation modes in the general design, a brief review of the different presentation solutions and autonomous levels is described. As a result, it is presented the final software design, the components details and the system integration requirements. Finally, it is also concluded with some of the tests that have been conducted to validate the design and construction of the human-machine interface

Entorno de supervisión para vehículos aéreos no tripulados

La Universidad Politécnica de Madrid está investigando en el campo de la robótica inteligente, concretamente con el empleo de vehículos aéreos no tripulados (UAV). El objetivo final que se persigue con las investigaciones en este campo es el desarrollo de sistemas capaces de operar de forma más autónoma en un amplio espectro de situaciones. Dentro de este marco, este trabajo fin de grado se centra en el desarrollo de un sistema de supervisión para UAVs que persigue facilitar la monitorización de la ejecución de los procesos y facilitar la inclusión de procedimientos para incrementar la tolerancia a los fallos software. A lo largo de esta memoria se ofrece una revisión del estado del arte en el ámbito de la robótica, haciendo especial hincapié en la robótica inteligente con los métodos de desarrollo existentes y la definición de los distintos marcos de clasificación de la autonomía. También se ofrece una vista a las distintas técnicas existentes para lograr una mayor tolerancia a los fallos software, de entre las que han sido seleccionadas varias de ellas en la realización de este trabajo. Finalmente se describe el sistema de supervisión desarrollado, explicando primero el sistema desde un punto de vista funcional para más adelante adentrarse en la solución técnica elaborada. ---ABSTRACT--- The Universidad Politécnica de Madrid is currently handling several investigations regarding AI robotics, some of them are actually directing their efforts into the use of unmanned aerial vehicles (UAV). The goal in the long term for this investigations is the accomplishment of systems capable of operating autonomously, regardless of the situation the robot is place at. From this perspective, this final degree project focuses on de design and development of a supervision system for UAV’s, which function is to ease the monitoring of executing processes and the inclusion of fault tolerant procedures. During the development of this document a state of the art revision is offered, in which a thorough description through development methods and autonomy definitions for AI robotics is made. It is also offered a look around the different existing techniques for achieving a greater software fault tolerance, from which some of them were chosen for the development of this project. Finally the developed supervision system is described, first from a pure functional perspective of what the system should do and latter with a description of the actual technical solutions developed for this system.

Systematic Process for Building a Fault Diagnoser Based on Petri Nets Applied to a Helicopter

This work presents a systematic process for building a Fault Diagnoser (FD), based on Petri Nets (PNs) which has been applied to a small helicopter. This novel tool is able to detect both intermittent and permanent faults. The work carried out is discussed from theoretical and practical point of view. The procedure begins with a division of the whole system into subsystems, which are the devices that have to be modeled by using PN, considering both the normal and fault operations. Subsequently, the models are integrated into a global Petri Net diagnoser (PND) that is able to monitor a whole helicopter and show critical variables to the operator in order to determine the UAV health, preventing accidents in this manner. A Data Acquisition System (DAQ) has been designed for collecting data during the flights and feeding PN diagnoser with them. Several real flights (nominal or under failure) have been carried out to perform the diagnoser setup and verify its performance. A summary of the validation results obtained during real flight tests is also included. An extensive use of this tool will improve preventive maintenance protocols for UAVs (especially helicopters) and allow establishing recommendations in regulations

Multi-UAV Coordination and Control Interface

The interest in missions with multiple Unmanned Aerial Vehicles (UAVs) has increased significantly in last years. These missions take advantage of the use of fleets instead of single UAVs to ensure the success, reduce the duration or increase the goals of the mission. In addition, they allow performing tasks that require multiple agents and certain coordination (e.g. surveillance of large areas or transport of heavy loads). Nevertheless, these missions suppose a challenge in terms of control and monitoring. In fact, the workload of the operators rises with the utilization of multiple UAVs and payloads, since they have to analyze more information, make more decisions and generate more commands during the mission. This work addresses the operator workload problem in multi-UAV missions by reducing and selecting the information. Two approaches are considered: a first one that selects the information according to the mission state, and a second one that selects it according to the operator preferences. The result is an interface that is able to control the amount of information and show what is relevant for mission and operator at the time.