Analysis of Thematic Classified Aerial Images Trough Multispectral and LIDAR Data

The application of thematic maps obtained through the classification of remote images needs the obtained products with an optimal accuracy. The registered images from the airplanes display a very satisfactory spatial resolution, but the classical methods of thematic classification not always give better results than when the registered data from satellite are used. In order to improve these results of classification, in this work, the LIDAR sensor data from first return (Light Detection And Ranging) registered simultaneously with the spectral sensor data from airborne are jointly used. The final results of the thematic classification of the scene object of study have been obtained, quantified and discussed with and without LIDAR data, after applying different methods: Maximum Likehood Classification, Support Vector Machine with four different functions kernel and Isodata clustering algorithm (ML, SVM-L, SVM-P, SVM-RBF, SVM-S, Isodata). The best results are obtained for SVM with Sigmoide kernel. These allow the correlation with others different physical parameters with great interest like Manning hydraulic coefficient, for their incorporation in a GIS and their application in hydraulic modeling.

Aerial video geo-registration using terrain models from dense and coherent stereo matching

In the context of aerial imagery, one of the first steps toward a coherent processing of the information contained in multiple images is geo-registration, which consists in assigning geographic 3D coordinates to the pixels of the image. This enables accurate alignment and geo-positioning of multiple images, detection of moving objects and fusion of data acquired from multiple sensors. To solve this problem there are different approaches that require, in addition to a precise characterization of the camera sensor, high resolution referenced images or terrain elevation models, which are usually not publicly available or out of date. Building upon the idea of developing technology that does not need a reference terrain elevation model, we propose a geo-registration technique that applies variational methods to obtain a dense and coherent surface elevation model that is used to replace the reference model. The surface elevation model is built by interpolation of scattered 3D points, which are obtained in a two-step process following a classical stereo pipeline: first, coherent disparity maps between image pairs of a video sequence are estimated and then image point correspondences are back-projected. The proposed variational method enforces continuity of the disparity map not only along epipolar lines (as done by previous geo-registration techniques) but also across them, in the full 2D image domain. In the experiments, aerial images from synthetic video sequences have been used to validate the proposed technique.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

La evolución del paisaje fluvial en la confluencia de los ríos Tajo y Jarama

La tesis doctoral que se presenta realiza un análisis de la evolución del paisaje fluvial de las riberas de los ríos Tajo y Jarama en el entorno de Aranjuez desde una perspectiva múltiple. Contempla y conjuga aspectos naturales, tales como los hidrológicos, geomorfológicos y ecológicos; también culturales, como la regulación hidrológica y la gestión del agua, las intervenciones en cauce y márgenes, la evolución de la propiedad y los cambios de usos del suelo, fundamentalmente. Este análisis ha permitido identificar el sistema de factores, dinámico y complejo, que ha creado este paisaje, así como las interrelaciones, conexiones, condicionantes y dependencias de los descriptores paisajísticos considerados. Por ejemplo, se han estudiado las relaciones cruzadas observadas entre dinámica fluvial-propiedad de la tierra-estado de conservación, cuestiones que hasta la fecha no habían sido tratadas, evaluadas o cuantificadas en otros trabajos dedicados a esta zona. La investigación se ha organizado en tres fases fundamentales que han dado lugar a los capítulos centrales del documento (capítulos 2, 3 y 4). En primer lugar, se ha realizado una caracterización de los factores, naturales y culturales, que organizan el paisaje de este territorio eminentemente fluvial (geomorfología, factores climáticos e hidrológicos, vegetación, propiedad de la tierra y elementos culturales de significación paisajística). A continuación, se ha realizado el estudio de la evolución del paisaje fluvial mediante el análisis de diversos elementos, previamente identificados y caracterizados. Para ello se han procesado imágenes aéreas correspondientes a cinco series temporales así como varios planos antiguos, obteniendo una amplia base de datos que se ha analizado estadísticamente. Finalmente, se han contrastado los resultados parciales obtenidos en los capítulos anteriores, lo que ha permitido identificar relaciones causales entre los factores que organizan el paisaje y la evolución de los elementos que lo constituyen. También, interconexiones entre factores o entre elementos. Este método de trabajo ha resultado muy útil para la comprensión del funcionamiento y evolución de un sistema complejo, como el paisaje de la vega de Aranjuez, un territorio con profundas y antiguas intervenciones culturales donde lo natural, en cualquier caso, siempre subyace. Es posible que la principal aportación de este trabajo, también su diferencia más destacada respecto a otros estudios de paisaje, haya sido mostrar una visión completa y exhaustiva de todos los factores que han intervenido en la conformación y evolución del paisaje fluvial, destacando las relaciones que se establecen entre ellos. Esta manera de proceder puede tener una interesante faceta aplicada, de tal manera que resulta un instrumento muy útil para el diseño de planes de gestión de este territorio fluvial. No en vano, una parte sustancial de la vega del Tajo-Jarama en Aranjuez es un Lugar de Importancia Comunitaria (LIC) y su posterior e ineludible declaración como Zona de Especial Conservación (ZEC) de la Red Natura 2000, de acuerdo con lo establecido en la Directiva 92/43/CE, exige la elaboración de un Plan de Gestión que, en gran medida, podría nutrirse de lo presentado, analizado e interpretado en este trabajo. En este sentido, conviene señalar la conciencia ya asumida de considerar, por su carácter integrador de la realidad territorial, el paisaje como elemento clave para la gestión adecuada de la naturaleza y el territorio. Por otra parte, se considera que los resultados de esta Tesis Doctoral permitirían plantear medidas para la puesta en valor de un paisaje sobresaliente, cuyos límites sobrepasan con creces los que en la actualidad conforman el Paisaje Cultural declarado por la UNESCO. En suma, el análisis de este espacio fluvial realizado con la profundidad y amplitud que permite el método de trabajo seguido puede utilizarse para el diseño de estrategias que dirijan la evolución de este territorio en una línea que garantice su conservación global en términos paisajísticos, patrimoniales y ecológicos, permitiendo además, de este modo, su uso equilibrado como recurso económico, cultural o educativo. This doctoral thesis shows an analysis of fluvial landscape evolution from multiple perspectives on the banks of Tagus and Jarama rivers, around Aranjuez. The thesis contemplates and combines natural features, such as hydrological, geomorphological and ecological features, as well as cultural features, like hydrological regulation and water management, interventions in channels and margins, changes in ownership and land use changes, mainly. This analysis has allowed to identify the factors system, dynamic and complex, that this landscape has created, as well as the interrelationships, connections, constraints and dependencies among considered landscape descriptors. For example, we have studied the relationships observed among fluvial dynamics- land ownership -conservation status, issues not addressed, assessed or quantified up to now in other works about this area. The research is organized into three major phases that led to the paper's central chapters (Chapters 2, 3 and 4). First, there has been a characterization of the factors, both natural and cultural, that organize the landscape of this predominantly fluvial area (geomorphology, climate and hydrological factors, vegetation, land and cultural elements of landscape significance). Then, it was made to study of fluvial landscape evolution by analyzing various elements previously identified and characterized. Aerial images were processed for five series and several old maps, obtaining an extensive database, that has been analyzed statistically. Finally, we have contrasted the partial results obtained in the previous chapters, making it possible to identify causal relationships between the factors that organize the landscape and the evolution of the elements that constitute it. This working method has been very useful for understanding the operation and evolution of a complex system, as the landscape of the Vega de Aranjuez, a territory with deep and ancient cultural interventions where anyway, nature feature always lies. It is possible that the main contribution of this work, also its most prominent difference compared with other studies of landscape, has been to show a complete and exhaustive view of all factors involved in the formation and evolution of the fluvial landscape, highlighting the relationships established among them. This approach could have an interesting applied facet, so that is a very useful tool for designing management plans on this river territory. Not surprisingly, a substantial part of the valley of the Tagus-Jarama in Aranjuez is a Site of Community Importance (SCI) and their subsequent and inevitable declaration as Special Area of Conservation (SAC) of the Natura 2000 network, in accordance with the provisions Directive 92/43/EC, requires the development of a management plan that largely could draw on what was presented, analyzed and interpreted in this paper. In this regard, it should be noted conscience and assumed to consider, on the inclusiveness of territorial reality, the landscape as a key element for the proper management of nature and territory. On the other hand, it is considered that the results of this thesis allow to propose measures for enhancement of outstanding scenery, which go well beyond the boundaries that currently the Cultural Landscape declared by UNESCO. In sum, the analysis of this river area made with the depth and breadth that enables working method can be used to design strategies that address the evolution of this territory in a line that guarantees global conservation landscape terms, heritage and ecological, also, allowing its use as a balancing economic, cultural or educational resource.

Land Cover Mapping by an Optimized Object-Oriented Approach. Case of Study Mediterranean Landscapes

Remote sensing information from spaceborne and airborne platforms continues to provide valuable data for different environmental monitoring applications. In this sense, high spatial resolution im-agery is an important source of information for land cover mapping. For the processing of high spa-tial resolution images, the object-based methodology is one of the most commonly used strategies. However, conventional pixel-based methods, which only use spectral information for land cover classification, are inadequate for classifying this type of images. This research presents a method-ology to characterise Mediterranean land covers in high resolution aerial images by means of an object-oriented approach. It uses a self-calibrating multi-band region growing approach optimised by pre-processing the image with a bilateral filtering. The obtained results show promise in terms of both segmentation quality and computational efficiency.

Fusion of probabilistic knowledge-based classification rules and learning automata for automatic recognition of digital images

In this paper, the fusion of probabilistic knowledge-based classification rules and learning automata theory is proposed and as a result we present a set of probabilistic classification rules with self-learning capability. The probabilities of the classification rules change dynamically guided by a supervised reinforcement process aimed at obtaining an optimum classification accuracy. This novel classifier is applied to the automatic recognition of digital images corresponding to visual landmarks for the autonomous navigation of an unmanned aerial vehicle (UAV) developed by the authors. The classification accuracy of the proposed classifier and its comparison with well-established pattern recognition methods is finally reported.

Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

Aerial Coverage Path Planning applied to Mapping

In the last decade we have seen how small and light weight aerial platforms - aka, Mini Unmanned Aerial Vehicles (MUAV) - shipped with heterogeneous sensors have become a 'most wanted' Remote Sensing (RS) tool. Most of the off-the-shelf aerial systems found in the market provide way-point navigation. However, they do not rely on a tool that compute the aerial trajectories considering all the aspects that allow optimizing the aerial missions. One of the most demanded RS applications of MUAV is image surveying. The images acquired are typically used to build a high-resolution image, i.e., a mosaic of the workspace surface. Although, it may be applied to any other application where a sensor-based map must be computed. This thesis provides a study of this application and a set of solutions and methods to address this kind of aerial mission by using a fleet of MUAVs. In particular, a set of algorithms are proposed for map-based sampling, and aerial coverage path planning (ACPP). Regarding to map-based sampling, the approaches proposed consider workspaces with different shapes and surface characteristics. The workspace is sampled considering the sensor characteristics and a set of mission requirements. The algorithm applies different computational geometry approaches, providing a unique way to deal with workspaces with different shape and surface characteristics in order to be surveyed by one or more MUAVs. This feature introduces a previous optimization step before path planning. After that, the ACPP problem is theorized and a set of ACPP algorithms to compute the MUAVs trajectories are proposed. The problem addressed herein is the problem to coverage a wide area by using MUAVs with limited autonomy. Therefore, the mission must be accomplished in the shortest amount of time. The aerial survey is usually subject to a set of workspace restrictions, such as the take-off and landing positions as well as a safety distance between elements of the fleet. Moreover, it has to avoid forbidden zones to y. Three different algorithms have been studied to address this problem. The approaches studied are based on graph searching, heuristic and meta-heuristic approaches, e.g., mimic, evolutionary. Finally, an extended survey of field experiments applying the previous methods, as well as the materials and methods adopted in outdoor missions is presented. The reported outcomes demonstrate that the findings attained from this thesis improve ACPP mission for mapping purpose in an efficient and safe manner.

Aerial Coverage Path Planning applied to Mapping

In the last decade we have seen how small and light weight aerial platforms - aka, Mini Unmanned Aerial Vehicles (MUAV) - shipped with heterogeneous sensors have become a 'most wanted' Remote Sensing (RS) tool. Most of the off-the-shelf aerial systems found in the market provide way-point navigation. However, they do not rely on a tool that compute the aerial trajectories considering all the aspects that allow optimizing the aerial missions. One of the most demanded RS applications of MUAV is image surveying. The images acquired are typically used to build a high-resolution image, i.e., a mosaic of the workspace surface. Although, it may be applied to any other application where a sensor-based map must be computed. This thesis provides a study of this application and a set of solutions and methods to address this kind of aerial mission by using a fleet of MUAVs. In particular, a set of algorithms are proposed for map-based sampling, and aerial coverage path planning (ACPP). Regarding to map-based sampling, the approaches proposed consider workspaces with different shapes and surface characteristics. The workspace is sampled considering the sensor characteristics and a set of mission requirements. The algorithm applies different computational geometry approaches, providing a unique way to deal with workspaces with different shape and surface characteristics in order to be surveyed by one or more MUAVs. This feature introduces a previous optimization step before path planning. After that, the ACPP problem is theorized and a set of ACPP algorithms to compute the MUAVs trajectories are proposed. The problem addressed herein is the problem to coverage a wide area by using MUAVs with limited autonomy. Therefore, the mission must be accomplished in the shortest amount of time. The aerial survey is usually subject to a set of workspace restrictions, such as the take-off and landing positions as well as a safety distance between elements of the fleet. Moreover, it has to avoid forbidden zones to y. Three different algorithms have been studied to address this problem. The approaches studied are based on graph searching, heuristic and meta-heuristic approaches, e.g., mimic, evolutionary. Finally, an extended survey of field experiments applying the previous methods, as well as the materials and methods adopted in outdoor missions is presented. The reported outcomes demonstrate that the findings attained from this thesis improve ACPP mission for mapping purpose in an efficient and safe manner.

What is the optimal stitching orientation from an aerial survey perspective?

Mosaicing is a technique that allows obtaining a large high resolution image by stitching several images together. These base images are usually acquired from an elevated point of view. Until recently, low-altitude image acquisition has been performed typically by using using airplanes, as well as other manned platforms. However, mini unmanned aerial vehicles (MUAV) endowed with a camera have lately made this task more available for small for cicil applications, for example for small farmers in order to obtain accurate agronomic information about their crop fields. The stitching orientation, or the image acquisition orientation usually coincides with the aircraft heading assuming a downwards orientation of the camera. In this paper, the efect of the image orientation in the eficiency of the aerial coverage path planning is studied. Moreover, an algorithm to compute an optimal stitching orientation angle is proposed and results are numerically compared with classical approaches.