Analysis of Thematic Classified Aerial Images Trough Multispectral and LIDAR Data

The application of thematic maps obtained through the classification of remote images needs the obtained products with an optimal accuracy. The registered images from the airplanes display a very satisfactory spatial resolution, but the classical methods of thematic classification not always give better results than when the registered data from satellite are used. In order to improve these results of classification, in this work, the LIDAR sensor data from first return (Light Detection And Ranging) registered simultaneously with the spectral sensor data from airborne are jointly used. The final results of the thematic classification of the scene object of study have been obtained, quantified and discussed with and without LIDAR data, after applying different methods: Maximum Likehood Classification, Support Vector Machine with four different functions kernel and Isodata clustering algorithm (ML, SVM-L, SVM-P, SVM-RBF, SVM-S, Isodata). The best results are obtained for SVM with Sigmoide kernel. These allow the correlation with others different physical parameters with great interest like Manning hydraulic coefficient, for their incorporation in a GIS and their application in hydraulic modeling.

Omnidirectional bearing-only see-and-avoid for small aerial robots:

Motivated by the growing interest in unmanned aerial system's applications in indoor and outdoor settings and the standardisation of visual sensors as vehicle payload. This work presents a collision avoidance approach based on omnidirectional cameras that does not require the estimation of range between two platforms to resolve a collision encounter. It will achieve a minimum separation between the two vehicles involved by maximising the view-angle given by the omnidirectional sensor. Only visual information is used to achieve avoidance under a bearing-only visual servoing approach. We provide theoretical problem formulation, as well as results from real flight using small quadrotors

3D Object following based on visual information for Unmanned Aerial Vehicles

This article presents a novel system and a control strategy for visual following of a 3D moving object by an Unmanned Aerial Vehicle UAV. The presented strategy is based only on the visual information given by an adaptive tracking method based on the color information, which jointly with the dynamics of a camera fixed to a rotary wind UAV are used to develop an Image-based visual servoing IBVS system. This system is focused on continuously following a 3D moving target object, maintaining it with a fixed distance and centered on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Precision Agriculture without borders: Practical issues and improvements in farmland coverage with aerial vehicles.

This work presents a solution for the aerial coverage of a field by using a fleet of aerial vehicles. The use of Unmanned Aerial Vehicles allows to obtain high resolution mosaics to be used in Precision Agriculture techniques. This report is focus on providing a solution for the full simultaneous coverage problem taking into account restrictions as the required spatial resolution and overlap while maintaining similar light conditions and safety operation of the drones. Results obtained from real field tests are finally reported

A waypoint-based mission planner for a farmland coverage with an aerial robot - a precision farming tool

Remote sensing (RS) with aerial robots is becoming more usual in every day time in Precision Agriculture (PA) practices, do to their advantages over conventional methods. Usually, available commercial platforms providing off-the-shelf waypoint navigation are adopted to perform visual surveys over crop fields, with the purpose to acquire specific image samples. The way in which a waypoint list is computed and dispatched to the aerial robot when mapping non empty agricultural workspaces has not been yet discussed. In this paper we propose an offline mission planner approach that computes an efficient coverage path subject to some constraints by decomposing the environment approximately into cells. Therefore, the aim of this work is contributing with a feasible waypoints-based tool to support PA practices

Techniques for Area Discretization and Coverage in Aerial Photography for Precision Agriculture employing mini quad-rotors

Remote sensed imagery acquired with mini aerial vehicles, in conjunction with GIS technology enable a meticulous analysis from surveyed agricultural sites. This paper sums up the ongoing work in area discretization and coverage with mini quad-­?rotors applied to Precision Agriculture practices under the project RHEA.

A Multiresolution Image Alignment Technique Based on Direct Methods for Pose Estimation of Aerial Vehicles

In this paper, we seek to expand the use of direct methods in real-time applications by proposing a vision-based strategy for pose estimation of aerial vehicles. The vast majority of approaches make use of features to estimate motion. Conversely, the strategy we propose is based on a MR (Multi-Resolution) implementation of an image registration technique (Inverse Compositional Image Alignment ICIA) using direct methods. An on-board camera in a downwards-looking configuration, and the assumption of planar scenes, are the bases of the algorithm. The motion between frames (rotation and translation) is recovered by decomposing the frame-to-frame homography obtained by the ICIA algorithm applied to a patch that covers around the 80% of the image. When the visual estimation is required (e.g. GPS drop-out), this motion is integrated with the previous known estimation of the vehicles' state, obtained from the on-board sensors (GPS/IMU), and the subsequent estimations are based only on the vision-based motion estimations. The proposed strategy is tested with real flight data in representative stages of a flight: cruise, landing, and take-off, being two of those stages considered critical: take-off and landing. The performance of the pose estimation strategy is analyzed by comparing it with the GPS/IMU estimations. Results show correlation between the visual estimation obtained with the MR-ICIA and the GPS/IMU data, that demonstrate that the visual estimation can be used to provide a good approximation of the vehicle's state when it is required (e.g. GPS drop-outs). In terms of performance, the proposed strategy is able to maintain an estimation of the vehicle's state for more than one minute, at real-time frame rates based, only on visual information.

Aerial Object Following Using Visual Fuzzy Servoing

This article presents a visual servoing system to follow a 3D moving object by a Micro Unmanned Aerial Vehicle (MUAV). The presented control strategy is based only on the visual information given by an adaptive tracking method based on the colour information. A visual fuzzy system has been developed for servoing the camera situated on a rotary wing MAUV, that also considers its own dynamics. This system is focused on continuously following of an aerial moving target object, maintaining it with a fixed safe distance and centred on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Aerial Remote Sensing in Agriculture: A Practical Approach to Area Coverage and Path Planning for Fleets of Mini Aerial Robots

In this paper, a system that allows applying precision agriculture techniques is described. The application is based on the deployment of a team of unmanned aerial vehicles that are able to take georeferenced pictures in order to create a full map by applying mosaicking procedures for postprocessing. The main contribution of this work is practical experimentation with an integrated tool. Contributions in different fields are also reported. Among them is a new one-phase automatic task partitioning manager, which is based on negotiation among the aerial vehicles, considering their state and capabilities. Once the individual tasks are assigned, an optimal path planning algorithm is in charge of determining the best path for each vehicle to follow. Also, a robust flight control based on the use of a control law that improves the maneuverability of the quadrotors has been designed. A set of field tests was performed in order to analyze all the capabilities of the system, from task negotiations to final performance. These experiments also allowed testing control robustness under different weather conditions.

Utilización del cultivo plurianual de pataca (Helianthus tuberosus L.) para la producción de hidratos de carbono fermentables a partir de los tallos

La pataca (Helianthus tuberosus L.) es una especie de cultivo con un alto potencial en la producción de hidratos de carbono de reserva en forma de polifructanos, especialmente inulina, que se acumulan temporalmente en los tallos en forma de polisacáridos para translocarse posteriormente a los tubérculos, donde son almacenados. Aunque tradicionalmente el producto de interés del cultivo son los tubérculos, que acumulan gran cantidad de hidratos de carbono fermentables (HCF) cuando se recogen al final del ciclo de desarrollo, en este trabajo se pretende evaluar el potencial de la pataca como productor de HCF a partir de los tallos cosechados en el momento de máximo contenido en HCF, mediante un sistema de cultivo plurianual. Se han realizado los siguientes estudios: i) Determinación del momento óptimo de cosecha en ensayos con 12 clones ii) Potencial del cultivo plurianual de la pataca en términos de producción anual de biomasa aérea y de HCF en cosechas sucesivas, iii) Ensayos de conservación de la biomasa aérea, iv) Estimación de los costes de las dos modalidades de cultivo de pataca para producción de HCF y v) Estimación de la sostenibilidad energética de la producción de bioetanol mediante la utilización de los subproductos. Para la determinación del momento óptimo de la cosecha de la biomasa aérea se ensayaron 12 clones de diferente precocidad en Madrid; 4 tempranos (Huertos de Moya, C-17, Columbia y D-19) y 8 tardíos (Boniches, China, K-8, Salmantina, Nahodka, C-13, INIA y Violeta de Rennes). El máximo contenido en HCF tuvo lugar en el estado fenológico de botón floral-flor que además coincidió con la máxima producción de biomasa aérea. De acuerdo con los resultados obtenidos, la cosecha de los clones tempranos se debería realizar en el mes de julio y en los clones tardíos en septiembre, siendo éstos últimos más productivos. La producción media más representativa entre los 12 clones, obtenida en el estado fenológico de botón floral fue de 23,40 t ms/ha (clon INIA), con un contenido medio en HCF de 30,30 % lo que supondría una producción potencial media de 7,06 t HCF/ha. La producción máxima en HCF se obtuvo en el clon Boniches con 7,61 t/ha y 22,81 t ms/ha de biomasa aérea. En el sistema de cultivo plurianual la cantidad de tallos por unidad de superficie aumenta cada año debido a la cantidad de tubérculos que van quedando en el terreno, sobre todo a partir del 3er año, lo que produce la disminución del peso unitario de los tallos, con el consiguiente riesgo de encamado. El aclareo de los tallos nacidos a principios de primavera mediante herbicidas tipo Glifosato o mediante una labor de rotocultor rebaja la densidad final de tallos y mejora los rendimientos del cultivo. En las experiencias de conservación de la biomasa aérea se obtuvo una buena conservación por un período de 6 meses de los HCF contenidos en los tallos secos empacados y almacenados bajo cubierta. Considerando que el rendimiento práctico de la fermentación alcohólica es de 0,5 l de etanol por cada kg de azúcar, la producción potencial de etanol para una cosecha de tallos de 7,06 t de HCF/ha sería de 3.530 l/ha. El bagazo producido en la extracción de los HCF de la biomasa aérea supondría 11,91 t/ha lo que utilizado para fines térmicos supone más de 3 veces la energía primaria requerida en el proceso de producción de etanol, considerando un poder calorífico inferior de 3.832,6 kcal/kg. Para una producción de HCF a partir de la biomasa aérea de 7,06 t/ha y en tubérculos al final del ciclo de 12,11 t/ha, los costes de producción estimados para cada uno de ellos fueron de 184,69 €/t para los HCF procedentes de la biomasa aérea y 311,30 €/t para los de tubérculos. Como resultado de este trabajo se puede concluir que la producción de HCF a partir de la biomasa aérea de pataca en cultivo plurianual, es viable desde un punto de vista técnico, con reducción de los costes de producción respecto al sistema tradicional de cosecha de tubérculos. Entre las ventajas técnicas de esta modalidad de cultivo, cabe destacar: la reducción de operaciones de cultivo, la facilidad y menor coste de la cosecha, y la posibilidad de conservación de los HCF en la biomasa cosechada sin mermas durante varios meses. Estas ventajas, compensan con creces el menor rendimiento por unidad de superficie que se obtiene con este sistema de cultivo frente al de cosecha de los tubérculos. Jerusalem artichoke (Helianthus tuberosus L.) (JA) is a crop with a high potential for the production of carbohydrates in the form of polyfructans, especially inulin, which are temporarily accumulated in the stems in the form of polysaccharides. Subsequently they are translocated to the tubers, where they are finally accumulated. In this work the potential of Jerusalem artichoke for fermentable carbohydrates from stems that are harvested at their peak of carbohydrates accumulation is assessed as compared to the traditional cultivation system that aims at the production of tubers harvested at the end of the growth cycle. Tubers are storage organs of polyfructans, namely fermentable carbohydrates. Studies addressed in this work were: i) Determination of the optimum period of time for stem harvesting as a function of clone precocity in a 12-clone field experiment; ii) Study of the potential of JA poly-annual crop regarding the annual yield of aerial biomass and fermentable carbohydrates (HCF) over the years; iii) Tests of storage of the aerial biomass, iv) Comparative analysis of the two JA cultivation systems for HCF production: the poly-annual system for aerial biomass harvesting versus the annual cultivation system for tubers and v) Estimation of the energy sustainability of the bioethanol production by using by-products of the production chain. In order to determine the best period of time for aerial biomass harvesting twelve JA clones of different precocity were tested in Madrid: four early clones (Huertos de Moya, C-17, Columbia and D-19) and eight late clones (Boniches, China, K-8 , Salmantina, Nahodka, C-13, INIA and Violeta de Rennes). Best time was between the phenological stages of floral buds (closed capitula) and blossom (opened capitula), period in which the peak of biomass production coincides with the peak of HCF accumulation in the stems. According to the results, the early clones should be harvested in July and the late ones in September, being the late clones more productive. The clone named INIA was the one that exhibited more steady yields in biomass over the 12 clones experimented. The average potential biomass production of this clone was 23.40 t dm/ha when harvested at the floral buds phenological stage; mean HCF content is 30.30%, representing 7.06 t HCF/ha yield. However, the highest HCF production was obtained for the clone Boniches, 7.61 t HCF/ha from a production of 22.81 t aerial biomass/ha. In the poly-annual cultivation system the number of stems per unit area increases over the years due to the increase in the number of tubers that are left under ground; this effect is particularly important after the 3rd year of the poly-annual crop and results in a decrease of the stems unit weight and a risk of lodging. Thinning of JA shoots in early spring, by means of an herbicide treatment based on glyphosate or by means of one pass with a rotary tiller, results in a decrease of the crop stem density and in higher crop yields. Tests of biomass storing showed that the method of keeping dried stems packed and stored under cover results in a good preservation of HCF for a period of six months at least. Assuming that the fermentation yield is 0.5 L ethanol per kg sugars and a HCF stem production of 7.06 t HCF/ha, the potential for bioethanol is estimated at 3530 L/ha. The use of bagasse -by-product of the process of HCF extraction from the JA stems- for thermal purposes would represent over 3 times the primary energy required for the industrial ethanol production process, assuming 11.91 t/ha bagasse and 3832.6 kcal/kg heating value. HCF production costs of 7.06 t HCF/ha yield from aerial biomass and HCF production costs of 12.11 t HCF/ha from tubers were estimated at 184.69 €/t HCF and 311.30 €/t HCF, respectively. It can be concluded that the production of HCF from JA stems, following a poly-annual cultivation system, can be feasible from a technical standpoint and lead to lower production costs as compared to the traditional annual cultivation system for the production of HCF from tubers. Among the technical advantages of the poly-annual cultivation system it is worth mentioning the reduction in crop operations, the ease and efficiency of harvesting operations and the possibility of HCF preservation without incurring in HCF losses during the storage period, which can last several months. These advantages might compensate the lower yield of HCF per unit area that is obtained in the poly-annual crop system, which aims at stems harvesting, versus the annual one, which involves tubers harvesting.

Cross-entropy optimization for scaling factors of a fuzzy controller: a see-and-avoid approach for unmanned aerial systems

The Cross-Entropy (CE) is an efficient method for the estimation of rare-event probabilities and combinatorial optimization. This work presents a novel approach of the CE for optimization of a Soft-Computing controller. A Fuzzy controller was designed to command an unmanned aerial system (UAS) for avoiding collision task. The only sensor used to accomplish this task was a forward camera. The CE is used to reach a near-optimal controller by modifying the scaling factors of the controller inputs. The optimization was realized using the ROS-Gazebo simulation system. In order to evaluate the optimization a big amount of tests were carried out with a real quadcopter.

Rapid prototyping framework for visual control of autonomous micro aerial vehicles

Rapid prototyping environments can speed up the research of visual control algorithms. We have designed and implemented a software framework for fast prototyping of visual control algorithms for Micro Aerial Vehicles (MAV). We have applied a combination of a proxy-based network communication architecture and a custom Application Programming Interface. This allows multiple experimental configurations, like drone swarms or distributed processing of a drone’s video stream. Currently, the framework supports a low-cost MAV: the Parrot AR.Drone. Real tests have been performed on this platform and the results show comparatively low figures of the extra communication delay introduced by the framework, while adding new functionalities and flexibility to the selected drone. This implementation is open-source and can be downloaded from www.vision4uav.com/?q=VC4MAV-FW

Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

Aerial video geo-registration using terrain models from dense and coherent stereo matching

In the context of aerial imagery, one of the first steps toward a coherent processing of the information contained in multiple images is geo-registration, which consists in assigning geographic 3D coordinates to the pixels of the image. This enables accurate alignment and geo-positioning of multiple images, detection of moving objects and fusion of data acquired from multiple sensors. To solve this problem there are different approaches that require, in addition to a precise characterization of the camera sensor, high resolution referenced images or terrain elevation models, which are usually not publicly available or out of date. Building upon the idea of developing technology that does not need a reference terrain elevation model, we propose a geo-registration technique that applies variational methods to obtain a dense and coherent surface elevation model that is used to replace the reference model. The surface elevation model is built by interpolation of scattered 3D points, which are obtained in a two-step process following a classical stereo pipeline: first, coherent disparity maps between image pairs of a video sequence are estimated and then image point correspondences are back-projected. The proposed variational method enforces continuity of the disparity map not only along epipolar lines (as done by previous geo-registration techniques) but also across them, in the full 2D image domain. In the experiments, aerial images from synthetic video sequences have been used to validate the proposed technique.

A vision-based strategy for autonomous aerial refueling tasks

Autonomous aerial refueling is a key enabling technology for both manned and unmanned aircraft where extended flight duration or range are required. The results presented within this paper offer one potential vision-based sensing solution, together with a unique test environment. A hierarchical visual tracking algorithm based on direct methods is proposed and developed for the purposes of tracking a drogue during the capture stage of autonomous aerial refueling, and of estimating its 3D position. Intended to be applied in real time to a video stream from a single monocular camera mounted on the receiver aircraft, the algorithm is shown to be highly robust, and capable of tracking large, rapid drogue motions within the frame of reference. The proposed strategy has been tested using a complex robotic testbed and with actual flight hardware consisting of a full size probe and drogue. Results show that the vision tracking algorithm can detect and track the drogue at real-time frame rates of more than thirty frames per second, obtaining a robust position estimation even with strong motions and multiple occlusions of the drogue.