Techniques for Area Discretization and Coverage in Aerial Photography for Precision Agriculture employing mini quad-rotors

Remote sensed imagery acquired with mini aerial vehicles, in conjunction with GIS technology enable a meticulous analysis from surveyed agricultural sites. This paper sums up the ongoing work in area discretization and coverage with mini quad-­?rotors applied to Precision Agriculture practices under the project RHEA.

Analysis of Thematic Classified Aerial Images Trough Multispectral and LIDAR Data

The application of thematic maps obtained through the classification of remote images needs the obtained products with an optimal accuracy. The registered images from the airplanes display a very satisfactory spatial resolution, but the classical methods of thematic classification not always give better results than when the registered data from satellite are used. In order to improve these results of classification, in this work, the LIDAR sensor data from first return (Light Detection And Ranging) registered simultaneously with the spectral sensor data from airborne are jointly used. The final results of the thematic classification of the scene object of study have been obtained, quantified and discussed with and without LIDAR data, after applying different methods: Maximum Likehood Classification, Support Vector Machine with four different functions kernel and Isodata clustering algorithm (ML, SVM-L, SVM-P, SVM-RBF, SVM-S, Isodata). The best results are obtained for SVM with Sigmoide kernel. These allow the correlation with others different physical parameters with great interest like Manning hydraulic coefficient, for their incorporation in a GIS and their application in hydraulic modeling.

Omnidirectional bearing-only see-and-avoid for small aerial robots:

Motivated by the growing interest in unmanned aerial system's applications in indoor and outdoor settings and the standardisation of visual sensors as vehicle payload. This work presents a collision avoidance approach based on omnidirectional cameras that does not require the estimation of range between two platforms to resolve a collision encounter. It will achieve a minimum separation between the two vehicles involved by maximising the view-angle given by the omnidirectional sensor. Only visual information is used to achieve avoidance under a bearing-only visual servoing approach. We provide theoretical problem formulation, as well as results from real flight using small quadrotors

3D Object following based on visual information for Unmanned Aerial Vehicles

This article presents a novel system and a control strategy for visual following of a 3D moving object by an Unmanned Aerial Vehicle UAV. The presented strategy is based only on the visual information given by an adaptive tracking method based on the color information, which jointly with the dynamics of a camera fixed to a rotary wind UAV are used to develop an Image-based visual servoing IBVS system. This system is focused on continuously following a 3D moving target object, maintaining it with a fixed distance and centered on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Precision Agriculture without borders: Practical issues and improvements in farmland coverage with aerial vehicles.

This work presents a solution for the aerial coverage of a field by using a fleet of aerial vehicles. The use of Unmanned Aerial Vehicles allows to obtain high resolution mosaics to be used in Precision Agriculture techniques. This report is focus on providing a solution for the full simultaneous coverage problem taking into account restrictions as the required spatial resolution and overlap while maintaining similar light conditions and safety operation of the drones. Results obtained from real field tests are finally reported

A waypoint-based mission planner for a farmland coverage with an aerial robot - a precision farming tool

Remote sensing (RS) with aerial robots is becoming more usual in every day time in Precision Agriculture (PA) practices, do to their advantages over conventional methods. Usually, available commercial platforms providing off-the-shelf waypoint navigation are adopted to perform visual surveys over crop fields, with the purpose to acquire specific image samples. The way in which a waypoint list is computed and dispatched to the aerial robot when mapping non empty agricultural workspaces has not been yet discussed. In this paper we propose an offline mission planner approach that computes an efficient coverage path subject to some constraints by decomposing the environment approximately into cells. Therefore, the aim of this work is contributing with a feasible waypoints-based tool to support PA practices

A Multiresolution Image Alignment Technique Based on Direct Methods for Pose Estimation of Aerial Vehicles

In this paper, we seek to expand the use of direct methods in real-time applications by proposing a vision-based strategy for pose estimation of aerial vehicles. The vast majority of approaches make use of features to estimate motion. Conversely, the strategy we propose is based on a MR (Multi-Resolution) implementation of an image registration technique (Inverse Compositional Image Alignment ICIA) using direct methods. An on-board camera in a downwards-looking configuration, and the assumption of planar scenes, are the bases of the algorithm. The motion between frames (rotation and translation) is recovered by decomposing the frame-to-frame homography obtained by the ICIA algorithm applied to a patch that covers around the 80% of the image. When the visual estimation is required (e.g. GPS drop-out), this motion is integrated with the previous known estimation of the vehicles' state, obtained from the on-board sensors (GPS/IMU), and the subsequent estimations are based only on the vision-based motion estimations. The proposed strategy is tested with real flight data in representative stages of a flight: cruise, landing, and take-off, being two of those stages considered critical: take-off and landing. The performance of the pose estimation strategy is analyzed by comparing it with the GPS/IMU estimations. Results show correlation between the visual estimation obtained with the MR-ICIA and the GPS/IMU data, that demonstrate that the visual estimation can be used to provide a good approximation of the vehicle's state when it is required (e.g. GPS drop-outs). In terms of performance, the proposed strategy is able to maintain an estimation of the vehicle's state for more than one minute, at real-time frame rates based, only on visual information.

Aerial Object Following Using Visual Fuzzy Servoing

This article presents a visual servoing system to follow a 3D moving object by a Micro Unmanned Aerial Vehicle (MUAV). The presented control strategy is based only on the visual information given by an adaptive tracking method based on the colour information. A visual fuzzy system has been developed for servoing the camera situated on a rotary wing MAUV, that also considers its own dynamics. This system is focused on continuously following of an aerial moving target object, maintaining it with a fixed safe distance and centred on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Aerial Remote Sensing in Agriculture: A Practical Approach to Area Coverage and Path Planning for Fleets of Mini Aerial Robots

In this paper, a system that allows applying precision agriculture techniques is described. The application is based on the deployment of a team of unmanned aerial vehicles that are able to take georeferenced pictures in order to create a full map by applying mosaicking procedures for postprocessing. The main contribution of this work is practical experimentation with an integrated tool. Contributions in different fields are also reported. Among them is a new one-phase automatic task partitioning manager, which is based on negotiation among the aerial vehicles, considering their state and capabilities. Once the individual tasks are assigned, an optimal path planning algorithm is in charge of determining the best path for each vehicle to follow. Also, a robust flight control based on the use of a control law that improves the maneuverability of the quadrotors has been designed. A set of field tests was performed in order to analyze all the capabilities of the system, from task negotiations to final performance. These experiments also allowed testing control robustness under different weather conditions.

Estudio de las formas de aplicación de la fotografía sobre los materiales pétreos para la arquitectura = Study of photography application methods over architectural stony materials

Desde que el Hombre era morador de las cavernas ha sido manifiesto su deseo innato por grabar y reproducir "instantáneas con las que perpetuarse o sobre las que mirarse ". La aparición y desarrollo de la fotografía como medio para poder captar y fijar "la imagen directa de la realidad circundante " pronto se convierte en un nuevo lenguaje estético y poético que permite al artista la interpretación y reflexión de lo observado. Se imprime a la imagen el carácter de la mirada del fotógrafo, estableciendo un diálogo conceptual con el juego de luces. La presente Tesis plantea la creación de una nueva piel de arquitectura mediante la impresión fotográfica sobre materiales pétreos. La búsqueda de la expresividad de los materiales como soporte de expresión artística implica un cambio de escala al trasladar la instantánea fotográfica a la arquitectura y la aplicación de un nuevo soporte al imprimir la fotografía sobre materiales arquitectónicos. Se justifica la elección del dispositivo láser CO2 como sistema de impresión fotográfica sobre los materiales pétreos arquitectónicos, como la técnica que permite la unión física de la imagen y el proyecto arquitectónico, generando un valor añadido a través del arte de la fotografía. Se justifica la elección de los materiales investigados, Silestone® Blanco Zeus y GRC® con TX Active® Aria, de forma que la investigación de esta nueva piel de arquitectura abarca tanto la envolvente del edificio como su volumen interior, permitiendo cerrar el círculo arquitectónico "in&out" y dota al proyecto arquitectónico de un valor añadido al introducir conceptos sostenibles de carácter estético y medioambiental. Se realiza una consulta a las empresas del sector arquitectónico relacionadas directamente con la producción y distribución de los materiales Silestone® y GRC®, así como a las empresas especializadas en sistemas de impresión fotográfica sobre materiales, acerca del estado del arte. Se recorre la Historia de la fotografía desde sus orígenes hasta el desarrollo de la era digital y se analiza su condición artística. Se recopilan los sistemas de impresión fotográfica que han evolucionado en paralelo con los dispositivos de captura de la instantánea fotográfica y se describe en profundidad el sistema de impresión fotográfica mediante dispositivo láser CO2. Se describen los procesos de fabricación, las características técnicas, cualidades y aplicaciones de los materiales pétreos arquitectónicos Silestone® Blanco Zeus y GRC® con TX Active® Aria. Se explica la técnica utilizada para la captación de la imagen fotográfica, su justificación artística y su proceso de impresión mediante dispositivo láser CO2 bajo diferentes parámetros sobre muestras de los materiales arquitectónicos investigados. Se comprueba la viabilidad de desarrollo de la nueva piel de arquitectura sobre Silestone® Blanco Zeus y GRC® con TX Active® Aria sometiendo a las piezas impresas bajo diferentes parámetros a tres ensayos de laboratorio. En cada uno de ellos se concreta el objetivo y procedimiento del ensayo, la enumeración de las muestras ensayadas y los parámetros bajo los que han sido impresas, el análisis de los resultados del ensayo y las conclusiones del ensayo. Ensayo de amplitud térmica. Se determina el grado de afectación de las imágenes impresas bajo la acción de contrastes térmicos. Series de muestras de Silestone® Blanco Zeus y GRC® con TX Active® Aria impresas con láser CO2 se someten a ciclos de contraste frío-calor de 12 horas de duración para una amplitud térmica total de 102°C. Se realiza una toma sistemática de fotografías microscópicas con lupa de aumento de cada pieza antes y después de los ciclos frío-calor y la observación de las transformaciones que experimentan los materiales bajo la acción del láser CO2. Ensayo de exposición a la acción de la radiación ultravioleta (UV). Se determina el grado de afectación de las imágenes impresas al activar la capacidad autolimpiante de partículas orgánicas. Una serie de muestras de GRC® con TX Active® Aria impresa con láser CO2 se someten a ciclos de exposición de radiación ultravioleta de 26 horas de duración. Se somete la serie a un procedimiento de activación del aditivo TX Active®. Se simula la contaminación orgánica mediante la aplicación controlada de Rodamina B, tinte orgánico, y se simula la radiación UV mediante el empleo de una bombilla de emisión de rayos ultravioleta. Se realiza una toma sistemática de fotografías macroscópicas de la serie de muestras ensayadas: antes de aplicación de la Rodamina B, momento 00:00h, momento 04:00h y momento 26:00h del ensayo. Se procede a la descarga y análisis del histograma de las fotografías como registro de la actividad fotocatalítica. Ensayo de la capacidad autodescontaminante del GRC® con TX Active® impreso con láser CO2. Se comprueba si la capacidad autodescontaminante del GRC® con TX Active® se ve alterada como consecuencia de la impresión de la imagen fotográfica impresa con láser CO2. Serie de muestras de GRC® con TX Active® Aria impresa con láser CO2 se someten a test de capacidad autodescontaminante: atmósfera controlada y contaminada con óxidos de nitrógeno en los que se coloca cada pieza ensayada bajo la acción de una lámpara de emisión de radiación ultravioleta (UV). Se registra la actividad fotocatalítica en base a la variación de concentración de óxido de nitrógeno. Se recopila el análisis e interpretación de los resultados de los ensayos de laboratorio y se elaboran las conclusiones generales de la investigación. Se sintetizan las futuras líneas de investigación que, a partir de las investigaciones realizadas y de sus conclusiones generales, podrían desarrollarse en el ámbito de la impresión fotográfica sobre materiales arquitectónicos. Se describe el rendimiento tecnológico y artístico generado por las investigaciones previas que han dado origen y desarrollo a la Tesis Doctoral. ABSTRACT Since ancient time, humanity has been driven by an innate wish to reproduce and engrave "snapshots that could help to perpetúate or to look at one self". Photography's birth and its development as a mean to capture and fix "the direct image of the surrounding reality" quickly becomes a new aesthetical and poetical language allowing the artist to interpret and think over what has been observed. The photographer's eye is imprinted onto the image, and so the conceptual dialogue between the artist and the light beams begins. The current thesis suggests the creation of a new architectural skin through photography imprinting over stony materials. The search for material's expressiveness as a medium of artistic expression involves a change of scale as it transfers photographic snapshot into architecture and the use of a new photographic printing support over architectural materials. CO2 laser is the chosen printing system for this technique as it allows the physical union of the image and the architectonic project, generating an added value through the art of photography. The researched materials selected were Silestone®, Blanco Zeus and GRC® with TX Active® Aria. This new architectural skin contains the building surrounding as well as its interior volume, closing the architectonic "in & out" circle and adding a value to the project by introducing aesthetical and environmental sustainable concepts. Architecture companies related to the production and distribution of materials like Silestone® and GRC®, as well as companies specialized in photography printing over materials were consulted to obtain a State of the Art. A thorough analysis of photography's History from its origins to the digital era development was made and its artistic condition was studied in this thesis. In this study the author also makes a compilation of several photographic printing systems that evolved together with photographic snapshot devices. The CO2 laser-based photographic printing system is also described in depth. Regarding stony materials of architecture like Silestone®, Blanco Zeus and GRC® with TX Active® Aria, the present study also describes their manufacture processes as well as technical features, quality and application. There is also an explanation about the technique to capture the photographic image, its artistic justification and its CO2 laser-based printing system over the researched materials under different parameters. We also tested the feasibility of this new architectural skin over Silestone® Blanco Zeus and GRC® with TX Active® Aria. The pieces were tested under different parameters in three laboratory trials. Each trial comprises of an explanation of its objective and its process, the samples were numbered and the printing parameters were specified. Finally, with the analysis of the results some conclusions were drawn. In the thermal amplitude trial we tried to determine how printed images were affected as a result of the action of thermal contrasts. Series of samples of Silestone® Blanco Zeus and GRC® with TX Active® Aria printed with CO2 laser were subjected to several 12h warm-cold cycles for thermal total amplitude of 102oc. Each sample was captured systematically with microscopic enhanced lenses before and after cold-warm cycles. The changes experienced by these materials under the effect of CO2 laser were observed and recorded. Trial regarding the Ultraviolet Radiation (UR) effect on images. We determined to which extent printed images were affected once the self-cleaning organic particles were activated. This time GRC® with TX Active® Aria samples printed with CO2 laser were exposed to a 26h UR cycle. The samples were subjected to the activation of TX Active® additive. Through the controlled application of Rodamine B and organic dye we were able to simulate the organic contamination process. UR was simulated using an ultraviolet beam emission bulb. A systematic capture of macroscopic pictures of the tested sample series was performed at different time points: before Rodamine B application, at moment 00:00h, moment 04:00h and moment 26:00h of the trial. Picture's histogram was downloaded and analyzed as a log of photocatalytic activity. Trial regarding the self-decontaminating ability of GRC® with TX Active® printed with CO2 laser. We tested if this self-decontaminating ability is altered as a result of CO2 laser printed image. GRC® with TX Active® Aria samples printed with CO2 laser, were subject to self-decontaminating ability tests with controlled and nitrogen oxide contaminated atmosphere. Each piece was put under the action of an UR emission lamp. Photocatalytic activity was recorded according to the variation in nitrogen oxide concentration. The results of the trial and their interpretation as well as the general conclusions of the research are also compiled in the present study. Study conclusions enable to draw future research lines of potential applications of photographic printing over architecture materials. Previous research generated an artistic and technological outcome that led to the development of this doctoral thesis.

Cross-entropy optimization for scaling factors of a fuzzy controller: a see-and-avoid approach for unmanned aerial systems

The Cross-Entropy (CE) is an efficient method for the estimation of rare-event probabilities and combinatorial optimization. This work presents a novel approach of the CE for optimization of a Soft-Computing controller. A Fuzzy controller was designed to command an unmanned aerial system (UAS) for avoiding collision task. The only sensor used to accomplish this task was a forward camera. The CE is used to reach a near-optimal controller by modifying the scaling factors of the controller inputs. The optimization was realized using the ROS-Gazebo simulation system. In order to evaluate the optimization a big amount of tests were carried out with a real quadcopter.

Rapid prototyping framework for visual control of autonomous micro aerial vehicles

Rapid prototyping environments can speed up the research of visual control algorithms. We have designed and implemented a software framework for fast prototyping of visual control algorithms for Micro Aerial Vehicles (MAV). We have applied a combination of a proxy-based network communication architecture and a custom Application Programming Interface. This allows multiple experimental configurations, like drone swarms or distributed processing of a drone’s video stream. Currently, the framework supports a low-cost MAV: the Parrot AR.Drone. Real tests have been performed on this platform and the results show comparatively low figures of the extra communication delay introduced by the framework, while adding new functionalities and flexibility to the selected drone. This implementation is open-source and can be downloaded from www.vision4uav.com/?q=VC4MAV-FW

Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

Aerial video geo-registration using terrain models from dense and coherent stereo matching

In the context of aerial imagery, one of the first steps toward a coherent processing of the information contained in multiple images is geo-registration, which consists in assigning geographic 3D coordinates to the pixels of the image. This enables accurate alignment and geo-positioning of multiple images, detection of moving objects and fusion of data acquired from multiple sensors. To solve this problem there are different approaches that require, in addition to a precise characterization of the camera sensor, high resolution referenced images or terrain elevation models, which are usually not publicly available or out of date. Building upon the idea of developing technology that does not need a reference terrain elevation model, we propose a geo-registration technique that applies variational methods to obtain a dense and coherent surface elevation model that is used to replace the reference model. The surface elevation model is built by interpolation of scattered 3D points, which are obtained in a two-step process following a classical stereo pipeline: first, coherent disparity maps between image pairs of a video sequence are estimated and then image point correspondences are back-projected. The proposed variational method enforces continuity of the disparity map not only along epipolar lines (as done by previous geo-registration techniques) but also across them, in the full 2D image domain. In the experiments, aerial images from synthetic video sequences have been used to validate the proposed technique.

A vision-based strategy for autonomous aerial refueling tasks

Autonomous aerial refueling is a key enabling technology for both manned and unmanned aircraft where extended flight duration or range are required. The results presented within this paper offer one potential vision-based sensing solution, together with a unique test environment. A hierarchical visual tracking algorithm based on direct methods is proposed and developed for the purposes of tracking a drogue during the capture stage of autonomous aerial refueling, and of estimating its 3D position. Intended to be applied in real time to a video stream from a single monocular camera mounted on the receiver aircraft, the algorithm is shown to be highly robust, and capable of tracking large, rapid drogue motions within the frame of reference. The proposed strategy has been tested using a complex robotic testbed and with actual flight hardware consisting of a full size probe and drogue. Results show that the vision tracking algorithm can detect and track the drogue at real-time frame rates of more than thirty frames per second, obtaining a robust position estimation even with strong motions and multiple occlusions of the drogue.