Calibración geométrica de cámaras no métricas. Estudio de metodologías y modelos matemáticos de distorsión

La Fotogrametría, como ciencia y técnica de obtención de información tridimensional del espacio objeto a partir de imágenes bidimensionales, requiere de medidas de precisión y en ese contexto, la calibración geométrica de cámaras ocupa un lugar importante. El conocimiento de la geometría interna de la cámara es fundamental para lograr mayor precisión en las medidas realizadas. En Fotogrametría Aérea se utilizan cámaras métricas (fabricadas exclusivamente para aplicaciones cartográficas), que incluyen objetivos fotográficos con sistemas de lentes complejos y de alta calidad. Pero en Fotogrametría de Objeto Cercano se está trabajando cada vez con más asiduidad con cámaras no métricas, con ópticas de peor calidad que exigen una calibración geométrica antes o después de cada trabajo. El proceso de calibración encierra tres conceptos fundamentales: modelo de cámara, modelo de distorsión y método de calibración. El modelo de cámara es un modelo matemático que aproxima la transformación proyectiva original a la realidad física de las lentes. Ese modelo matemático incluye una serie de parámetros entre los que se encuentran los correspondientes al modelo de distorsión, que se encarga de corregir los errores sistemáticos de la imagen. Finalmente, el método de calibración propone el método de estimación de los parámetros del modelo matemático y la técnica de optimización a emplear. En esta Tesis se propone la utilización de un patrón de calibración bidimensional que se desplaza en la dirección del eje óptico de la cámara, ofreciendo así tridimensionalidad a la escena fotografiada. El patrón incluye un número elevado de marcas, lo que permite realizar ensayos con distintas configuraciones geométricas. Tomando el modelo de proyección perspectiva (o pinhole) como modelo de cámara, se realizan ensayos con tres modelos de distorsión diferentes, el clásico de distorsión radial y tangencial propuesto por D.C. Brown, una aproximación por polinomios de Legendre y una interpolación bicúbica. De la combinación de diferentes configuraciones geométricas y del modelo de distorsión más adecuado, se llega al establecimiento de una metodología de calibración óptima. Para ayudar a la elección se realiza un estudio de las precisiones obtenidas en los distintos ensayos y un control estereoscópico de un panel test construido al efecto. ABSTRACT Photogrammetry, as science and technique for obtaining three-dimensional information of the space object from two-dimensional images, requires measurements of precision and in that context, the geometric camera calibration occupies an important place. The knowledge of the internal geometry of the camera is fundamental to achieve greater precision in measurements made. Metric cameras (manufactured exclusively for cartographic applications), including photographic lenses with complex lenses and high quality systems are used in Aerial Photogrammetry. But in Close Range Photogrammetry is working increasingly more frequently with non-metric cameras, worst quality optical components which require a geometric calibration before or after each job. The calibration process contains three fundamental concepts: camera model, distortion model and method of calibration. The camera model is a mathematical model that approximates the original projective transformation to the physical reality of the lenses. The mathematical model includes a series of parameters which include the correspondents to the model of distortion, which is in charge of correcting the systematic errors of the image. Finally, the calibration method proposes the method of estimation of the parameters of the mathematical modeling and optimization technique to employ. This Thesis is proposing the use of a pattern of two dimensional calibration that moves in the direction of the optical axis of the camera, thus offering three-dimensionality to the photographed scene. The pattern includes a large number of marks, which allows testing with different geometric configurations. Taking the projection model perspective (or pinhole) as a model of camera, tests are performed with three different models of distortion, the classical of distortion radial and tangential proposed by D.C. Brown, an approximation by Legendre polynomials and bicubic interpolation. From the combination of different geometric configurations and the most suitable distortion model, brings the establishment of a methodology for optimal calibration. To help the election, a study of the information obtained in the various tests and a purpose built test panel stereoscopic control is performed.

Analysis of Thematic Classified Aerial Images Trough Multispectral and LIDAR Data

The application of thematic maps obtained through the classification of remote images needs the obtained products with an optimal accuracy. The registered images from the airplanes display a very satisfactory spatial resolution, but the classical methods of thematic classification not always give better results than when the registered data from satellite are used. In order to improve these results of classification, in this work, the LIDAR sensor data from first return (Light Detection And Ranging) registered simultaneously with the spectral sensor data from airborne are jointly used. The final results of the thematic classification of the scene object of study have been obtained, quantified and discussed with and without LIDAR data, after applying different methods: Maximum Likehood Classification, Support Vector Machine with four different functions kernel and Isodata clustering algorithm (ML, SVM-L, SVM-P, SVM-RBF, SVM-S, Isodata). The best results are obtained for SVM with Sigmoide kernel. These allow the correlation with others different physical parameters with great interest like Manning hydraulic coefficient, for their incorporation in a GIS and their application in hydraulic modeling.

Omnidirectional bearing-only see-and-avoid for small aerial robots:

Motivated by the growing interest in unmanned aerial system's applications in indoor and outdoor settings and the standardisation of visual sensors as vehicle payload. This work presents a collision avoidance approach based on omnidirectional cameras that does not require the estimation of range between two platforms to resolve a collision encounter. It will achieve a minimum separation between the two vehicles involved by maximising the view-angle given by the omnidirectional sensor. Only visual information is used to achieve avoidance under a bearing-only visual servoing approach. We provide theoretical problem formulation, as well as results from real flight using small quadrotors

3D Object following based on visual information for Unmanned Aerial Vehicles

This article presents a novel system and a control strategy for visual following of a 3D moving object by an Unmanned Aerial Vehicle UAV. The presented strategy is based only on the visual information given by an adaptive tracking method based on the color information, which jointly with the dynamics of a camera fixed to a rotary wind UAV are used to develop an Image-based visual servoing IBVS system. This system is focused on continuously following a 3D moving target object, maintaining it with a fixed distance and centered on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Precision Agriculture without borders: Practical issues and improvements in farmland coverage with aerial vehicles.

This work presents a solution for the aerial coverage of a field by using a fleet of aerial vehicles. The use of Unmanned Aerial Vehicles allows to obtain high resolution mosaics to be used in Precision Agriculture techniques. This report is focus on providing a solution for the full simultaneous coverage problem taking into account restrictions as the required spatial resolution and overlap while maintaining similar light conditions and safety operation of the drones. Results obtained from real field tests are finally reported

A waypoint-based mission planner for a farmland coverage with an aerial robot - a precision farming tool

Remote sensing (RS) with aerial robots is becoming more usual in every day time in Precision Agriculture (PA) practices, do to their advantages over conventional methods. Usually, available commercial platforms providing off-the-shelf waypoint navigation are adopted to perform visual surveys over crop fields, with the purpose to acquire specific image samples. The way in which a waypoint list is computed and dispatched to the aerial robot when mapping non empty agricultural workspaces has not been yet discussed. In this paper we propose an offline mission planner approach that computes an efficient coverage path subject to some constraints by decomposing the environment approximately into cells. Therefore, the aim of this work is contributing with a feasible waypoints-based tool to support PA practices

Techniques for Area Discretization and Coverage in Aerial Photography for Precision Agriculture employing mini quad-rotors

Remote sensed imagery acquired with mini aerial vehicles, in conjunction with GIS technology enable a meticulous analysis from surveyed agricultural sites. This paper sums up the ongoing work in area discretization and coverage with mini quad-­?rotors applied to Precision Agriculture practices under the project RHEA.

A Multiresolution Image Alignment Technique Based on Direct Methods for Pose Estimation of Aerial Vehicles

In this paper, we seek to expand the use of direct methods in real-time applications by proposing a vision-based strategy for pose estimation of aerial vehicles. The vast majority of approaches make use of features to estimate motion. Conversely, the strategy we propose is based on a MR (Multi-Resolution) implementation of an image registration technique (Inverse Compositional Image Alignment ICIA) using direct methods. An on-board camera in a downwards-looking configuration, and the assumption of planar scenes, are the bases of the algorithm. The motion between frames (rotation and translation) is recovered by decomposing the frame-to-frame homography obtained by the ICIA algorithm applied to a patch that covers around the 80% of the image. When the visual estimation is required (e.g. GPS drop-out), this motion is integrated with the previous known estimation of the vehicles' state, obtained from the on-board sensors (GPS/IMU), and the subsequent estimations are based only on the vision-based motion estimations. The proposed strategy is tested with real flight data in representative stages of a flight: cruise, landing, and take-off, being two of those stages considered critical: take-off and landing. The performance of the pose estimation strategy is analyzed by comparing it with the GPS/IMU estimations. Results show correlation between the visual estimation obtained with the MR-ICIA and the GPS/IMU data, that demonstrate that the visual estimation can be used to provide a good approximation of the vehicle's state when it is required (e.g. GPS drop-outs). In terms of performance, the proposed strategy is able to maintain an estimation of the vehicle's state for more than one minute, at real-time frame rates based, only on visual information.

Aerial Object Following Using Visual Fuzzy Servoing

This article presents a visual servoing system to follow a 3D moving object by a Micro Unmanned Aerial Vehicle (MUAV). The presented control strategy is based only on the visual information given by an adaptive tracking method based on the colour information. A visual fuzzy system has been developed for servoing the camera situated on a rotary wing MAUV, that also considers its own dynamics. This system is focused on continuously following of an aerial moving target object, maintaining it with a fixed safe distance and centred on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Aerial Remote Sensing in Agriculture: A Practical Approach to Area Coverage and Path Planning for Fleets of Mini Aerial Robots

In this paper, a system that allows applying precision agriculture techniques is described. The application is based on the deployment of a team of unmanned aerial vehicles that are able to take georeferenced pictures in order to create a full map by applying mosaicking procedures for postprocessing. The main contribution of this work is practical experimentation with an integrated tool. Contributions in different fields are also reported. Among them is a new one-phase automatic task partitioning manager, which is based on negotiation among the aerial vehicles, considering their state and capabilities. Once the individual tasks are assigned, an optimal path planning algorithm is in charge of determining the best path for each vehicle to follow. Also, a robust flight control based on the use of a control law that improves the maneuverability of the quadrotors has been designed. A set of field tests was performed in order to analyze all the capabilities of the system, from task negotiations to final performance. These experiments also allowed testing control robustness under different weather conditions.

Cross-entropy optimization for scaling factors of a fuzzy controller: a see-and-avoid approach for unmanned aerial systems

The Cross-Entropy (CE) is an efficient method for the estimation of rare-event probabilities and combinatorial optimization. This work presents a novel approach of the CE for optimization of a Soft-Computing controller. A Fuzzy controller was designed to command an unmanned aerial system (UAS) for avoiding collision task. The only sensor used to accomplish this task was a forward camera. The CE is used to reach a near-optimal controller by modifying the scaling factors of the controller inputs. The optimization was realized using the ROS-Gazebo simulation system. In order to evaluate the optimization a big amount of tests were carried out with a real quadcopter.

Rapid prototyping framework for visual control of autonomous micro aerial vehicles

Rapid prototyping environments can speed up the research of visual control algorithms. We have designed and implemented a software framework for fast prototyping of visual control algorithms for Micro Aerial Vehicles (MAV). We have applied a combination of a proxy-based network communication architecture and a custom Application Programming Interface. This allows multiple experimental configurations, like drone swarms or distributed processing of a drone’s video stream. Currently, the framework supports a low-cost MAV: the Parrot AR.Drone. Real tests have been performed on this platform and the results show comparatively low figures of the extra communication delay introduced by the framework, while adding new functionalities and flexibility to the selected drone. This implementation is open-source and can be downloaded from www.vision4uav.com/?q=VC4MAV-FW

Exploring Cultural Heritage Resources in a 3D Collaborative Environment

Cultural heritage is a complex and diverse concept, which brings together a wide domain of information. Resources linked to a cultural heritage site may consist of physical artefacts, books, works of art, pictures, historical maps, aerial photographs, archaeological surveys and 3D models. Moreover, all these resources are listed and described by a set of a variety of metadata specifications that allow their online search and consultation on the most basic characteristics of them. Some examples include Norma ISO 19115, Dublin Core, AAT, CDWA, CCO, DACS, MARC, MoReq, MODS, MuseumDat, TGN, SPECTRUM, VRA Core and Z39.50. Gateways are in place to fit in these metadata standards into those used in a SDI (ISO 19115 or INSPIRE), but substantial work still remains to be done for the complete incorporation of cultural heritage information. Therefore, the aim of this paper is to demonstrate how the complexity of cultural heritage resources can be dealt with by a visual exploration of their metadata within a 3D collaborative environment. The 3D collaborative environments are promising tools that represent the new frontier of our capacity of learning, understanding, communicating and transmitting culture.

Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

Aerial video geo-registration using terrain models from dense and coherent stereo matching

In the context of aerial imagery, one of the first steps toward a coherent processing of the information contained in multiple images is geo-registration, which consists in assigning geographic 3D coordinates to the pixels of the image. This enables accurate alignment and geo-positioning of multiple images, detection of moving objects and fusion of data acquired from multiple sensors. To solve this problem there are different approaches that require, in addition to a precise characterization of the camera sensor, high resolution referenced images or terrain elevation models, which are usually not publicly available or out of date. Building upon the idea of developing technology that does not need a reference terrain elevation model, we propose a geo-registration technique that applies variational methods to obtain a dense and coherent surface elevation model that is used to replace the reference model. The surface elevation model is built by interpolation of scattered 3D points, which are obtained in a two-step process following a classical stereo pipeline: first, coherent disparity maps between image pairs of a video sequence are estimated and then image point correspondences are back-projected. The proposed variational method enforces continuity of the disparity map not only along epipolar lines (as done by previous geo-registration techniques) but also across them, in the full 2D image domain. In the experiments, aerial images from synthetic video sequences have been used to validate the proposed technique.