On precise three-dimensional environment modeling via UAV-based photogrammetric systems

Abstract : Images acquired from unmanned aerial vehicles (UAVs) can provide data with unprecedented spatial and temporal resolution for three-dimensional (3D) modeling. Solutions developed for this purpose are mainly operating based on photogrammetry concepts, namely UAV-Photogrammetry Systems (UAV-PS). Such systems are used in applications where both geospatial and visual information of the environment is required. These applications include, but are not limited to, natural resource management such as precision agriculture, military and police-related services such as traffic-law enforcement, precision engineering such as infrastructure inspection, and health services such as epidemic emergency management. UAV-photogrammetry systems can be differentiated based on their spatial characteristics in terms of accuracy and resolution. That is some applications, such as precision engineering, require high-resolution and high-accuracy information of the environment (e.g. 3D modeling with less than one centimeter accuracy and resolution). In other applications, lower levels of accuracy might be sufficient, (e.g. wildlife management needing few decimeters of resolution). However, even in those applications, the specific characteristics of UAV-PSs should be well considered in the steps of both system development and application in order to yield satisfying results. In this regard, this thesis presents a comprehensive review of the applications of unmanned aerial imagery, where the objective was to determine the challenges that remote-sensing applications of UAV systems currently face. This review also allowed recognizing the specific characteristics and requirements of UAV-PSs, which are mostly ignored or not thoroughly assessed in recent studies. Accordingly, the focus of the first part of this thesis is on exploring the methodological and experimental aspects of implementing a UAV-PS. The developed system was extensively evaluated for precise modeling of an open-pit gravel mine and performing volumetric-change measurements. This application was selected for two main reasons. Firstly, this case study provided a challenging environment for 3D modeling, in terms of scale changes, terrain relief variations as well as structure and texture diversities. Secondly, open-pit-mine monitoring demands high levels of accuracy, which justifies our efforts to improve the developed UAV-PS to its maximum capacities. The hardware of the system consisted of an electric-powered helicopter, a high-resolution digital camera, and an inertial navigation system. The software of the system included the in-house programs specifically designed for camera calibration, platform calibration, system integration, onboard data acquisition, flight planning and ground control point (GCP) detection. The detailed features of the system are discussed in the thesis, and solutions are proposed in order to enhance the system and its photogrammetric outputs. The accuracy of the results was evaluated under various mapping conditions, including direct georeferencing and indirect georeferencing with different numbers, distributions and types of ground control points. Additionally, the effects of imaging configuration and network stability on modeling accuracy were assessed. The second part of this thesis concentrates on improving the techniques of sparse and dense reconstruction. The proposed solutions are alternatives to traditional aerial photogrammetry techniques, properly adapted to specific characteristics of unmanned, low-altitude imagery. Firstly, a method was developed for robust sparse matching and epipolar-geometry estimation. The main achievement of this method was its capacity to handle a very high percentage of outliers (errors among corresponding points) with remarkable computational efficiency (compared to the state-of-the-art techniques). Secondly, a block bundle adjustment (BBA) strategy was proposed based on the integration of intrinsic camera calibration parameters as pseudo-observations to Gauss-Helmert model. The principal advantage of this strategy was controlling the adverse effect of unstable imaging networks and noisy image observations on the accuracy of self-calibration. The sparse implementation of this strategy was also performed, which allowed its application to data sets containing a lot of tie points. Finally, the concepts of intrinsic curves were revisited for dense stereo matching. The proposed technique could achieve a high level of accuracy and efficiency by searching only through a small fraction of the whole disparity search space as well as internally handling occlusions and matching ambiguities. These photogrammetric solutions were extensively tested using synthetic data, close-range images and the images acquired from the gravel-pit mine. Achieving absolute 3D mapping accuracy of 11±7 mm illustrated the success of this system for high-precision modeling of the environment.

Development of a Real-Time Hierarchical 3D Path Planning Algorithm for Unmanned Aerial Vehicles

Unmanned aerial vehicles (UAVs) frequently operate in partially or entirely unknown environments. As the vehicle traverses the environment and detects new obstacles, rapid path replanning is essential to avoid collisions. This thesis presents a new algorithm called Hierarchical D* Lite (HD*), which combines the incremental algorithm D* Lite with a novel hierarchical path planning approach to replan paths sufficiently fast for real-time operation. Unlike current hierarchical planning algorithms, HD* does not require map corrections before planning a new path. Directional cost scale factors, path smoothing, and Catmull-Rom splines are used to ensure the resulting paths are feasible. HD* sacrifices optimality for real-time performance. Its computation time and path quality are dependent on the map size, obstacle density, sensor range, and any restrictions on planning time. For the most complex scenarios tested, HD* found paths within 10% of optimal in under 35 milliseconds.

RISK-BASED MULTIOBJECTIVE PATH PLANNING AND DESIGN OPTIMIZATION FOR UNMANNED AERIAL VEHICLES

Safe operation of unmanned aerial vehicles (UAVs) over populated areas requires reducing the risk posed by a UAV if it crashed during its operation. We considered several types of UAV risk-based path planning problems and developed techniques for estimating the risk to third parties on the ground. The path planning problem requires making trade-offs between risk and flight time. Four optimization approaches for solving the problem were tested; a network-based approach that used a greedy algorithm to improve the original solution generated the best solutions with the least computational effort. Additionally, an approach for solving a combined design and path planning problems was developed and tested. This approach was extended to solve robust risk-based path planning problem in which uncertainty about wind conditions would affect the risk posed by a UAV.

Unmanned Aerial Vehicle: tecnologie e prospettive future

Partendo dalla definizione di UAV e UAS, arrivando a quella di drone, nella tesi saranno definiti i termini precedenti, ossia un sistema aereo senza pilota a bordo, la nascita del termine drone e le tendenze attuali. Dopo una precisa classificazione nelle quattro categorie principali (droni per hobbisti, commerciali e militari di me- dia grandezza, militari specifici di grandi dimensioni e stealth da combattimento) saranno descritti gli ambiti di utilizzo: da un lato quello militare e della sicurezza, dall’altro quello civile e scientifico. I capitoli centrali della tesi saranno il cuore dell’opera: l’architettura dell’UAV sarà descritta analizzando la totalità delle sue componenti, sia hardware che software. Verranno, quindi, analizzati i problemi relativi alla sicurezza, focalizzandosi sull’hacking di un UAV, illustrandone le varie tecniche e contromisure (tra cui anche come nascondersi da un drone). Il lavoro della tesi prosegue nei capitoli successivi con un’attenta trattazione della normativa vigente e dell’etica dei droni (nonché del diritto ad uccidere con tali sistemi). Il capitolo relativo alla tecnologia stealth sarà importante per capire le modalità di occultamento, le tendenze attuali e i possibili sviluppi futuri degli UAV militari da combattimento. Il capitolo finale sugli sviluppi futuri esporrà le migliorie tecnologiche e gli obiettivi degli UAV negli anni a venire, insieme ad eventuali utilizzi sia militari che civili. La ricerca sarà orientata verso sistemi miniaturizzati, multiple UAV e swarming.

Evolutionary optimisation methods with uncertainty for modern multidisciplinary design in aeronautical engineering

One of the new challenges in aeronautics is combining and accounting for multiple disciplines while considering uncertainties or variability in the design parameters or operating conditions. This paper describes a methodology for robust multidisciplinary design optimisation when there is uncertainty in the operating conditions. The methodology, which is based on canonical evolution algorithms, is enhanced by its coupling with an uncertainty analysis technique. The paper illustrates the use of this methodology on two practical test cases related to Unmanned Aerial Systems (UAS). These are the ideal candidates due to the multi-physics involved and the variability of missions to be performed. Results obtained from the optimisation show that the method is effective to find useful Pareto non-dominated solutions and demonstrate the use of robust design techniques.

UAS mission path planning system (MPPS) using hybrid-game coupled to multi-objective optimiser

This paper presents the application of advanced optimization techniques to unmanned aerial system mission path planning system (MPPS) using multi-objective evolutionary algorithms (MOEAs). Two types of multi-objective optimizers are compared; the MOEA nondominated sorting genetic algorithm II and a hybrid-game strategy are implemented to produce a set of optimal collision-free trajectories in a three-dimensional environment. The resulting trajectories on a three-dimensional terrain are collision-free and are represented by using Bézier spline curves from start position to target and then target to start position or different positions with altitude constraints. The efficiency of the two optimization methods is compared in terms of computational cost and design quality. Numerical results show the benefits of adding a hybrid-game strategy to a MOEA and for a MPPS.

Uncertainty based MDO of UAS using HAPMOEA

CFD has been successfully used in the optimisation of aerodynamic surfaces using a given set of parameters such as Mach numbers and angle of attack. While carrying out a multidisciplinary design optimisation one deals with situations where the parameters have some uncertain attached. Any optimisation carried out for fixed values of input parameters gives a design which may be totally unacceptable under off-design conditions. The challenge is to develop a robust design procedure which takes into account the fluctuations in the input parameters. In this work, we attempt this using a modified Taguchi approach. This is incorporated into an evolutionary algorithm with many features developed in house. The method is tested for an UCAV design which simultaneously handles aerodynamics, electromagnetics and maneuverability. Results demonstrate that the method has considerable potential.

Embedding human expert cognition into autonomous UAS trajectory planning

This paper presents a new approach for the inclusion of human expert cognition into autonomous trajectory planning for unmanned aerial systems (UASs) operating in low-altitude environments. During typical UAS operations, multiple objectives may exist; therefore, the use of multicriteria decision aid techniques can potentially allow for convergence to trajectory solutions which better reflect overall mission requirements. In that context, additive multiattribute value theory has been applied to optimize trajectories with respect to multiple objectives. A graphical user interface was developed to allow for knowledge capture from a human decision maker (HDM) through simulated decision scenarios. The expert decision data gathered are converted into value functions and corresponding criteria weightings using utility additive theory. The inclusion of preferences elicited from HDM data within an automated decision system allows for the generation of trajectories which more closely represent the candidate HDM decision preferences. This approach has been demonstrated in this paper through simulation using a fixed-wing UAS operating in low-altitude environments.

Multidisciplinary design and flight testing of a remote gas/particle airborne sensor system

The main objective of this paper is to describe the development of a remote sensing airborne air sampling system for Unmanned Aerial Systems (UAS) and provide the capability for the detection of particle and gas concentrations in real time over remote locations. The design of the air sampling methodology started by defining system architecture, and then by selecting and integrating each subsystem. A multifunctional air sampling instrument, with capability for simultaneous measurement of particle and gas concentrations was modified and integrated with ARCAA’s Flamingo UAS platform and communications protocols. As result of the integration process, a system capable of both real time geo-location monitoring and indexed-link sampling was obtained. Wind tunnel tests were conducted in order to evaluate the performance of the air sampling instrument in controlled nonstationary conditions at the typical operational velocities of the UAS platform. Once the remote fully operative air sampling system was obtained, the problem of mission design was analyzed through the simulation of different scenarios. Furthermore, flight tests of the complete air sampling system were then conducted to check the dynamic characteristics of the UAS with the air sampling system and to prove its capability to perform an air sampling mission following a specific flight path.

A multi-layered approach for site detection in UAS emergency landing scenarios using geometry-based image segmentation

This paper presents an alternative approach to image segmentation by using the spatial distribution of edge pixels as opposed to pixel intensities. The segmentation is achieved by a multi-layered approach and is intended to find suitable landing areas for an aircraft emergency landing. We combine standard techniques (edge detectors) with novel developed algorithms (line expansion and geometry test) to design an original segmentation algorithm. Our approach removes the dependency on environmental factors that traditionally influence lighting conditions, which in turn have negative impact on pixel-based segmentation techniques. We present test outcomes on realistic visual data collected from an aircraft, reporting on preliminary feedback about the performance of the detection. We demonstrate consistent performances over 97% detection rate.

Recursive actuator fault detection and diagnosis for emergency landing of UASs

This paper presents a practical recursive fault detection and diagnosis (FDD) scheme for online identification of actuator faults for unmanned aerial systems (UASs) based on the unscented Kalman filtering (UKF) method. The proposed FDD algorithm aims to monitor health status of actuators and provide indication of actuator faults with reliability, offering necessary information for the design of fault-tolerant flight control systems to compensate for side-effects and improve fail-safe capability when actuator faults occur. The fault detection is conducted by designing separate UKFs to detect aileron and elevator faults using a nonlinear six degree-of-freedom (DOF) UAS model. The fault diagnosis is achieved by isolating true faults by using the Bayesian Classifier (BC) method together with a decision criterion to avoid false alarms. High-fidelity simulations with and without measurement noise are conducted with practical constraints considered for typical actuator fault scenarios, and the proposed FDD exhibits consistent effectiveness in identifying occurrence of actuator faults, verifying its suitability for integration into the design of fault-tolerant flight control systems for emergency landing of UASs.

Multi-rotor with suspended load: System dynamics and control toolbox

There is an increasing demand for Unmanned Aerial Systems (UAS) to carry suspended loads as this can provide significant benefits to several applications in agriculture, law enforcement and construction. The load impact on the underlying system dynamics should not be neglected as significant feedback forces may be induced on the vehicle during certain flight manoeuvres. The constant variation in operating point induced by the slung load also causes conventional controllers to demand increased control effort. Much research has focused on standard multi-rotor position and attitude control with and without a slung load. However, predictive control schemes, such as Nonlinear Model Predictive Control (NMPC), have not yet been fully explored. To this end, we present a novel controller for safe and precise operation of multi-rotors with heavy slung load in three dimensions. The paper describes a System Dynamics and Control Simulation Toolbox for use with MATLAB/SIMULINK which includes a detailed simulation of the multi-rotor and slung load as well as a predictive controller to manage the nonlinear dynamics whilst accounting for system constraints. It is demonstrated that the controller simultaneously tracks specified waypoints and actively damps large slung load oscillations. A linear-quadratic regulator (LQR) is derived and control performance is compared. Results show the improved performance of the predictive controller for a larger flight envelope, including aggressive manoeuvres and large slung load displacements. The computational cost remains relatively small, amenable to practical implementations.

Real-time wind speed estimation and compensation for improved flight

This paper presents the development and experimental validation of a prototype system for online estimation and compensation of wind disturbances onboard small Rotorcraft unmanned aerial systems (RUAS). The proposed approach consists of integrating a small pitot-static system onboard the vehicle and using simple but effective algorithms for estimating the wind speed in real time. The baseline flight controller has been augmented with a feed-forward term to compensate for these wind disturbances, thereby improving the flight performance of small RUAS in windy conditions. The paper also investigates the use of online airspeed measurements in a closed-loop for controlling the RUAS forward motion without the aid of a global positioning system (GPS). The results of more than 80 flights with a RUAS confirm the validity of our approach.

Drones y el orden legal internacional. Tecnología, estrategia y largas cadenas de acción

The main thesis of this article is that the increasing recourse to the use of unmanned aerial systems in asymmetric warfare and the beginning routinization of U.S. drone operations represent part of an evolutionary change in the spatial ordering of global politics -- Using a heuristic framework based on actor-network theory, it is argued that practices of panoptic observation and selective airstrikes, being in need of legal justification, contribute to a reterritorialization of asymmetric conflicts -- Under a new normative spatial regime, a legal condition of state immaturity is constructed, which establishes a zone of conditional sovereignty subject to transnational aerial policing -- At the same time, this process is neither a deterministic result of the new technology nor a deliberate effect of policies to which drones are merely neutral instruments -- Rather, military technology and political decisions both form part of a long chain of action which has evolved under the specific circumstances of recent military interventions