A Novel Framework for Real-time Traffic Flow Parameter Estimation from Aerial Videos

Thesis (Master's)--University of Washington, 2016-06

Structural Health Monitoring using Index Based Reasoning for Unmanned Aerial Vehicles

Unmanned Aerial Vehicles (UAVs) may develop cracks, erosion, delamination or other damages due to aging, fatigue or extreme loads. Identifying these damages is critical for the safe and reliable operation of the systems. ^ Structural Health Monitoring (SHM) is capable of determining the conditions of systems automatically and continually through processing and interpreting the data collected from a network of sensors embedded into the systems. With the desired awareness of the systems’ health conditions, SHM can greatly reduce operational cost and speed up maintenance processes. ^ The purpose of this study is to develop an effective, low-cost, flexible and fault tolerant structural health monitoring system. The proposed Index Based Reasoning (IBR) system started as a simple look-up-table based diagnostic system. Later, Fast Fourier Transformation analysis and neural network diagnosis with self-learning capabilities were added. The current version is capable of classifying different health conditions with the learned characteristic patterns, after training with the sensory data acquired from the operating system under different status. ^ The proposed IBR systems are hierarchy and distributed networks deployed into systems to monitor their health conditions. Each IBR node processes the sensory data to extract the features of the signal. Classifying tools are then used to evaluate the local conditions with health index (HI) values. The HI values will be carried to other IBR nodes in the next level of the structured network. The overall health condition of the system can be obtained by evaluating all the local health conditions. ^ The performance of IBR systems has been evaluated by both simulation and experimental studies. The IBR system has been proven successful on simulated cases of a turbojet engine, a high displacement actuator, and a quad rotor helicopter. For its application on experimental data of a four rotor helicopter, IBR also performed acceptably accurate. The proposed IBR system is a perfect fit for the low-cost UAVs to be the onboard structural health management system. It can also be a backup system for aircraft and advanced Space Utility Vehicles. ^

The Role of Agriculture and Food Consumption in Tropical Conservation

A growing human population, shifting human dietary habits, and climate change are negatively affecting global ecosystems on a massive scale. Expanding agricultural areas to feed a growing population drives extensive habitat loss, and climate change compounds stresses on both food security and ecosystems. Understanding the negative effects of human diet and climate change on agricultural and natural ecosystems provides a context within which potential technological and behavioral solutions can be proposed to help maximize conservation. The purpose of this research was to (1) examine the potential effects of climate change on the suitability of areas for commercial banana plantations in Latin America in the 2050s and how shifts in growing areas could affect protected areas; (2) test the ability of small unmanned aerial vehicles (UAVs) to map productivity of banana plantations as a potential tool for increasing yields and decreasing future plantation expansions; (3) project the effects on biodiversity of increasing rates of animal product consumption in developing megadiverse countries; and (4) estimate the capacity of global pasture biomass production and Fischer-Tropsch hydrocarbon synthesis (IGCC-FT) processing to meet electricity, gasoline and diesel needs. The results indicate that (1) the overall extent of areas suitable for conventional banana cultivation is predicted to decrease by 19% by 2050 because of a hotter and drier climate, but all current banana exporting countries are predicted to maintain some suitable areas with no effects on protected areas; (2) Spatial patterns of NDVI and ENDVI were significantly positively correlated with several metrics of fruit yield and quality, indicating that UAV systems can be used in banana plantations to map spatial patterns of fruit yield; (3) Livestock production is the single largest driver of habitat loss, and both livestock and feedstock production are increasing in developing biodiverse tropical countries. Reducing global animal product consumption should therefore be at the forefront of strategies aimed at reducing biodiversity loss; (4) Removing livestock from global pasture lands and instead utilizing the biomass production could produce enough energy to meet 100% of the electricity, gasoline, and diesel needs of over 40 countries with extensive grassland ecosystems, primarily in tropical developing countries.

On precise three-dimensional environment modeling via UAV-based photogrammetric systems

Abstract : Images acquired from unmanned aerial vehicles (UAVs) can provide data with unprecedented spatial and temporal resolution for three-dimensional (3D) modeling. Solutions developed for this purpose are mainly operating based on photogrammetry concepts, namely UAV-Photogrammetry Systems (UAV-PS). Such systems are used in applications where both geospatial and visual information of the environment is required. These applications include, but are not limited to, natural resource management such as precision agriculture, military and police-related services such as traffic-law enforcement, precision engineering such as infrastructure inspection, and health services such as epidemic emergency management. UAV-photogrammetry systems can be differentiated based on their spatial characteristics in terms of accuracy and resolution. That is some applications, such as precision engineering, require high-resolution and high-accuracy information of the environment (e.g. 3D modeling with less than one centimeter accuracy and resolution). In other applications, lower levels of accuracy might be sufficient, (e.g. wildlife management needing few decimeters of resolution). However, even in those applications, the specific characteristics of UAV-PSs should be well considered in the steps of both system development and application in order to yield satisfying results. In this regard, this thesis presents a comprehensive review of the applications of unmanned aerial imagery, where the objective was to determine the challenges that remote-sensing applications of UAV systems currently face. This review also allowed recognizing the specific characteristics and requirements of UAV-PSs, which are mostly ignored or not thoroughly assessed in recent studies. Accordingly, the focus of the first part of this thesis is on exploring the methodological and experimental aspects of implementing a UAV-PS. The developed system was extensively evaluated for precise modeling of an open-pit gravel mine and performing volumetric-change measurements. This application was selected for two main reasons. Firstly, this case study provided a challenging environment for 3D modeling, in terms of scale changes, terrain relief variations as well as structure and texture diversities. Secondly, open-pit-mine monitoring demands high levels of accuracy, which justifies our efforts to improve the developed UAV-PS to its maximum capacities. The hardware of the system consisted of an electric-powered helicopter, a high-resolution digital camera, and an inertial navigation system. The software of the system included the in-house programs specifically designed for camera calibration, platform calibration, system integration, onboard data acquisition, flight planning and ground control point (GCP) detection. The detailed features of the system are discussed in the thesis, and solutions are proposed in order to enhance the system and its photogrammetric outputs. The accuracy of the results was evaluated under various mapping conditions, including direct georeferencing and indirect georeferencing with different numbers, distributions and types of ground control points. Additionally, the effects of imaging configuration and network stability on modeling accuracy were assessed. The second part of this thesis concentrates on improving the techniques of sparse and dense reconstruction. The proposed solutions are alternatives to traditional aerial photogrammetry techniques, properly adapted to specific characteristics of unmanned, low-altitude imagery. Firstly, a method was developed for robust sparse matching and epipolar-geometry estimation. The main achievement of this method was its capacity to handle a very high percentage of outliers (errors among corresponding points) with remarkable computational efficiency (compared to the state-of-the-art techniques). Secondly, a block bundle adjustment (BBA) strategy was proposed based on the integration of intrinsic camera calibration parameters as pseudo-observations to Gauss-Helmert model. The principal advantage of this strategy was controlling the adverse effect of unstable imaging networks and noisy image observations on the accuracy of self-calibration. The sparse implementation of this strategy was also performed, which allowed its application to data sets containing a lot of tie points. Finally, the concepts of intrinsic curves were revisited for dense stereo matching. The proposed technique could achieve a high level of accuracy and efficiency by searching only through a small fraction of the whole disparity search space as well as internally handling occlusions and matching ambiguities. These photogrammetric solutions were extensively tested using synthetic data, close-range images and the images acquired from the gravel-pit mine. Achieving absolute 3D mapping accuracy of 11±7 mm illustrated the success of this system for high-precision modeling of the environment.

DESIGN, FABRICATION, AND PERFORMANCE CHARACTERIZATION OF MULTIFUNCTIONAL STRUCTURES TO HARVEST SOLAR ENERGY FOR FLAPPING WING AERIAL VEHICLES

Flapping Wing Aerial Vehicles (FWAVs) have the capability to combine the benefits of both fixed wing vehicles and rotary vehicles. However, flight time is limited due to limited on-board energy storage capacity. For most Unmanned Aerial Vehicle (UAV) operators, frequent recharging of the batteries is not ideal due to lack of nearby electrical outlets. This imposes serious limitations on FWAV flights. The approach taken to extend the flight time of UAVs was to integrate photovoltaic solar cells onto different structures of the vehicle to harvest and use energy from the sun. Integration of the solar cells can greatly improve the energy capacity of an UAV; however, this integration does effect the performance of the UAV and especially FWAVs. The integration of solar cells affects the ability of the vehicle to produce the aerodynamic forces necessary to maintain flight. This PhD dissertation characterizes the effects of solar cell integration on the performance of a FWAV. Robo Raven, a recently developed FWAV, is used as the platform for this work. An additive manufacturing technique was developed to integrate photovoltaic solar cells into the wing and tail structures of the vehicle. An approach to characterizing the effects of solar cell integration to the wings, tail, and body of the UAV is also described. This approach includes measurement of aerodynamic forces generated by the vehicle and measurements of the wing shape during the flapping cycle using Digital Image Correlation. Various changes to wing, body, and tail design are investigated and changes in performance for each design are measured. The electrical performance from the solar cells is also characterized. A new multifunctional performance model was formulated that describes how integration of solar cells influences the flight performance. Aerodynamic models were developed to describe effects of solar cell integration force production and performance of the FWAV. Thus, performance changes can be predicted depending on changes in design. Sensing capabilities of the solar cells were also discovered and correlated to the deformation of the wing. This demonstrated that the solar cells were capable of: (1) Lightweight and flexible structure to generate aerodynamic forces, (2) Energy harvesting to extend operational time and autonomy, (3) Sensing of an aerodynamic force associated with wing deformation. Finally, different flexible photovoltaic materials with higher efficiencies are investigated, which enable the multifunctional wings to provide enough solar power to keep the FWAV aloft without batteries as long as there is enough sunlight to power the vehicle.

Development of a Real-Time Hierarchical 3D Path Planning Algorithm for Unmanned Aerial Vehicles

Unmanned aerial vehicles (UAVs) frequently operate in partially or entirely unknown environments. As the vehicle traverses the environment and detects new obstacles, rapid path replanning is essential to avoid collisions. This thesis presents a new algorithm called Hierarchical D* Lite (HD*), which combines the incremental algorithm D* Lite with a novel hierarchical path planning approach to replan paths sufficiently fast for real-time operation. Unlike current hierarchical planning algorithms, HD* does not require map corrections before planning a new path. Directional cost scale factors, path smoothing, and Catmull-Rom splines are used to ensure the resulting paths are feasible. HD* sacrifices optimality for real-time performance. Its computation time and path quality are dependent on the map size, obstacle density, sensor range, and any restrictions on planning time. For the most complex scenarios tested, HD* found paths within 10% of optimal in under 35 milliseconds.

RISK-BASED MULTIOBJECTIVE PATH PLANNING AND DESIGN OPTIMIZATION FOR UNMANNED AERIAL VEHICLES

Safe operation of unmanned aerial vehicles (UAVs) over populated areas requires reducing the risk posed by a UAV if it crashed during its operation. We considered several types of UAV risk-based path planning problems and developed techniques for estimating the risk to third parties on the ground. The path planning problem requires making trade-offs between risk and flight time. Four optimization approaches for solving the problem were tested; a network-based approach that used a greedy algorithm to improve the original solution generated the best solutions with the least computational effort. Additionally, an approach for solving a combined design and path planning problems was developed and tested. This approach was extended to solve robust risk-based path planning problem in which uncertainty about wind conditions would affect the risk posed by a UAV.

Optical Wireless Communication for Mobile Platforms

The past few decades have witnessed the widespread adaptation of wireless devices such as cellular phones and Wifi-connected laptops, and demand for wireless communication is expected to continue to increase. Though radio frequency (RF) communication has traditionally dominated in this application space, recent decades have seen an increasing interest in the use of optical wireless (OW) communication to supplement RF communications. In contrast to RF communication technology, OW systems offer the use of largely unregulated electromagnetic spectrum and large bandwidths for communication. They also offer the potential to be highly secure against jamming and eavesdropping. Interest in OW has become especially keen in light of the maturation of light-emitting diode (LED) technology. This maturation, and the consequent emerging ubiquity of LED technology in lighting systems, has motivated the exploration of LEDs for wireless communication purposes in a wide variety of applications. Recent interest in this field has largely focused on the potential for indoor local area networks (LANs) to be realized with increasingly common LED-based lighting systems. We envision the use of LED-based OW to serve as a supplement to RF technology in communication between mobile platforms, which may include automobiles, robots, or unmanned aerial vehicles (UAVs). OW technology may be especially useful in what are known as RF-denied environments, in which RF communication may be prohibited or undesirable. The use of OW in these settings presents major challenges. In contrast to many RF systems, OWsystems that operate at ranges beyond a few meters typically require relatively precise alignment. For example, some laser-based optical wireless communication systems require alignment precision to within small fractions of a degree. This level of alignment precision can be difficult to maintain between mobile platforms. Additionally, the use of OW systems in outdoor settings presents the challenge of interference from ambient light, which can be much brighter than any LED transmitter. This thesis addresses these challenges to the use of LED-based communication between mobile platforms. We propose and analyze a dual-link LED-based system that uses one link with a wide transmission beam and relaxed alignment constraints to support a more narrow, precisely aligned, higher-data-rate link. The use of an optical link with relaxed alignment constraints to support the alignment of a more precisely aligned link motivates our exploration of a panoramic imaging receiver for estimating the range and bearing of neighboring nodes. The precision of such a system is analyzed and an experimental system is realized. Finally, we present an experimental prototype of a self-aligning LED-based link.

Motion Planning and Controls with Safety and Temporal Constraints

Motion planning, or trajectory planning, commonly refers to a process of converting high-level task specifications into low-level control commands that can be executed on the system of interest. For different applications, the system will be different. It can be an autonomous vehicle, an Unmanned Aerial Vehicle(UAV), a humanoid robot, or an industrial robotic arm. As human machine interaction is essential in many of these systems, safety is fundamental and crucial. Many of the applications also involve performing a task in an optimal manner within a given time constraint. Therefore, in this thesis, we focus on two aspects of the motion planning problem. One is the verification and synthesis of the safe controls for autonomous ground and air vehicles in collision avoidance scenarios. The other part focuses on the high-level planning for the autonomous vehicles with the timed temporal constraints. In the first aspect of our work, we first propose a verification method to prove the safety and robustness of a path planner and the path following controls based on reachable sets. We demonstrate the method on quadrotor and automobile applications. Secondly, we propose a reachable set based collision avoidance algorithm for UAVs. Instead of the traditional approaches of collision avoidance between trajectories, we propose a collision avoidance scheme based on reachable sets and tubes. We then formulate the problem as a convex optimization problem seeking control set design for the aircraft to avoid collision. We apply our approach to collision avoidance scenarios of quadrotors and fixed-wing aircraft. In the second aspect of our work, we address the high level planning problems with timed temporal logic constraints. Firstly, we present an optimization based method for path planning of a mobile robot subject to timed temporal constraints, in a dynamic environment. Temporal logic (TL) can address very complex task specifications such as safety, coverage, motion sequencing etc. We use metric temporal logic (MTL) to encode the task specifications with timing constraints. We then translate the MTL formulae into mixed integer linear constraints and solve the associated optimization problem using a mixed integer linear program solver. We have applied our approach on several case studies in complex dynamical environments subjected to timed temporal specifications. Secondly, we also present a timed automaton based method for planning under the given timed temporal logic specifications. We use metric interval temporal logic (MITL), a member of the MTL family, to represent the task specification, and provide a constructive way to generate a timed automaton and methods to look for accepting runs on the automaton to find an optimal motion (or path) sequence for the robot to complete the task.

Reconocimiento de una plataforma para aterrizaje de UAV mediante procesamiento de imágenes

En el año 2016 se vendieron en EE.UU más de un millón de Unmanned Aerial Vehicles (UAVs, Vehículos aéreos no tripulados), casi el doble que el año anterior, país del que se dispone de información. Para el año 2020 se estima que este mercado alcance los 5.600 millones de dólares en todo el mundo, creciendo a un ritmo del 30% anual. Este crecimiento demuestra que existe un mercado en expansión con muchas y diversas oportunidades de investigación. El rango de aplicaciones en los que se utiliza este tipo de vehículos es innumerable. Desde finales del s.XX, los UAVs han estado presentes en multitud de aplicaciones, principalmente en misiones de reconocimiento. Su principal ventaja radica en que pueden ser utilizados en situaciones de alto riesgo sin suponer una amenaza para ningún tripulante. En los últimos años, la fabricación de vehículos asequibles económicamente ha permitido que su uso se extienda a otros sectores. A día de hoy uno de los campos en los que ha adquirido gran relevancia es en agricultura, contribuyendo a la automatización y monitorización de cultivos, pero también se ha extendido su uso a diferentes sistemas, tales como seguridad, cartografía o monitorización, entre otros [1]. Es en esta situación en la que se propone el proyecto SALACOM [2], que explora la posibilidad de utilizar esta tecnología en sistemas de repuesta rápida para la detección y contención de vertidos contaminantes en entornos acuáticos con el apoyo de vehículos autónomos marinos de superficie (USV, Unmanned Surface Vehicles). En el mencionado proyecto se pretende utilizar sistemas UAVs para detectar y analizar las zonas de vertido y proveer la información respecto a la localización y las técnicas de contención adecuadas a los sistemas USV. Una vez se haya realizado el análisis de la situación del vertido, los USV trabajarían conjuntamente con los UAVs para desplegar las barreras de protección seleccionadas en la zona afectada. Para esto, los UAVs o drones, términos similares en lo que respecta a este proyecto y que a lo largo de esta memoria se usarán indistintamente, deben ser capaces de despegar desde los USV y volver a aterrizar sobre ellos una vez realizada su labor. El proyecto que se describe en la presente memoria se centra en la fase de aterrizaje y, más concretamente, en la detección de la plataforma seleccionada como plantilla mediante técnicas de tratamiento de imágenes. Esto serviría como sistema de apoyo para guiar el dron hacia la plataforma para que pueda realizar el descenso correctamente y finalizar así su misión o bien para realizar operaciones de recarga de la batería. El dron está equipado con la correspondiente cámara de visión a bordo, con la que obtiene las imágenes, las procesa e identifica la plataforma para dirigirse hacia ella, si bien, dado que el sistema de procesamiento de imágenes no se encuentra totalmente operativo, este trabajo se centra en el desarrollo de una aplicación software independiente del sistema de visión a bordo del dron, basada en el desarrollo de técnicas de reconocimiento de la plataforma. La plataforma a utilizar proviene de una patente [3], consistente en una figura geométrica con formas características, de muy difícil aparición en entornos de exterior. La figura pintada en negro se halla impresa sobre un panel de fondo blanco de 1m × 1m de superficie. En este trabajo se han explorado diversas opciones disponibles para realizar la identificación de las regiones de interés. El principal objetivo es realizar la selección de una tecnología que pueda cumplir potencialmente con los criterios necesarios para llevar a cabo la tarea y seleccionar los métodos de detección adecuados para realizar la identificación de la figura contenida en la plataforma. Se ha pretendido utilizar tecnologías de fácil uso, amplío soporte y, cuando ha sido posible, de código libre. Todo ello integrado en una aplicación informática, que es la que se presenta en el presente trabajo.

Mission and Scenario Planning for Unmanned Aerial Vehicles (Path Planning and Collision Avoidance Systems)

As unmanned autonomous vehicles (UAVs) are being widely utilized in military and civil applications, concerns are growing about mission safety and how to integrate dierent phases of mission design. One important barrier to a coste ective and timely safety certication process for UAVs is the lack of a systematic approach for bridging the gap between understanding high-level commander/pilot intent and implementation of intent through low-level UAV behaviors. In this thesis we demonstrate an entire systems design process for a representative UAV mission, beginning from an operational concept and requirements and ending with a simulation framework for segments of the mission design, such as path planning and decision making in collision avoidance. In this thesis, we divided this complex system into sub-systems; path planning, collision detection and collision avoidance. We then developed software modules for each sub-system

Validação Computacional tendo em vista a Interoperabilidade entre os Veículos Aéreos Não Tripulados

Nos dias de hoje, com o contínuo desenvolvimento e inovação no campo dos UAVs (Unmanned Aerial Vehciles), o mundo já tem como adquiridos os benefícios que estes sistemas podem fornecer. Os benefícios obtidos com a aplicação destes sistemas abrange tanto as forças armadas como industrias e organizações civis. Todas as nações e indústrias querem ter uma cota parte no futuro desta tecnologia. Diferentes UAVs foram desenvolvidos, mas estes, diferem em termos de arquitetura e protocolos de comunicação. Protocolos como o STANAG 4586, MAVLink, JAUS e ROS são só alguns exemplos. A proliferação de informação através destes sistemas e as suas consolas de comando e controlo é uma das principais preocupações, principalmente pelas forças armadas. Uma das principais prioridades é combinar forças de diferentes nações, principalmente pelos membros NATO. A necessidade de uma consola para cada tipo de sistema devido à falta de padronização apresenta assim um problema. É conhecida a necessidade de uma padronização em termos de arquitetura por camadas e de comunicação tendo em vista a interoperabilidade entre estes sistemas. Não existe nenhuma que esteja a ser implementada como documento padrão. Pretende-se que o STANAG 4586 seja o documento padrão para os membros NATO e, por conseguinte, todos os esforços estão direcionados em desenvolver sistemas que o consigam implementar. Os diferentes UAVs já existentes possuem o seu próprio protocolo de comunicação e a alteração de toda a sua estrutura não é fácil. A ideia de fazer uma conversão de linguagens como alternativa surge como uma solução teórica ótima. Utilizando um piloto automático que comunica com a sua consola através da linguagem MAVLink esta dissertação tem como objetivo desenvolver um programa computacional que converta as mensagens MAVLink em STANAG 4586 e estudar se o tempo de conversão é operacionalmente válido tendo em conta os requisitos operacionais dos sistemas.

The role of agriculture and food consumption in tropical conservation

A growing human population, shifting human dietary habits, and climate change are negatively affecting global ecosystems on a massive scale. Expanding agricultural areas to feed a growing population drives extensive habitat loss, and climate change compounds stresses on both food security and ecosystems. Understanding the negative effects of human diet and climate change on agricultural and natural ecosystems provides a context within which potential technological and behavioral solutions can be proposed to help maximize conservation. The purpose of this research was to (1) examine the potential effects of climate change on the suitability of areas for commercial banana plantations in Latin America in the 2050s and how shifts in growing areas could affect protected areas; (2) test the ability of small unmanned aerial vehicles (UAVs) to map productivity of banana plantations as a potential tool for increasing yields and decreasing future plantation expansions; (3) project the effects on biodiversity of increasing rates of animal product consumption in developing megadiverse countries; and (4) estimate the capacity of global pasture biomass production and Fischer-Tropsch hydrocarbon synthesis (IGCC-FT) processing to meet electricity, gasoline and diesel needs. The results indicate that (1) the overall extent of areas suitable for conventional banana cultivation is predicted to decrease by 19% by 2050 because of a hotter and drier climate, but all current banana exporting countries are predicted to maintain some suitable areas with no effects on protected areas; (2) Spatial patterns of NDVI and ENDVI were significantly positively correlated with several metrics of fruit yield and quality, indicating that UAV systems can be used in banana plantations to map spatial patterns of fruit yield; (3) Livestock production is the single largest driver of habitat loss, and both livestock and feedstock production are increasing in developing biodiverse tropical countries. Reducing global animal product consumption should therefore be at the forefront of strategies aimed at reducing biodiversity loss; (4) Removing livestock from global pasture lands and instead utilizing the biomass production could produce enough energy to meet 100% of the electricity, gasoline, and diesel needs of over 40 countries with extensive grassland ecosystems, primarily in tropical developing countries.^

Design of a Novel Auto-Rotating UAV Platform for Underground Mine Cavity Surveying

The purpose of this paper is to investigate the potential for use of UAVs in underground mines and present a prototype design for a novel autorotating UAV platform for underground 3D data collection.