Motion Planning and Controls with Safety and Temporal Constraints

Motion planning, or trajectory planning, commonly refers to a process of converting high-level task specifications into low-level control commands that can be executed on the system of interest. For different applications, the system will be different. It can be an autonomous vehicle, an Unmanned Aerial Vehicle(UAV), a humanoid robot, or an industrial robotic arm. As human machine interaction is essential in many of these systems, safety is fundamental and crucial. Many of the applications also involve performing a task in an optimal manner within a given time constraint. Therefore, in this thesis, we focus on two aspects of the motion planning problem. One is the verification and synthesis of the safe controls for autonomous ground and air vehicles in collision avoidance scenarios. The other part focuses on the high-level planning for the autonomous vehicles with the timed temporal constraints. In the first aspect of our work, we first propose a verification method to prove the safety and robustness of a path planner and the path following controls based on reachable sets. We demonstrate the method on quadrotor and automobile applications. Secondly, we propose a reachable set based collision avoidance algorithm for UAVs. Instead of the traditional approaches of collision avoidance between trajectories, we propose a collision avoidance scheme based on reachable sets and tubes. We then formulate the problem as a convex optimization problem seeking control set design for the aircraft to avoid collision. We apply our approach to collision avoidance scenarios of quadrotors and fixed-wing aircraft. In the second aspect of our work, we address the high level planning problems with timed temporal logic constraints. Firstly, we present an optimization based method for path planning of a mobile robot subject to timed temporal constraints, in a dynamic environment. Temporal logic (TL) can address very complex task specifications such as safety, coverage, motion sequencing etc. We use metric temporal logic (MTL) to encode the task specifications with timing constraints. We then translate the MTL formulae into mixed integer linear constraints and solve the associated optimization problem using a mixed integer linear program solver. We have applied our approach on several case studies in complex dynamical environments subjected to timed temporal specifications. Secondly, we also present a timed automaton based method for planning under the given timed temporal logic specifications. We use metric interval temporal logic (MITL), a member of the MTL family, to represent the task specification, and provide a constructive way to generate a timed automaton and methods to look for accepting runs on the automaton to find an optimal motion (or path) sequence for the robot to complete the task.

Vigilância e monitorização dos espaços marítimos sob soberania ou jurisdição portuguesa

Portugal tem uma das maiores Zonas Económicas Exclusivas a nível mundial, encerrando este espaço marítimo uma riqueza que ainda não se encontra devidamente aferida, mas que se julga ser enorme. Por ela passam anualmente milhares de navios, com os mais diversos destinos e transportando as mais variadas cargas. A posição geostratégica do país coloca-o no centro de algumas das mais movimentadas rotas marítimas, sendo por isso de extrema importância vigiar e monitorizar as águas portuguesas, por forma a garantir que as leis e regulamentos de direito internacional marítimo são cumpridos e que o interesse nacional é devidamente salvaguardado. Deste modo, a presente dissertação tem como objeto de estudo os sistemas de vigilância e monitorização marítimos, pretendendo constituir-se como um contributo para a melhoria do atual sistema de vigilância e monitorização dos espaços marítimos sob soberania ou jurisdição portuguesa, focando-se, para tal, nos sistemas aéreos e espaciais para a deteção de meios de superfície. Para tal, numa primeira parte considera-se estudar o ambiente marítimo e as ameaças que o afetam. Na segunda parte estudam-se os atuais sistemas que contribuem para o conhecimento situacional marítimo em Portugal culminando na terceira parte com o estudo dos meios e sensores que permitem melhorar a cobertura do espaço marítimo, com o objetivo final de garantir a segurança no mar. Através do estudo realizado foi possível concluir-se que as aeronaves não tripuladas afiguram-se como o futuro mais imediato para o esclarecimento do panorama marítimo, sendo que os satélites surgem numa segunda linha, pois apesar dos seus custos mais elevados, poderão também dar um enorme contributo para o conhecimento situacional marítimo ao serem capazes de cobrir maiores áreas e mais rapidamente.

Validação Computacional tendo em vista a Interoperabilidade entre os Veículos Aéreos Não Tripulados

Nos dias de hoje, com o contínuo desenvolvimento e inovação no campo dos UAVs (Unmanned Aerial Vehciles), o mundo já tem como adquiridos os benefícios que estes sistemas podem fornecer. Os benefícios obtidos com a aplicação destes sistemas abrange tanto as forças armadas como industrias e organizações civis. Todas as nações e indústrias querem ter uma cota parte no futuro desta tecnologia. Diferentes UAVs foram desenvolvidos, mas estes, diferem em termos de arquitetura e protocolos de comunicação. Protocolos como o STANAG 4586, MAVLink, JAUS e ROS são só alguns exemplos. A proliferação de informação através destes sistemas e as suas consolas de comando e controlo é uma das principais preocupações, principalmente pelas forças armadas. Uma das principais prioridades é combinar forças de diferentes nações, principalmente pelos membros NATO. A necessidade de uma consola para cada tipo de sistema devido à falta de padronização apresenta assim um problema. É conhecida a necessidade de uma padronização em termos de arquitetura por camadas e de comunicação tendo em vista a interoperabilidade entre estes sistemas. Não existe nenhuma que esteja a ser implementada como documento padrão. Pretende-se que o STANAG 4586 seja o documento padrão para os membros NATO e, por conseguinte, todos os esforços estão direcionados em desenvolver sistemas que o consigam implementar. Os diferentes UAVs já existentes possuem o seu próprio protocolo de comunicação e a alteração de toda a sua estrutura não é fácil. A ideia de fazer uma conversão de linguagens como alternativa surge como uma solução teórica ótima. Utilizando um piloto automático que comunica com a sua consola através da linguagem MAVLink esta dissertação tem como objetivo desenvolver um programa computacional que converta as mensagens MAVLink em STANAG 4586 e estudar se o tempo de conversão é operacionalmente válido tendo em conta os requisitos operacionais dos sistemas.

POSE ESTIMATION AND 3D RECONSTRUCTION USING SENSOR FUSION

A camera maps 3-dimensional (3D) world space to a 2-dimensional (2D) image space. In the process it loses the depth information, i.e., the distance from the camera focal point to the imaged objects. It is impossible to recover this information from a single image. However, by using two or more images from different viewing angles this information can be recovered, which in turn can be used to obtain the pose (position and orientation) of the camera. Using this pose, a 3D reconstruction of imaged objects in the world can be computed. Numerous algorithms have been proposed and implemented to solve the above problem; these algorithms are commonly called Structure from Motion (SfM). State-of-the-art SfM techniques have been shown to give promising results. However, unlike a Global Positioning System (GPS) or an Inertial Measurement Unit (IMU) which directly give the position and orientation respectively, the camera system estimates it after implementing SfM as mentioned above. This makes the pose obtained from a camera highly sensitive to the images captured and other effects, such as low lighting conditions, poor focus or improper viewing angles. In some applications, for example, an Unmanned Aerial Vehicle (UAV) inspecting a bridge or a robot mapping an environment using Simultaneous Localization and Mapping (SLAM), it is often difficult to capture images with ideal conditions. This report examines the use of SfM methods in such applications and the role of combining multiple sensors, viz., sensor fusion, to achieve more accurate and usable position and reconstruction information. This project investigates the role of sensor fusion in accurately estimating the pose of a camera for the application of 3D reconstruction of a scene. The first set of experiments is conducted in a motion capture room. These results are assumed as ground truth in order to evaluate the strengths and weaknesses of each sensor and to map their coordinate systems. Then a number of scenarios are targeted where SfM fails. The pose estimates obtained from SfM are replaced by those obtained from other sensors and the 3D reconstruction is completed. Quantitative and qualitative comparisons are made between the 3D reconstruction obtained by using only a camera versus that obtained by using the camera along with a LIDAR and/or an IMU. Additionally, the project also works towards the performance issue faced while handling large data sets of high-resolution images by implementing the system on the Superior high performance computing cluster at Michigan Technological University.

The role of agriculture and food consumption in tropical conservation

A growing human population, shifting human dietary habits, and climate change are negatively affecting global ecosystems on a massive scale. Expanding agricultural areas to feed a growing population drives extensive habitat loss, and climate change compounds stresses on both food security and ecosystems. Understanding the negative effects of human diet and climate change on agricultural and natural ecosystems provides a context within which potential technological and behavioral solutions can be proposed to help maximize conservation. The purpose of this research was to (1) examine the potential effects of climate change on the suitability of areas for commercial banana plantations in Latin America in the 2050s and how shifts in growing areas could affect protected areas; (2) test the ability of small unmanned aerial vehicles (UAVs) to map productivity of banana plantations as a potential tool for increasing yields and decreasing future plantation expansions; (3) project the effects on biodiversity of increasing rates of animal product consumption in developing megadiverse countries; and (4) estimate the capacity of global pasture biomass production and Fischer-Tropsch hydrocarbon synthesis (IGCC-FT) processing to meet electricity, gasoline and diesel needs. The results indicate that (1) the overall extent of areas suitable for conventional banana cultivation is predicted to decrease by 19% by 2050 because of a hotter and drier climate, but all current banana exporting countries are predicted to maintain some suitable areas with no effects on protected areas; (2) Spatial patterns of NDVI and ENDVI were significantly positively correlated with several metrics of fruit yield and quality, indicating that UAV systems can be used in banana plantations to map spatial patterns of fruit yield; (3) Livestock production is the single largest driver of habitat loss, and both livestock and feedstock production are increasing in developing biodiverse tropical countries. Reducing global animal product consumption should therefore be at the forefront of strategies aimed at reducing biodiversity loss; (4) Removing livestock from global pasture lands and instead utilizing the biomass production could produce enough energy to meet 100% of the electricity, gasoline, and diesel needs of over 40 countries with extensive grassland ecosystems, primarily in tropical developing countries.^

The Role of B5G UAV-Aided Mobile Networks

With the advent of 5G, several novel network paradigms and technologies have been proposed to fulfil the key requirements imposed. Flexibility, efficiency and scalability, along with sustainability and convenience for expenditure have to be addressed in targeting these brand new needs. Among novel paradigms introduced in the scientific literature in recent years, a constant and increasing interest lies in the use of unmanned aerial vehicles (UAVs) as network nodes supporting the legacy terrestrial network for service provision. Their inherent features of moving nodes make them able to be deployed on-demand in real-time. Which, in practical terms, means having them acting as a base station (BS) when and where there is the highest need. This thesis investigates then the potential role of UAV-aided mobile radio networks, in order to validate the concept of adding an aerial network component and assess the system performance, from early to later stages of its deployment. This study is intended for 5G and beyond systems, to allow time for the technology to mature. Since advantages can be manyfold, the aerial network component is considered at the network layer under several aspects, from connectivity to radio resource management. A particular emphasis is given to trajectory design, because of the efficiency and flexibility it potentially adds to the infrastructure. Two different frameworks have been proposed, to take into account both a re-adaptable heuristic and an optimal solution. Moreover, diverse use cases are taken under analysis, from mobile broadband to machine and vehicular communications. The thesis aim is thus to discuss the potential and advantages of UAV-aided systems from a broad perspective. Results demonstrate that the technology has good prospects for diverse scenarios with a few arrangements.

Modelling, simulation and control of rotary-wing aircraft in multi-agent scenario

The topic of this thesis is the design and the implementation of mathematical models and control system algorithms for rotary-wing unmanned aerial vehicles to be used in cooperative scenarios. The use of rotorcrafts has many attractive advantages, since these vehicles have the capability to take-off and land vertically, to hover and to move backward and laterally. Rotary-wing aircraft missions require precise control characteristics due to their unstable and heavy coupling aspects. As a matter of fact, flight test is the most accurate way to evaluate flying qualities and to test control systems. However, it may be very expensive and/or not feasible in case of early stage design and prototyping. A good compromise is made by a preliminary assessment performed by means of simulations and a reduced flight testing campaign. Consequently, having an analytical framework represents an important stage for simulations and control algorithm design. In this work mathematical models for various helicopter configurations are implemented. Different flight control techniques for helicopters are presented with theoretical background and tested via simulations and experimental flight tests on a small-scale unmanned helicopter. The same platform is used also in a cooperative scenario with a rover. Control strategies, algorithms and their implementation to perform missions are presented for two main scenarios. One of the main contributions of this thesis is to propose a suitable control system made by a classical PID baseline controller augmented with L1 adaptive contribution. In addition a complete analytical framework and the study of the dynamics and the stability of a synch-rotor are provided. At last, the implementation of cooperative control strategies for two main scenarios that include a small-scale unmanned helicopter and a rover.

Hybrid ground-aerial mesh networks for IoT monitoring applications: network design and software platform development

The Internet of Things (IoT) has grown rapidly in recent years, leading to an increased need for efficient and secure communication between connected devices. Wireless Sensor Networks (WSNs) are composed of small, low-power devices that are capable of sensing and exchanging data, and are often used in IoT applications. In addition, Mesh WSNs involve intermediate nodes forwarding data to ensure more robust communication. The integration of Unmanned Aerial Vehicles (UAVs) in Mesh WSNs has emerged as a promising solution for increasing the effectiveness of data collection, as UAVs can act as mobile relays, providing extended communication range and reducing energy consumption. However, the integration of UAVs and Mesh WSNs still poses new challenges, such as the design of efficient control and communication strategies. This thesis explores the networking capabilities of WSNs and investigates how the integration of UAVs can enhance their performance. The research focuses on three main objectives: (1) Ground Wireless Mesh Sensor Networks, (2) Aerial Wireless Mesh Sensor Networks, and (3) Ground/Aerial WMSN integration. For the first objective, we investigate the use of the Bluetooth Mesh standard for IoT monitoring in different environments. The second objective focuses on deploying aerial nodes to maximize data collection effectiveness and QoS of UAV-to-UAV links while maintaining the aerial mesh connectivity. The third objective investigates hybrid WMSN scenarios with air-to-ground communication links. One of the main contribution of the thesis consists in the design and implementation of a software framework called "Uhura", which enables the creation of Hybrid Wireless Mesh Sensor Networks and abstracts and handles multiple M2M communication stacks on both ground and aerial links. The operations of Uhura have been validated through simulations and small-scale testbeds involving ground and aerial devices.

Deep learning based style transfer for low altitude aerial imagery

Unmanned Aerial Vehicle (UAVs) equipped with cameras have been fast deployed to a wide range of applications, such as smart cities, agriculture or search and rescue applications. Even though UAV datasets exist, the amount of open and quality UAV datasets is limited. So far, we want to overcome this lack of high quality annotation data by developing a simulation framework for a parametric generation of synthetic data. The framework accepts input via a serializable format. The input specifies which environment preset is used, the objects to be placed in the environment along with their position and orientation as well as additional information such as object color and size. The result is an environment that is able to produce UAV typical data: RGB image from the UAVs camera, altitude, roll, pitch and yawn of the UAV. Beyond the image generation process, we improve the resulting image data photorealism by using Synthetic-To-Real transfer learning methods. Transfer learning focuses on storing knowledge gained while solving one problem and applying it to a different - although related - problem. This approach has been widely researched in other affine fields and results demonstrate it to be an interesing area to investigate. Since simulated images are easy to create and synthetic-to-real translation has shown good quality results, we are able to generate pseudo-realistic images. Furthermore, object labels are inherently given, so we are capable of extending the already existing UAV datasets with realistic quality images and high resolution meta-data. During the development of this thesis we have been able to produce a result of 68.4% on UAVid. This can be considered a new state-of-art result on this dataset.

Sistema de visão para percepção e navegação de um UAV Quadrotor

Mestrado em Engenharia Electrotécnica e de Computadores

Desenvolvimento de módulos para planeamento e controlo de execução de missões de veículos aéreos não tripulados

Com a evolução da tecnologia, os UAVs (unmanned aerial vehicles) são cada vez mais utilizados, não só em missões de risco para o ser Humano, mas também noutro tipo de missões, como é o caso de missões de inspeção, vigilância, busca e salvamento. Isto devese ao baixo custo das plataformas assim como à sua enorme fiabilidade e facilidade de operação. Esta dissertação surge da necessidade de aumentar a autonomia dos UAVs do projeto PITVANT (Projeto de Investigação e Tecnologia em Veículos Aéreos Não Tripulados), projeto de investigação colaborativa entre a AFA (Academia da Força Aérea) e a FEUP (Faculdade de Engenharia da Universidade do Porto), relativamente ao planeamento de trajetórias entre dois pontos no espaço, evitando os obstáculos que intersetem o caminho. Para executar o planeamento da trajetória mais curta entre dois pontos, foi implementado o algoritmo de pesquisa A*, por ser um algoritmo de pesquisa de soluções ótimas. A área de pesquisa é decomposta em células regulares e o centro das células são os nós de pesquisa do A*. O tamanho de cada célula é dependente da dinâmica de cada aeronave. Para que as aeronaves não colidam com os obstáculos, foi desenvolvido um método numérico baseado em relações trigonométricas para criar uma margem de segurança em torno de cada obstáculo. Estas margens de segurança são configuráveis, sendo o seu valor por defeito igual ao raio mínimo de curvatura da aeronave à velocidade de cruzeiro. De forma a avaliar a sua escalabilidade, o algoritmo foi avaliado com diferentes números de obstáculos. As métricas utilizadas para avaliação do algoritmo foram o tempo de computação do mesmo e o comprimento do trajeto obtido. Foi ainda comparado o desempenho do algoritmo desenvolvido com um algoritmo já implementado, do tipo fast marching.

Radar Based Collision detection developments on USV ROAZ II

This work presents the integration of obstacle detection and analysis capabilities in a coherent and advanced C&C framework allowing mixed-mode control in unmanned surface systems. The collision avoidance work has been successfully integrated in an operational autonomous surface vehicle and demonstrated in real operational conditions. We present the collision avoidance system, the ROAZ autonomous surface vehicle and the results obtained at sea tests. Limitations of current COTS radar systems are also discussed and further research directions are proposed towards the development and integration of advanced collision avoidance systems taking in account the different requirements in unmanned surface vehicles.

Planeamento de Trajetória para Operações de Busca e Salvamento com UAVs

Os sistemas autónomos trazem como mais valia aos cenários de busca e salvamento a possibilidade de minimizar a presença de Humanos em situações de perigo e a capacidade de aceder a locais de difícil acesso. Na dissertação propõe-se endereçar novos métodos para perceção e navegação de veículos aéreos não tripulados (UAV), tendo como foco principal o planeamento de trajetórias e deteção de obstáculos. No que respeita à perceção foi desenvolvido um método para gerar clusters tendo por base os voxels gerados pelo Octomap. Na área de navegação, foram desenvolvidos dois novos métodos de planeamento de trajetórias, GPRM (Grid Probabilistic Roadmap) e PPRM (Particle Probabilistic Roadmap), que tem como método base para o seu desenvolvimento o PRM. O primeiro método desenvolvido, GPRM, espalha as partículas numa grid pré-definida, construindo posteriormente o roadmap na área determinada pela grid e com isto estima o trajeto mais curto até ao ponto destino. O segundo método desenvolvido, PPRM, espalha as partículas pelo cenário de aplicação, gera o roadmap considerando o mapa total e atribui uma probabilidade que irá permitir definir a trajetória otimizada. Para analisar a performance de cada método em comparação com o PRM, efetua-se a sua avaliação em três cenários distintos com recurso ao simulador MORSE.

Da guerra remota: a desumanização do poder aéreo, a interferência e a interação humana no futuro da guerra

Dissertação apresentada para cumprimento dos requisitos necessários à obtenção do grau de Doutor em Relações Internacionais, especialidade de Estudos de Segurança e Estratégia

UAV - based imagery processing using structure from motion and remote sensing technology

With the recent advances in technology and miniaturization of devices such as GPS or IMU, Unmanned Aerial Vehicles became a feasible platform for a Remote Sensing applications. The use of UAVs compared to the conventional aerial platforms provides a set of advantages such as higher spatial resolution of the derived products. UAV - based imagery obtained by a user grade cameras introduces a set of problems which have to be solved, e. g. rotational or angular differences or unknown or insufficiently precise IO and EO camera parameters. In this work, UAV - based imagery of RGB and CIR type was processed using two different workflows based on PhotoScan and VisualSfM software solutions resulting in the DSM and orthophoto products. Feature detection and matching parameters influence on the result quality as well as a processing time was examined and the optimal parameter setup was presented. Products of the both workflows were compared in terms of a quality and a spatial accuracy. Both workflows were compared by presenting the processing times and quality of the results. Finally, the obtained products were used in order to demonstrate vegetation classification. Contribution of the IHS transformations was examined with respect to the classification accuracy.