Integrated architecture for vision-based indoor localization and mapping of a quadrotor micro-air vehicle

Atualmente os sistemas de pilotagem autónoma de quadricópteros estão a ser desenvolvidos de forma a efetuarem navegação em espaços exteriores, onde o sinal de GPS pode ser utilizado para definir waypoints de navegação, modos de position e altitude hold, returning home, entre outros. Contudo, o problema de navegação autónoma em espaços fechados sem que se utilize um sistema de posicionamento global dentro de uma sala, subsiste como um problema desafiante e sem solução fechada. Grande parte das soluções são baseadas em sensores dispendiosos, como o LIDAR ou como sistemas de posicionamento externos (p.ex. Vicon, Optitrack). Algumas destas soluções reservam a capacidade de processamento de dados dos sensores e dos algoritmos mais exigentes para sistemas de computação exteriores ao veículo, o que também retira a componente de autonomia total que se pretende num veículo com estas características. O objetivo desta tese pretende, assim, a preparação de um sistema aéreo não-tripulado de pequeno porte, nomeadamente um quadricóptero, que integre diferentes módulos que lhe permitam simultânea localização e mapeamento em espaços interiores onde o sinal GPS ´e negado, utilizando, para tal, uma câmara RGB-D, em conjunto com outros sensores internos e externos do quadricóptero, integrados num sistema que processa o posicionamento baseado em visão e com o qual se pretende que efectue, num futuro próximo, planeamento de movimento para navegação. O resultado deste trabalho foi uma arquitetura integrada para análise de módulos de localização, mapeamento e navegação, baseada em hardware aberto e barato e frameworks state-of-the-art disponíveis em código aberto. Foi também possível testar parcialmente alguns módulos de localização, sob certas condições de ensaio e certos parâmetros dos algoritmos. A capacidade de mapeamento da framework também foi testada e aprovada. A framework obtida encontra-se pronta para navegação, necessitando apenas de alguns ajustes e testes.

Recent international activity in cooperative vehicle-highway automation systems /

This report summarizes the current state of the art in cooperative vehicle-highway automation systems in Europe and Asia based on a series of meetings, demonstrations, and site visits, combined with the results of literature review. This review covers systems that provide drivers with a range of automation capabilities, from driver assistance to fully automated driving, with an emphasis on cooperative systems that involve active exchanges of information between the vehicles and the roadside and among separate vehicles. The trends in development and deployment of these systems are examined by country, and the similarities and differences relative to the U.S. situation are noted, leading toward recommendations for future U.S. action. The Literature Review on Recent International Activity in Cooperative Vehicle-Highway Automation Systems is published separately as FHWA-HRT-13-025.

Literature review on recent international activity in cooperative vehicle-highway automation systems /

This literature review supports the report, Recent International Activity in Cooperative Vehicle-Highway Automation Systems. It reviews the published literature in English dating from 2007 or later about non-U.S.-based work on cooperative vehicle-highway automation systems. This review covers work performed in Europe and Japan, with application to transit buses, heavy trucks, and passenger cars. In addition to fully automated driving of the vehicles (without human intervention), it also covers partial automation systems, which automate subsets of the total driving process. Recent International Activity in Cooperative Vehicle Highway Automation Systems is published separately as FHWA-HRT-12-033.

Designing a Computer Vision System for Underwater Robotic Interventions

This thesis deals with the challenging problem of designing systems able to perceive objects in underwater environments. In the last few decades research activities in robotics have advanced the state of art regarding intervention capabilities of autonomous systems. State of art in fields such as localization and navigation, real time perception and cognition, safe action and manipulation capabilities, applied to ground environments (both indoor and outdoor) has now reached such a readiness level that it allows high level autonomous operations. On the opposite side, the underwater environment remains a very difficult one for autonomous robots. Water influences the mechanical and electrical design of systems, interferes with sensors by limiting their capabilities, heavily impacts on data transmissions, and generally requires systems with low power consumption in order to enable reasonable mission duration. Interest in underwater applications is driven by needs of exploring and intervening in environments in which human capabilities are very limited. Nowadays, most underwater field operations are carried out by manned or remotely operated vehicles, deployed for explorations and limited intervention missions. Manned vehicles, directly on-board controlled, expose human operators to risks related to the stay in field of the mission, within a hostile environment. Remotely Operated Vehicles (ROV) currently represent the most advanced technology for underwater intervention services available on the market. These vehicles can be remotely operated for long time but they need support from an oceanographic vessel with multiple teams of highly specialized pilots. Vehicles equipped with multiple state-of-art sensors and capable to autonomously plan missions have been deployed in the last ten years and exploited as observers for underwater fauna, seabed, ship wrecks, and so on. On the other hand, underwater operations like object recovery and equipment maintenance are still challenging tasks to be conducted without human supervision since they require object perception and localization with much higher accuracy and robustness, to a degree seldom available in Autonomous Underwater Vehicles (AUV). This thesis reports the study, from design to deployment and evaluation, of a general purpose and configurable platform dedicated to stereo-vision perception in underwater environments. Several aspects related to the peculiar environment characteristics have been taken into account during all stages of system design and evaluation: depth of operation and light conditions, together with water turbidity and external weather, heavily impact on perception capabilities. The vision platform proposed in this work is a modular system comprising off-the-shelf components for both the imaging sensors and the computational unit, linked by a high performance ethernet network bus. The adopted design philosophy aims at achieving high flexibility in terms of feasible perception applications, that should not be as limited as in case of a special-purpose and dedicated hardware. Flexibility is required by the variability of underwater environments, with water conditions ranging from clear to turbid, light backscattering varying with daylight and depth, strong color distortion, and other environmental factors. Furthermore, the proposed modular design ensures an easier maintenance and update of the system over time. Performance of the proposed system, in terms of perception capabilities, has been evaluated in several underwater contexts taking advantage of the opportunity offered by the MARIS national project. Design issues like energy power consumption, heat dissipation and network capabilities have been evaluated in different scenarios. Finally, real-world experiments, conducted in multiple and variable underwater contexts, including open sea waters, have led to the collection of several datasets that have been publicly released to the scientific community. The vision system has been integrated in a state of the art AUV equipped with a robotic arm and gripper, and has been exploited in the robot control loop to successfully perform underwater grasping operations.

Methane fluxes and carbonate deposits at eastern Mediterranean cold seeps

High acoustic seafloor-backscatter signals characterize hundreds of patches of methane-derived authigenic carbonates and chemosynthetic communities associated with hydrocarbon seepage on the Nile Deep Sea Fan (NDSF) in the Eastern Mediterranean Sea. During a high-resolution ship-based multibeam survey covering a ~ 225 km**2 large seafloor area in the Central Province of the NDSF we identified 163 high-backscatter patches at water depths between 1500 and 1800 m, and investigated the source, composition, turnover, flux and fate of emitted hydrocarbons. Systematic Parasound single beam echosounder surveys of the water column showed hydroacoustic anomalies (flares), indicative of gas bubble streams, above 8% of the high-backscatter patches. In echosounder records flares disappeared in the water column close to the upper limit of the gas hydrate stability zone located at about 1350 m water depth due to decomposition of gas hydrate skins and subsequent gas dissolution. Visual inspection of three high-backscatter patches demonstrated that sediment cementation has led to the formation of continuous flat pavements of authigenic carbonates typically 100 to 300 m in diameter. Volume estimates, considering results from high-resolution autonomous underwater vehicle (AUV)-based multibeam mapping, were used to calculate the amount of carbonate-bound carbon stored in these slabs. Additionally, the flux of methane bubbles emitted at one high-backscatter patch was estimated (0.23 to 2.3 × 10**6 mol a**-1) by combined AUV flare mapping with visual observations by remotely operated vehicle (ROV). Another high-backscatter patch characterized by single carbonate pieces, which were widely distributed and interspaced with sediments inhabited by thiotrophic, chemosynthetic organisms, was investigated using in situ measurements with a benthic chamber and ex situ sediment core incubation and allowed for estimates of the methane consumption (0.1 to 1 × 10**6 mol a**-1) and dissolved methane flux (2 to 48 × 10**6 mol a**-1). Our comparison of dissolved and gaseous methane fluxes as well as methane-derived carbonate reservoirs demonstrates the need for quantitative assessment of these different methane escape routes and their interaction with the geo-, bio-, and hydrosphere at cold seeps.

Integrated Monitoring of Mola mola Behaviour in Space and Time

Over the last decade, ocean sunfish movements have been monitored worldwide using various satellite tracking methods. This study reports the near-real time monitoring of fine-scale (< 10 m) behaviour of sunfish. The study was conducted in southern Portugal in May 2014 and involved satellite tags and underwater and surface robotic vehicles to measure both the movements and the contextual environment of the fish. A total of four individuals were tracked using custom-made GPS satellite tags providing geolocation estimates of fine-scale resolution. These accurate positions further informed sunfish areas of restricted search (ARS), which were directly correlated to steep thermal frontal zones. Simultaneously, and for two different occasions, an Autonomous Underwater Vehicle (AUV) video-recorded the path of the tracked fish and detected buoyant particles in the water column. Importantly, the densities of these particles were also directly correlated to steep thermal gradients. Thus, both sunfish foraging behaviour (ARS) and possibly prey densities, were found to be influenced by analogous environmental conditions. In addition, the dynamic structure of the water transited by the tracked individuals was described by a Lagrangian modelling approach. The model informed the distribution of zooplankton in the region, both horizontally and in the water column, and the resultant simulated densities positively correlated with sunfish ARS behaviour estimator (rs = 0.184, p<0.001). The model also revealed that tracked fish opportunistically displace with respect to subsurface current flow. Thus, we show how physical forcing and current structure provide a rationale for a predator’s fine-scale behaviour observed over a two weeks in May 2014.

Integrated Monitoring of Mola mola Behaviour in Space and Time

Over the last decade, ocean sunfish movements have been monitored worldwide using various satellite tracking methods. This study reports the near-real time monitoring of fine-scale (< 10 m) behaviour of sunfish. The study was conducted in southern Portugal in May 2014 and involved satellite tags and underwater and surface robotic vehicles to measure both the movements and the contextual environment of the fish. A total of four individuals were tracked using custom-made GPS satellite tags providing geolocation estimates of fine-scale resolution. These accurate positions further informed sunfish areas of restricted search (ARS), which were directly correlated to steep thermal frontal zones. Simultaneously, and for two different occasions, an Autonomous Underwater Vehicle (AUV) video-recorded the path of the tracked fish and detected buoyant particles in the water column. Importantly, the densities of these particles were also directly correlated to steep thermal gradients. Thus, both sunfish foraging behaviour (ARS) and possibly prey densities, were found to be influenced by analogous environmental conditions. In addition, the dynamic structure of the water transited by the tracked individuals was described by a Lagrangian modelling approach. The model informed the distribution of zooplankton in the region, both horizontally and in the water column, and the resultant simulated densities positively correlated with sunfish ARS behaviour estimator (rs = 0.184, p<0.001). The model also revealed that tracked fish opportunistically displace with respect to subsurface current flow. Thus, we show how physical forcing and current structure provide a rationale for a predator’s fine-scale behaviour observed over a two weeks in May 2014.

Autonomous Vehicles and Commercial Real Estate

Hardly a day goes by without the release of a handful of news stories about autonomous vehicles (or AVs for short). The proverbial “tipping point” of awareness has been reached in the public consciousness as AV technology is quickly becoming the new focus of firms from Silicon Valley to Detroit and beyond. Automation has, and will continue to have far-reaching implications for many human activities, but for driving, the technology is here. Google has been in talks with automaker Ford (1), Elon Musk has declared that Tesla will have the appropriate technology in two years (2), GM is paired-up with Lyft (3), Uber is in development-mode (4), Microsoft and Volvo have announced a partnership (5), Apple has been piloting its top-secret project “Titan” (6), Toyota is working on its own technology (7), as is BMW (8). Audi (9) made a splash by sending a driverless A7 concept car 550 miles from San Francisco to Las Vegas just in time to roll-into the 2016 Consumer Electronics Show. Clearly, the race is on.

Geometric versus Model Predictive Control based guidance algorithms for fixed-wing UAVs in the presence of very strong wind fields.

The recent years have witnessed increased development of small, autonomous fixed-wing Unmanned Aerial Vehicles (UAVs). In order to unlock widespread applicability of these platforms, they need to be capable of operating under a variety of environmental conditions. Due to their small size, low weight, and low speeds, they require the capability of coping with wind speeds that are approaching or even faster than the nominal airspeed. In this thesis, a nonlinear-geometric guidance strategy is presented, addressing this problem. More broadly, a methodology is proposed for the high-level control of non-holonomic unicycle-like vehicles in the presence of strong flowfields (e.g. winds, underwater currents) which may outreach the maximum vehicle speed. The proposed strategy guarantees convergence to a safe and stable vehicle configuration with respect to the flowfield, while preserving some tracking performance with respect to the target path. As an alternative approach, an algorithm based on Model Predictive Control (MPC) is developed, and a comparison between advantages and disadvantages of both approaches is drawn. Evaluations in simulations and a challenging real-world flight experiment in very windy conditions confirm the feasibility of the proposed guidance approach.

CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION OF VEHICLE MOTION

Simultaneous Localization and Mapping (SLAM) is a procedure used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. A typical SLAM consists of four main components, namely, experimental setup (data gathering), vehicle pose estimation, feature extraction, and filtering. Feature extraction is the process of realizing significant features from the unknown environment such as corners, edges, walls, and interior features. In this work, an original feature extraction algorithm specific to distance measurements obtained through SONAR sensor data is presented. This algorithm has been constructed by combining the SONAR Salient Feature Extraction Algorithm and the Triangulation Hough Based Fusion with point-in-polygon detection. The reconstructed maps obtained through simulations and experimental data with the fusion algorithm are compared to the maps obtained with existing feature extraction algorithms. Based on the results obtained, it is suggested that the proposed algorithm can be employed as an option for data obtained from SONAR sensors in environment, where other forms of sensing are not viable. The algorithm fusion for feature extraction requires the vehicle pose estimation as an input, which is obtained from a vehicle pose estimation model. For the vehicle pose estimation, the author uses sensor integration to estimate the pose of the mobile vehicle. Different combinations of these sensors are studied (e.g., encoder, gyroscope, or encoder and gyroscope). The different sensor fusion techniques for the pose estimation are experimentally studied and compared. The vehicle pose estimation model, which produces the least amount of error, is used to generate inputs for the feature extraction algorithm fusion. In the experimental studies, two different environmental configurations are used, one without interior features and another one with two interior features. Numerical and experimental findings are discussed. Finally, the SLAM algorithm is implemented along with the algorithms for feature extraction and vehicle pose estimation. Three different cases are experimentally studied, with the floor of the environment intentionally altered to induce slipping. Results obtained for implementations with and without SLAM are compared and discussed. The present work represents a step towards the realization of autonomous inspection platforms for performing concurrent localization and mapping in harsh environments.

Implementación de un detector de coral utilizando filtros Gabor Wavelets y máquinas de aprendizaje

Este trabajo se enfoca en la implementación de un detector de arrecife de coral de desempeño rápido que se utiliza para un vehículo autónomo submarino (Autonomous Underwater Vehicle, AUV, por sus siglas en inglés). Una detección rápida de la presencia de coral asegura la estabilización del AUV frente al arrecife en el menor tiempo posible, evitando colisiones con el coral. La detección de coral se hace en una imagen que captura la escena que percibe la cámara del AUV. Se realiza una clasificación píxel por píxel entre dos clases: arrecife de coral y el plano de fondo que no es coral. A cada píxel de la imagen se le asigna un vector característico, el mismo que se genera mediante el uso de filtros Gabor Wavelets. Éstos son implementados en C++ y la librería OpenCV. Los vectores característicos son clasificados a través de nueve algoritmos de máquinas de aprendizaje. El desempeño de cada algoritmo se compara mediante la precisión y el tiempo de ejecución. El algoritmo de Árboles de Decisión resultó ser el más rápido y preciso de entre todos los algoritmos. Se creó una base de datos de 621 imágenes de corales de Belice (110 imágenes de entrenamiento y 511 imágenes de prueba).

Underwater acoustic networks: the FP7 UAN project

The EU-funded project UAN - Underwater Acoustic Network aims at conceiving, developing and testing at sea an innovative and operational concept for integrating in a unique communication system submerged, surface and aerial sensors with the objective of protecting off-shore and coastline critical infrastructures. A crucial aspect of the project consisted in the use of autonomous underwater vehicles (AUVs) as mobile nodes in the underwater acoustic communication network. In particular, AUVs have the role of adapting the network geometry to the variation of the acoustic channel. This paper reports on the project concept and vision as well as on the progress of its various development phases. The recent at-sea successes that have been demonstrated within the UAN framework are detailed and results of the final UAN project demonstration, UAN11, held in the May of 2011, are reported. The UAN network was in operation for five continuous days with up to five nodes, of which three of them were mobile nodes. © IFAC.

Design and characterization of automated agricultural vehicle systems

Nowadays, technological advancements have brought industry and research towards the automation of various processes. Automation brings a reduction in costs and an improvement in product quality. For this reason, companies are pushing research to investigate new technologies. The agriculture industry has always looked towards automating various processes, from product processing to storage. In the last years, the automation of harvest and cultivation phases also has become attractive, pushed by the advancement of autonomous driving. Nevertheless, ADAS systems are not enough. Merging different technologies will be the solution to obtain total automation of agriculture processes. For example, sensors that estimate products' physical and chemical properties can be used to evaluate the maturation level of fruit. Therefore, the fusion of these technologies has a key role in industrial process automation. In this dissertation, ADAS systems and sensors for precision agriculture will be both treated. Several measurement procedures for characterizing commercial 3D LiDARs will be proposed and tested to cope with the growing need for comparison tools. Axial errors and transversal errors have been investigated. Moreover, a measurement method and setup for evaluating the fog effect on 3D LiDARs will be proposed. Each presented measurement procedure has been tested. The obtained results highlight the versatility and the goodness of the proposed approaches. Regarding the precision agriculture sensors, a measurement approach for the Moisture Content and density estimation of crop directly on the field is presented. The approach regards the employment of a Near Infrared spectrometer jointly with Partial Least Square statistical analysis. The approach and the model will be described together with a first laboratory prototype used to evaluate the NIRS approach. Finally, a prototype for on the field analysis is realized and tested. The test results are promising, evidencing that the proposed approach is suitable for Moisture Content and density estimation.

A ROS-based software architecture for a versatile collaborative dual-armed autonomous mobile robot for the manufacturing industry

The industrial context is changing rapidly due to advancements in technology fueled by the Internet and Information Technology. The fourth industrial revolution counts integration, flexibility, and optimization as its fundamental pillars, and, in this context, Human-Robot Collaboration has become a crucial factor for manufacturing sustainability in Europe. Collaborative robots are appealing to many companies due to their low installation and running costs and high degree of flexibility, making them ideal for reshoring production facilities with a short return on investment. The ROSSINI European project aims to implement a true Human-Robot Collaboration by designing, developing, and demonstrating a modular and scalable platform for integrating human-centred robotic technologies in industrial production environments. The project focuses on safety concerns related to introducing a cobot in a shared working area and aims to lay the groundwork for a new working paradigm at the industrial level. The need for a software architecture suitable to the robotic platform employed in one of three use cases selected to deploy and test the new technology was the main trigger of this Thesis. The chosen application consists of the automatic loading and unloading of raw-material reels to an automatic packaging machine through an Autonomous Mobile Robot composed of an Autonomous Guided Vehicle, two collaborative manipulators, and an eye-on-hand vision system for performing tasks in a partially unstructured environment. The results obtained during the ROSSINI use case development were later used in the SENECA project, which addresses the need for robot-driven automatic cleaning of pharmaceutical bins in a very specific industrial context. The inherent versatility of mobile collaborative robots is evident from their deployment in the two projects with few hardware and software adjustments. The positive impact of Human-Robot Collaboration on diverse production lines is a motivation for future investments in research on this increasingly popular field by the industry.

Motion Control of an Advanced Electric Implement for Agricultural Autonomous Vehicles

Nowadays, the development of intelligent and autonomous vehicles used to perform agricultural activities is essential to improve quantity and quality of agricultural productions. Moreover, with automation techniques it is possible to reduce the usage of agrochemicals and minimize the pollution. The University of Bologna is developing an innovative system for orchard management called ORTO (Orchard Rapid Transportation System). This system involves an autonomous electric vehicle capable to perform agricultural activities inside an orchard structure. The vehicle is equipped with an implement capable to perform different tasks. The purpose of this thesis project is to control the vehicle and the implement to perform an inter-row grass mowing. This kind of task requires a synchronized motion between the traction motors and the implement motors. A motion control system has been developed to generate trajectories and manage their synchronization. Two main trajectories type have been used: a five order polynomial trajectory and a trapezoidal trajectory. These two kinds of trajectories have been chosen in order to perform a uniform grass mowing, paying a particular attention to the constrains of the system. To synchronize the motions, the electronic cams approach has been adopted. A master profile has been generated and all the trajectories have been linked to the master motion. Moreover, a safety system has been developed. The aim of this system is firstly to improve the safety during the motion, furthermore it allows to manage obstacle detection and avoidance. Using some particular techniques obstacles can be detected and recovery action can be performed to overcome the problem. Once the measured force reaches the predefined force threshold, then the vehicle stops immediately its motion. The whole project has been developed by employing Matlab and Simulink. Eventually, the software has been translated into C code and executed on the TI Lauchpad XL board.