Hybrid coordination of reinforcement learning-based behaviors for AUV control

This paper proposes a hybrid coordination method for behavior-based control architectures. The hybrid method takes advantages of the robustness and modularity in competitive approaches as well as optimized trajectories in cooperative ones. This paper shows the feasibility of applying this hybrid method with a 3D-navigation to an autonomous underwater vehicle (AUV). The behaviors are learnt online by means of reinforcement learning. A continuous Q-learning implemented with a feed-forward neural network is employed. Realistic simulations were carried out. The results obtained show the good performance of the hybrid method on behavior coordination as well as the convergence of the behaviors

Augmented state Kalman filtering for AUV navigation

Addresses the problem of estimating the motion of an autonomous underwater vehicle (AUV), while it constructs a visual map ("mosaic" image) of the ocean floor. The vehicle is equipped with a down-looking camera which is used to compute its motion with respect to the seafloor. As the mosaic increases in size, a systematic bias is introduced in the alignment of the images which form the mosaic. Therefore, this accumulative error produces a drift in the estimation of the position of the vehicle. When the arbitrary trajectory of the AUV crosses over itself, it is possible to reduce this propagation of image alignment errors within the mosaic. A Kalman filter with augmented state is proposed to optimally estimate both the visual map and the vehicle position

Probabilistic sonar scan matching for an AUV

This paper proposes MSISpIC, a probabilistic sonar scan matching algorithm for the localization of an autonomous underwater vehicle (AUV). The technique uses range scans gathered with a Mechanical Scanning Imaging Sonar (MSIS), the robot displacement estimated through dead-reckoning using a Doppler velocity log (DVL) and a motion reference unit (MRU). The proposed method is an extension of the pIC algorithm. An extended Kalman filter (EKF) is used to estimate the robot-path during the scan in order to reference all the range and bearing measurements as well as their uncertainty to a scan fixed frame before registering. The major contribution consists of experimentally proving that probabilistic sonar scan matching techniques have the potential to improve the DVL-based navigation. The algorithm has been tested on an AUV guided along a 600 m path within an abandoned marina underwater environment with satisfactory results

Pose-based SLAM with probabilistic scan matching algorithm using a mechanical scanned imaging sonar

This paper proposes a pose-based algorithm to solve the full SLAM problem for an autonomous underwater vehicle (AUV), navigating in an unknown and possibly unstructured environment. The technique incorporate probabilistic scan matching with range scans gathered from a mechanical scanning imaging sonar (MSIS) and the robot dead-reckoning displacements estimated from a Doppler velocity log (DVL) and a motion reference unit (MRU). The proposed method utilizes two extended Kalman filters (EKF). The first, estimates the local path travelled by the robot while grabbing the scan as well as its uncertainty and provides position estimates for correcting the distortions that the vehicle motion produces in the acoustic images. The second is an augment state EKF that estimates and keeps the registered scans poses. The raw data from the sensors are processed and fused in-line. No priory structural information or initial pose are considered. The algorithm has been tested on an AUV guided along a 600 m path within a marina environment, showing the viability of the proposed approach

Mission control system for dam inspection with an AUV

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

ICTINEUAUV wins the First SAUC-E competition

A pioneer team of students of the University of Girona decided to design and develop an autonomous underwater vehicle (AUV) called ICTINEU-AUV to face the Student Autonomous Underwater Challenge-Europe (SAUC-E). The prototype has evolved from the initial computer aided design (CAD) model to become an operative AUV in the short period of seven months. The open frame and modular design principles together with the compatibility with other robots previously developed at the lab have provided the main design philosophy. Hence, at the robot's core, two networked computers give access to a wide set of sensors and actuators. The Gentoo/Linux distribution was chosen as the onboard operating system. A software architecture based on a set of distributed objects with soft real time capabilities was developed and a hybrid control architecture including mission control, a behavioural layer and a robust map-based localization algorithm made ICTINEU-AUV the winning entry

On AUV control architecture

This paper surveys control architectures proposed in the literature and describes a control architecture that is being developed for a semi-autonomous underwater vehicle for intervention missions (SAUVIM) at the University of Hawaii. Conceived as hybrid, this architecture has been organized in three layers: planning, control and execution. The mission is planned with a sequence of subgoals. Each subgoal has a related task supervisor responsible for arranging a set of pre-programmed task modules in order to achieve the subgoal. Task modules are the key concept of the architecture. They are the main building blocks and can be dynamically re-arranged by the task supervisor. In our architecture, deliberation takes place at the planning layer while reaction is dealt through the parallel execution of the task modules. Hence, the system presents both a hierarchical and an heterarchical decomposition, being able to show a predictable response while keeping rapid reactivity to the dynamic environment

Visual SLAM for 3D large-scale seabed acquisition employing underwater vehicles

This paper presents a novel technique to align partial 3D reconstructions of the seabed acquired by a stereo camera mounted on an autonomous underwater vehicle. Vehicle localization and seabed mapping is performed simultaneously by means of an Extended Kalman Filter. Passive landmarks are detected on the images and characterized considering 2D and 3D features. Landmarks are re-observed while the robot is navigating and data association becomes easier but robust. Once the survey is completed, vehicle trajectory is smoothed by a Rauch-Tung-Striebel filter obtaining an even better alignment of the 3D views and yet a large-scale acquisition of the seabed

Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Equipo AVORA de Robótica Marina de la ULPGC: experiencia y Perspectivas

[ES] AVORA es un grupo de trabajo multidisciplinar integrado por 11 estudiantes (este año con un nuevo miembro, Michael Smith proveniente de la Universidad de Rhode Island, USA) y 3 supervisores. Se ha creado para diseñar y desarrollar un AUV (Autonomous Underwarter Vehicle) capaz de detectar objetos bajo el agua sin supervisión, así como navegar de manera autónoma con la ayuda de un sonar. Todo esto se está llevando a cabo para superar las misiones que se plantean en la competición europea SAUCE. Como novedad este año, el equipo tiene intenciones también de presentarse a la competición EUROATHLON

Prüftechnik für RFID bei Hochgeschwindigkeitsanwendungen im Transportwesen

Im Rahmen des Forschungsprojektes TagDrive wird die Applikation von passiven Transpondern auf 13,56 MHz-Basis untersucht, die auf der Fahrbahn befestigt und zur Spurführung und Navigation von Fahrzeugen verwendet werden. Um den Einsatz von Folientranspondern (Smart Labels) bei hohen Fahrzeuggeschwindigkeiten in Zusammenhang mit Transponderlesegeräten am Fahrzeug zu testen, wurde ein Hochgeschwindigkeitsprüfstand für Relativgeschwindigkeiten zwischen Transponder und Lesegerät von bis zu 100 km/h entwickelt und Versuche bei variierenden Leseabständen durchgeführt. Für mechanische Prüfungen der applizierten Transponder inklusive Gehäuse wurden Druckprüfungen mit Hilfe einer servohydraulischen Prüfeinrichtung durchgeführt und ein neuer pneumatischer Prüfstand entwickelt, der den Überrollvorgang durch ein Rad nachbildet. Für klimatische Tests unter anderem nach DIN EN 60068-2-67 bzw. IEC 68-2-67 wurde ein Klimaschrank verwendet.

(Supplemet S1) Silicon oxide and Potassium + Sodium oxide from the PACManus hydrothermal area

This study presents a systematic analysis and interpretation of autonomous underwater vehicle-based microbathymetry combined with remotely operated vehicle (ROV) video recordings, rock analyses and temperaturemeasurements within the PACManus hydrothermal area located on Pual Ridge in the Bismarck Sea of eastern Manus Basin. The data obtained during research cruise Magellan-06 and So-216 provides a framework for understanding the relationship between the volcanism, tectonismand hydrothermal activity. PACManus is a submarine felsic vocanically-hosted hydrothermal area that hosts multiple vent fields locatedwithin several hundredmeters of one another but with different fluid chemistries, vent temperatures and morphologies. The total area of hydrothermal activity is estimated to be 20,279m**2. Themicrobathymetrymaps combinedwith the ROV video observations allow for precise high-resolution mapping estimates of the areal extents of hydrothermal activity.We find the distribution of hydrothermal fields in the PACManus area is primarily controlled by volcanic features that include lava domes, thick andmassive blocky lava flows, breccias and feeder dykes. Spatial variation in the permeability of local volcanic facies appears to control the distribution of venting within a field.We define a three-stage chronological sequence for the volcanic evolution of the PACManus based on lava flow morphology, sediment cover and lava SiO2 concentration. In Stage-1, sparsely to moderately porphyritic dacite lavas (68-69.8 wt.% SiO2) erupted to form domes or cryptodomes. In Stage-2, aphyric lava with slightly lower SiO2 concentrations (67.2-67.9 wt.% SiO2) formed jumbled and pillowed lava flows. In the most recent phase Stage-3, massive blocky lavaswith 69 to 72.5wt.% SiO2were erupted throughmultiple vents constructing a volcanic ridge identified as the PACManus neovolcanic zone. The transition between these stages may be gradual and related to progressive heating of a silicic magma following a recharge event of hot, mantle-derived melts.

Dissipation rate measurements during R/V Maria S. Merian cruise MSM21/1b dives

GEOMAR's autonomous underwater vehicle (AUV Abyss REMUS 6000) was deployed within the framework of a multi-platform experiment in June 2012 with R/V Maria S. Merian cruise MSM21/1b at about 180 km downstream of Denmark Strait. The scientific payload included a pumped Seabird 49 FastCAT CTD system, a paroscientific pressure sensor, and shear and temperature microstructure profiler from Rockland Scientific Inc.. In total, six of eight AUV dives were carried out successfully. Aborts on three dives were caused by strong counter currents the AUV experienced in the Denmark Strait Overflow plume, which made the AUV fail to reach its waypoints on schedule. During all missions the AUV was programmed to dive at constant depth levels along? straight legs approximately parallel to chosen isobaths with a constant speed of 1.6 m s-1 through the water.

Physical oceanography from R/V Maria S. Merian cruise MSM21/1b dives

GEOMAR's autonomous underwater vehicle (AUV Abyss REMUS 6000) was deployed within the framework of a multi-platform experiment in June 2012 with R/V Maria S. Merian cruise MSM21/1b at about 180 km downstream of Denmark Strait. The scientific payload included a pumped Seabird 49 FastCAT CTD system, a paroscientific pressure sensor, and shear and temperature microstructure profiler from Rockland Scientific Inc.. In total, six of eight AUV dives were carried out successfully. Aborts on three dives were caused by strong counter currents the AUV experienced in the Denmark Strait Overflow plume, which made the AUV fail to reach its waypoints on schedule. During all missions the AUV was programmed to dive at constant depth levels along? straight legs approximately parallel to chosen isobaths with a constant speed of 1.6 m s-1 through the water.

Physical oceanography, current meter data, and microstructure measurements 180 km downstream of Denmark Strait sill during Maria S. Merian cruise MSM21/1b

The main objectives of the cruise MSM21/1b were to characterize the spatio-temporal variability of the Denmark Strait overflow and to identify processes responsible for the exchange of the overflow plume with ambient water downstream of Denmark Strait. A multi-platform approach was taken to achieve the goals, based on moorings, an autonomous underwater vehicle (AUV), as well as lowered and vessel-mounted observations. From these platforms, measurements of temperature, salinity, dissolved oxygen, current velocity, dissipation of turbulent kinetic energy, and bottom pressure were obtained.