Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Mission control system for dam inspection with an AUV

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

Model-based objects recognition in industrial environments for autonomous vehicles control

Behavior-based navigation of autonomous vehicles requires the recognition of the navigable areas and the potential obstacles. In this paper we describe a model-based objects recognition system which is part of an image interpretation system intended to assist the navigation of autonomous vehicles that operate in industrial environments. The recognition system integrates color, shape and texture information together with the location of the vanishing point. The recognition process starts from some prior scene knowledge, that is, a generic model of the expected scene and the potential objects. The recognition system constitutes an approach where different low-level vision techniques extract a multitude of image descriptors which are then analyzed using a rule-based reasoning system to interpret the image content. This system has been implemented using a rule-based cooperative expert system

The Protein quality control system manages plant defence compound synthesis

Jasmonates are ubiquitous oxylipin-derived phytohormones that are essential in the regulation of many development, growth and defence processes. Across the plant kingdom, jasmonates act as elicitors of the production of bioactive secondarymetabolites that serve in defence against attackers. Knowledge of the conserved jasmonate perception and early signalling machineries is increasing, but the downstream mechanisms that regulate defence metabolism remain largely unknown. Herewe showthat, in the legumeMedicago truncatula, jasmonate recruits the endoplasmic-reticulum-associated degradation (ERAD)quality control system tomanagethe production of triterpene saponins, widespread bioactive compounds that share a biogenic origin with sterols. An ERAD-type RING membraneanchor E3 ubiquitin ligase is co-expressed with saponin synthesis enzymes to control the activity of 3-hydroxy-3-methylglutaryl-CoA reductase (HMGR), the rate-limiting enzyme in the supply of the ubiquitous terpene precursor isopentenyl diphosphate. Thus, unrestrained bioactive saponin accumulationis prevented and plant development and integrity secured. This control apparatus is equivalent to the ERAD system that regulates sterol synthesis in yeasts and mammals but that uses distinct E3 ubiquitin ligases, of the HMGR degradation 1 (HRD1) type, to direct destruction of HMGR. Hence, the general principles for the management of sterol and triterpene saponin biosynthesis are conserved across eukaryotes but can be controlled by divergent regulatory cues.

The Protein quality control system manages plant defence compound synthesis

Jasmonates are ubiquitous oxylipin-derived phytohormones that are essential in the regulation of many development, growth and defence processes. Across the plant kingdom, jasmonates act as elicitors of the production of bioactive secondarymetabolites that serve in defence against attackers. Knowledge of the conserved jasmonate perception and early signalling machineries is increasing, but the downstream mechanisms that regulate defence metabolism remain largely unknown. Herewe showthat, in the legumeMedicago truncatula, jasmonate recruits the endoplasmic-reticulum-associated degradation (ERAD)quality control system tomanagethe production of triterpene saponins, widespread bioactive compounds that share a biogenic origin with sterols. An ERAD-type RING membraneanchor E3 ubiquitin ligase is co-expressed with saponin synthesis enzymes to control the activity of 3-hydroxy-3-methylglutaryl-CoA reductase (HMGR), the rate-limiting enzyme in the supply of the ubiquitous terpene precursor isopentenyl diphosphate. Thus, unrestrained bioactive saponin accumulationis prevented and plant development and integrity secured. This control apparatus is equivalent to the ERAD system that regulates sterol synthesis in yeasts and mammals but that uses distinct E3 ubiquitin ligases, of the HMGR degradation 1 (HRD1) type, to direct destruction of HMGR. Hence, the general principles for the management of sterol and triterpene saponin biosynthesis are conserved across eukaryotes but can be controlled by divergent regulatory cues.

The Protein quality control system manages plant defence compound synthesis

Jasmonates are ubiquitous oxylipin-derived phytohormones that are essential in the regulation of many development, growth and defence processes. Across the plant kingdom, jasmonates act as elicitors of the production of bioactive secondarymetabolites that serve in defence against attackers. Knowledge of the conserved jasmonate perception and early signalling machineries is increasing, but the downstream mechanisms that regulate defence metabolism remain largely unknown. Herewe showthat, in the legumeMedicago truncatula, jasmonate recruits the endoplasmic-reticulum-associated degradation (ERAD)quality control system tomanagethe production of triterpene saponins, widespread bioactive compounds that share a biogenic origin with sterols. An ERAD-type RING membraneanchor E3 ubiquitin ligase is co-expressed with saponin synthesis enzymes to control the activity of 3-hydroxy-3-methylglutaryl-CoA reductase (HMGR), the rate-limiting enzyme in the supply of the ubiquitous terpene precursor isopentenyl diphosphate. Thus, unrestrained bioactive saponin accumulationis prevented and plant development and integrity secured. This control apparatus is equivalent to the ERAD system that regulates sterol synthesis in yeasts and mammals but that uses distinct E3 ubiquitin ligases, of the HMGR degradation 1 (HRD1) type, to direct destruction of HMGR. Hence, the general principles for the management of sterol and triterpene saponin biosynthesis are conserved across eukaryotes but can be controlled by divergent regulatory cues.

The Protein quality control system manages plant defence compound synthesis

Jasmonates are ubiquitous oxylipin-derived phytohormones that are essential in the regulation of many development, growth and defence processes. Across the plant kingdom, jasmonates act as elicitors of the production of bioactive secondarymetabolites that serve in defence against attackers. Knowledge of the conserved jasmonate perception and early signalling machineries is increasing, but the downstream mechanisms that regulate defence metabolism remain largely unknown. Herewe showthat, in the legumeMedicago truncatula, jasmonate recruits the endoplasmic-reticulum-associated degradation (ERAD)quality control system tomanagethe production of triterpene saponins, widespread bioactive compounds that share a biogenic origin with sterols. An ERAD-type RING membraneanchor E3 ubiquitin ligase is co-expressed with saponin synthesis enzymes to control the activity of 3-hydroxy-3-methylglutaryl-CoA reductase (HMGR), the rate-limiting enzyme in the supply of the ubiquitous terpene precursor isopentenyl diphosphate. Thus, unrestrained bioactive saponin accumulationis prevented and plant development and integrity secured. This control apparatus is equivalent to the ERAD system that regulates sterol synthesis in yeasts and mammals but that uses distinct E3 ubiquitin ligases, of the HMGR degradation 1 (HRD1) type, to direct destruction of HMGR. Hence, the general principles for the management of sterol and triterpene saponin biosynthesis are conserved across eukaryotes but can be controlled by divergent regulatory cues.

ICTINEUAUV wins the First SAUC-E competition

A pioneer team of students of the University of Girona decided to design and develop an autonomous underwater vehicle (AUV) called ICTINEU-AUV to face the Student Autonomous Underwater Challenge-Europe (SAUC-E). The prototype has evolved from the initial computer aided design (CAD) model to become an operative AUV in the short period of seven months. The open frame and modular design principles together with the compatibility with other robots previously developed at the lab have provided the main design philosophy. Hence, at the robot's core, two networked computers give access to a wide set of sensors and actuators. The Gentoo/Linux distribution was chosen as the onboard operating system. A software architecture based on a set of distributed objects with soft real time capabilities was developed and a hybrid control architecture including mission control, a behavioural layer and a robust map-based localization algorithm made ICTINEU-AUV the winning entry

Using petri nets to specify and execute missions for autonomous underwater vehicles

This paper presents the design and implementation of a mission control system (MCS) for an autonomous underwater vehicle (AUV) based on Petri nets. In the proposed approach the Petri nets are used to specify as well as to execute the desired autonomous vehicle mission. The mission is easily described using an imperative programming language called mission control language (MCL) that formally describes the mission execution thread. A mission control language compiler (MCL-C) able to automatically translate the MCL into a Petri net is described and a real-time Petri net player that allows to execute the resulting Petri net onboard an AUV are also presented

Dynamic modelling of an underwater vehicle

Nessie is an Autonomous Underwater Vehicle (AUV) created by a team of students in the Heriot Watt University to compete in the Student Autonomous Underwater Competition, Europe (SAUC-E) in August 2006. The main objective of the project is to find the dynamic equation of the robot, dynamic model. With it, the behaviour of the robot will be easier to understand and movement tests will be available by computer without the need of the robot, what is a way to save time, batteries, money and the robot from water inside itself. The object of the second part in this project is setting a control system for Nessie by using the model

Automated monitoring of medical protocols: a secure and distributed architecture

The control of the right application of medical protocols is a key issue in hospital environments. For the automated monitoring of medical protocols, we need a domain-independent language for their representation and a fully, or semi, autonomous system that understands the protocols and supervises their application. In this paper we describe a specification language and a multi-agent system architecture for monitoring medical protocols. We model medical services in hospital environments as specialized domain agents and interpret a medical protocol as a negotiation process between agents. A medical service can be involved in multiple medical protocols, and so specialized domain agents are independent of negotiation processes and autonomous system agents perform monitoring tasks. We present the detailed architecture of the system agents and of an important domain agent, the database broker agent, that is responsible of obtaining relevant information about the clinical history of patients. We also describe how we tackle the problems of privacy, integrity and authentication during the process of exchanging information between agents.

Disseny d'un sistema HW/SW de control remot amb dispositius FPGA

El projecte que es presenta a continuació, té com a objectiu implementar un sistema HW/SW encastat en una FPGA, capaç d’executar funcions de control remot per infraroig en plataformes de televisió flexibles de Sony Corp. El disseny obtingut, s’incorporarà a un sistema més ampli de verificació i test de circuits impresos, dins del marc de producció SMD. La finalitat d’aquest projecte, és la realització d’un sistema flexible per a la implementació de comandaments de comunicació per infraroig amb circuits impresos. Prèviament, s’ha estudiat els conceptes bàsics referents a la implementació de sistemes amb FPGAs, la seva metodologia de desenvolupament i les principals característiques de la seva arquitectura. Com a especificacions, s’ha utilitzat l’estàndard de control remot per infraroig de Sony Corp SIRCS (Sony Infrared remote control system).

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Local model predictive control experiences with differential driven wheeled mobile robots

This work extends a previously developed research concerning about the use of local model predictive control in differential driven mobile robots. Hence, experimental results are presented as a way to improve the methodology by considering aspects as trajectory accuracy and time performance. In this sense, the cost function and the prediction horizon are important aspects to be considered. The aim of the present work is to test the control method by measuring trajectory tracking accuracy and time performance. Moreover, strategies for the integration with perception system and path planning are briefly introduced. In this sense, monocular image data can be used to plan safety trajectories by using goal attraction potential fields