Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Model-based objects recognition in industrial environments for autonomous vehicles control

Behavior-based navigation of autonomous vehicles requires the recognition of the navigable areas and the potential obstacles. In this paper we describe a model-based objects recognition system which is part of an image interpretation system intended to assist the navigation of autonomous vehicles that operate in industrial environments. The recognition system integrates color, shape and texture information together with the location of the vanishing point. The recognition process starts from some prior scene knowledge, that is, a generic model of the expected scene and the potential objects. The recognition system constitutes an approach where different low-level vision techniques extract a multitude of image descriptors which are then analyzed using a rule-based reasoning system to interpret the image content. This system has been implemented using a rule-based cooperative expert system

PRIM an Open Mobile Robot Platform. Motivation, Present and Future Trends

This paper is focused on the robot mobile platform PRIM (platform robot information multimedia). This robot has been made in order to cover two main needs of our group, on one hand the need for a full open mobile robotic platform that is very useful in fulfilling the teaching and research activity of our school community, and on the other hand with the idea of introducing an ethical product which would be useful as mobile multimedia information point as a service tool. This paper introduces exactly how the system is made up and explains just what the philosophy is behind this work. The navigation strategies and sensor fusion, where machine vision system is the most important one, are oriented towards goal achievement and are the key to the behaviour of the robot

Mission control system for dam inspection with an AUV

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

Disseny d’un comportament en ROS per a mantenir la posició d’un vehicle autònom submarí a partir de referències visuals

En el Centre d'Investigació en Robòtica Submarina (CIRS) de la Universitat de Gironaes disposa de diferents robots submarins els quals utilitzen una arquitectura software anomenada Component Oriented Layered-based Architecture for Autonomy ( COLA2 ), la qual ha estat desenvolupada per estudiants i professors del mateix centre. Per tal de fer aquesta arquitectura més accessible per a professors i estudiant d’altres centres la COLA2 s’està adaptant al Robot Operative System (ROS) que és un framework genèric per al desenvolupament d’aplicacions amb robots. Aquest projecte pretén dissenyar un comportament per al robot Girona500 que estigui desenvolupat dins la versió ROS de l’arquitectura COLA2. El comportament haurà de fer mantenir una determinada posició al robot amb informació visual de la càmera del robot i amb dades de navegació. La tasca de mantenir la posició es de vital importància per a poder realitzar intervencions submarines que requereixen de precisió i, precisament, el medi on es treballa no ajuda

Sistema de manteniment de l'equilibri per a un robot bípede

Dins el departament d’Electrònica, Informàtica i Automàtica de la Universitat de Girona s’han dissenyat i construït dues plataformes bípedes per a l’ús docent. La més evolucionada d’elles, finalitzada l’any 1999, està composada per dues cames d’alumini amb tres actuadors lineals cada una, simulant la funció del turmell, del genoll i del maluc. Els objectius que es pretenen aconseguir amb aquest projecte són molt concrets i tots ells estan destinats a millorar el funcionament del robot bípede. Aquests objectius són: (1) dissenyar dos graus de llibertat lineals en forma de pla XY per moure el pes que convingui per assegurar l’equilibri durant el moviment de la plataforma bípede, (2) dissenyar una placa amb una FPGA que generi senyals PWM pels vuit motors disponibles, que llegeixi els dos encoders dels motors del pla XY i que es comuniqui amb un PC equipat amb una tarja d’adquisició de dades específica, (3) dissenyar una placa de potència adequada pel control dels motors, (4) finalment realitzar un programa per comprovar el correcte funcionament de les plaques, dels actuadors i dels sensors utilitzats en la plataforma bípede

A situation analysis of the food control systems in Arab Gulf Cooperation Council (GCC) countries

Increased concerns over food safety have led to the adoption of international guidance on the key elements for national food control systems. This guidance had been used to conduct an initial assessment of the status of the food control systems in the countries belonging to the Gulf Cooperation Council. Our research has identified how the countries have been attempting to enhance their food control systems. Although the countries have different approaches to food control management, cooperation is leading to increased harmonization of legislation and food control practices. Progress is being made but there is evidence of some weakness where additional efforts may be needed. (c) 2009 Elsevier Ltd. All rights reserved.

Integrating control systems defined on the frame bundles of the space forms

This paper considers left-invariant control systems defined on the orthonormal frame bundles of simply connected manifolds of constant sectional curvature, namely the space forms Euclidean space E-3, the sphere S-3 and Hyperboloid H-3 with the corresponding frame bundles equal to the Euclidean group of motions SE(3), the rotation group SO(4) and the Lorentz group SO(1, 3). Orthonormal frame bundles of space forms coincide with their isometry groups and therefore the focus shifts to left-invariant control systems defined on Lie groups. In this paper a method for integrating these systems is given where the controls are time-independent. In the Euclidean case the elements of the Lie algebra se(3) are often referred to as twists. For constant twist motions, the corresponding curves g(t) is an element of SE(3) are known as screw motions, given in closed form by using the well known Rodrigues' formula. However, this formula is only applicable to the Euclidean case. This paper gives a method for computing the non-Euclidean screw motions in closed form. This involves decoupling the system into two lower dimensional systems using the double cover properties of Lie groups, then the lower dimensional systems are solved explicitly in closed form.

Impedance mismatch: some differences between the way humans and robots control interaction forces

In over forty years of research robots have made very little progress still largely confined to industrial manufacture and cute toys, yet in the same period computing has followed Moores Law where the capacity double roughly every two years. So why is there no Moores Law for robots? Two areas stand out as worthy of research to speedup progress. The first is to get a greater understanding of how human and animal brains control movement, the second to build a new generation of robots that have greater haptic sense, that is a better ability to adapt to the environment as it is encountered. A remarkable property of the cognitive-motor system in humans and animals is that it is slow. Recognising an object may take 250 mS, a reaction time of 150 mS is considered fast. Yet despite this slow system we are well designed to allow contact with the world in a variety of ways. We can anticipate an encounter, use the change of force as a means of communication and ignore sensory cues when they are not relevant. A better understanding of these process has allowed us to build haptic interfaces to mimic the interaction. Emerging from this understanding are new ways to control the contact between robots, the user and the environment. Rehabilitation robotics has all the elements in the subject to not only enable and change the lives of people with disabilities, but also to facilitate revolution change in classic robotics.

Remote control systems laboratory

Control systems theory can be a discipline difficult to learn without some laboratory help. With the help of focused laboratories this discipline turns to be very interesting to the students involved. The main problem is that laboratories aren't always available to students, and sometimes, when they are available, aren't big enough to a growing student population. Thus, with computer networks growing so fast, why don't create remote control labs that can be used by a large number of students? Why don't create remote control labs using Internetⓒ Copyright ?2001 IFAC Keywords: Remote Control, Computer Networks, Database, Educational Aids, Laboratory Education, Communication Control Applications.