Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Vision-based localization of an underwater robot in a structured environment

This paper presents a vision-based localization approach for an underwater robot in a structured environment. The system is based on a coded pattern placed on the bottom of a water tank and an onboard down looking camera. Main features are, absolute and map-based localization, landmark detection and tracking, and real-time computation (12.5 Hz). The proposed system provides three-dimensional position and orientation of the vehicle along with its velocity. Accuracy of the drift-free estimates is very high, allowing them to be used as feedback measures of a velocity-based low-level controller. The paper details the localization algorithm, by showing some graphical results, and the accuracy of the system

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

A system to evaluate the accuracy of a visual mosaicking methodology

When underwater vehicles navigate close to the ocean floor, computer vision techniques can be applied to obtain motion estimates. A complete system to create visual mosaics of the seabed is described in this paper. Unfortunately, the accuracy of the constructed mosaic is difficult to evaluate. The use of a laboratory setup to obtain an accurate error measurement is proposed. The system consists on a robot arm carrying a downward looking camera. A pattern formed by a white background and a matrix of black dots uniformly distributed along the surveyed scene is used to find the exact image registration parameters. When the robot executes a trajectory (simulating the motion of a submersible), an image sequence is acquired by the camera. The estimated motion computed from the encoders of the robot is refined by detecting, to subpixel accuracy, the black dots of the image sequence, and computing the 2D projective transform which relates two consecutive images. The pattern is then substituted by a poster of the sea floor and the trajectory is executed again, acquiring the image sequence used to test the accuracy of the mosaicking system

Towards a real-time vision-based navigation system for a small-class UUV

This paper deals with the problem of navigation for an unmanned underwater vehicle (UUV) through image mosaicking. It represents a first step towards a real-time vision-based navigation system for a small-class low-cost UUV. We propose a navigation system composed by: (i) an image mosaicking module which provides velocity estimates; and (ii) an extended Kalman filter based on the hydrodynamic equation of motion, previously identified for this particular UUV. The obtained system is able to estimate the position and velocity of the robot. Moreover, it is able to deal with visual occlusions that usually appear when the sea bottom does not have enough visual features to solve the correspondence problem in a certain area of the trajectory

Experimental Modeling and Control Strategies on an Open Mobile Robot Platform PRIM

This research work deals with the problem of modeling and design of low level speed controller for the mobile robot PRIM. The main objective is to develop an effective educational tool. On one hand, the interests in using the open mobile platform PRIM consist in integrating several highly related subjects to the automatic control theory in an educational context, by embracing the subjects of communications, signal processing, sensor fusion and hardware design, amongst others. On the other hand, the idea is to implement useful navigation strategies such that the robot can be served as a mobile multimedia information point. It is in this context, when navigation strategies are oriented to goal achievement, that a local model predictive control is attained. Hence, such studies are presented as a very interesting control strategy in order to develop the future capabilities of the system

PRIM an Open Mobile Robot Platform. Motivation, Present and Future Trends

This paper is focused on the robot mobile platform PRIM (platform robot information multimedia). This robot has been made in order to cover two main needs of our group, on one hand the need for a full open mobile robotic platform that is very useful in fulfilling the teaching and research activity of our school community, and on the other hand with the idea of introducing an ethical product which would be useful as mobile multimedia information point as a service tool. This paper introduces exactly how the system is made up and explains just what the philosophy is behind this work. The navigation strategies and sensor fusion, where machine vision system is the most important one, are oriented towards goal achievement and are the key to the behaviour of the robot

Integració de la posició GPS per fer el control del moviment d'un robot

L’objectiu d’aquest treball és fer un estudi dels diferents tipus de sistemes de posicionament global que hi ha en el mercat, elegir un mòdul receptor assequible per poder analitzar-lo i veure si disposa de les característiques adequades per integrar-lo en un robot autònom d’exploració del projecte Sant Bernardo. S’hauran de fer les anàlisis de la precisió del mòdul en les diferents direccions cardinals, es a dir Nord, Sud, Est, Oest i altura i veure la diferència d’error que hi ha en cada una, veure si la precisió varia molt en diferents situacions, com en cel obert, en sotabosc, costat i interior edificis. A més a més s’haurà de mirar la repetibilitat, la diferencia d’error amb diferentnombre de satèl•lits connectats i si disposa de suficient velocitat de processat per a podar corregir la posició del robot en moviment. Un cop analitzades les característiques del mòdul receptor elegit, es decidirà si aquest ésadequat per fer les correccions de posició del robot, o s’haurà d’adquirir un mòdul de característiques superiors i per tant molt més car per a dura a terme adequadament la correcció de la posició

Disseny i implementació d'un robot amb connexió WiFi

In the present work the prototype for a self-propelled embedded system with wireless connectivity has been carried out. The system is designed to receive commands from the Internet.

Vehículos-robot de guiado automático para almacenes y plantas de manufacturación

En este proyecto de final de carrera se realiza la gestión del tráfico de AGV y la simulación de su comportamiento al circular por una planta de estudio. Con la simulación se puede ver como varía el comportamiento de la planta al modificar el número de AGV y la velocidad a la que circulan. La planta objeto de estudio es un laboratorio de análisis clínico en el que se ha sustituido el sistema de transporte interno basado en cintas por uno con AGV, con lo que se ha podido comprobar que dicha sustitución es factible.

Spacial debris collector robot : An AI-based and autonomous debris collector satellite simulation

In this paper the core functions of an artificial intelligence (AI) for controlling a debris collector robot are designed and implemented. Using the robot operating system (ROS) as the base of this work a multi-agent system is built with abilities for task planning.

Totally implantable robot to treat chronic atrial fibrillation

Chronic atrial fibrillation affects millions of people worldwide. Its surgical treatment often fails to restore the transport function of the atrium. This study first introduces the concept of an atrial assist device (AAD) to restore the pump function of the atrium. The AAD is developed to be totally implantable in the human body with a transcutaneous energy transfer system to recharge the implanted battery. The ADD consists of a motorless pump based on artificial muscle technology, positioned on the external surface of the atrium to compress it and restore its muscular activity. A bench model reproduces the function of a fibrillating atrium to assess the circulatory support that this pump can provide. Atripump (Nanopowers SA, Switzerland) is a dome-shaped silicone-coated nitinol actuator 5 mm high, sutured on the external surface of the atrium. A pacemaker-like control unit drives the actuator that compresses the atrium, providing the mechanical support to the blood circulation. Electrical characteristics: the system is composed of one actuator that needs a minimal tension of 15 V and has a maximum current of 1.5 A with a 50% duty cycle. The implantable rechargeable battery is made of a cell having the following specifications: nominal tension of a cell: 4.1 V, tension after 90% of discharge: 3.5 V, nominal capacity of a cell: 163 mA h. The bench model consists of an open circuit made of latex bladder 60 mm in diameter filled with water. The bladder is connected to a vertically positioned tube that is filled to different levels, reproducing changes in cardiac preload. The Atripump is placed on the outer surface of the bladder. Pressure, volume and temperature changes were recorded. The contraction rate was 1 Hz with a power supply of 12 V, 400 mA for 200 ms. Preload ranged from 15 to 21 cm H(2)O. Maximal silicone membrane temperature was 55 degrees C and maximal temperature of the liquid environment was 35 degrees C. The pump produced a maximal work of 16 x 10(-3) J. Maximal volume pumped was 492 ml min(-1). This artificial muscle pump is compact, follows the Starling law and reproduces the hemodynamic performances of a normal atrium. It could represent a new tool to restore the atrial kick in persistent atrial fibrillation.

Mobile Robot Local Predictive Control under Perception Constraints

This research extends a previously developed work concerning about the use of local model predictive control in mobile robots. Hence, experimental results are presented as a way to improve the methodology by considering aspects as trajectory accuracy and time performance. In this sense, the cost function and the prediction horizon are important aspects to be considered. The platformused is a differential driven robot with a free rotating wheel. The aim of the present work is to test the control method by measuring trajectory tracking accuracy and time performance. Moreover, strategies for the integration with perception system and path planning are also introduced. In this sense, monocular image data provide an occupancy grid where safety trajectories are computed by using goal attraction potential fields

Mobile robot experimental modeling and control strategies using sensor fusion

This research work deals with the problem of modeling and design of low level speed controller for the mobile robot PRIM. The main objective is to develop an effective educational, and research tool. On one hand, the interests in using the open mobile platform PRIM consist in integrating several highly related subjects to the automatic control theory in an educational context, by embracing the subjects of communications, signal processing, sensor fusion and hardware design, amongst others. On the other hand, the idea is to implement useful navigation strategies such that the robot can be served as a mobile multimedia information point. It is in this context, when navigation strategies are oriented to goal achievement, that a local model predictive control is attained. Hence, such studies are presented as a very interesting control strategy in order to develop the future capabilities of the system. In this context the research developed includes the visual information as a meaningful source that allows detecting the obstacle position coordinates as well as planning the free obstacle trajectory that should be reached by the robot

Creació d’un model de simulació pels propulsors del robot submarí Girona 500 mitjançant CFD

S’arriba a un acord entre el grup de recerca GREFEMA i ViCOROB per estudiar els propulsors de palesutilitzats fins a l’actualitat en el robot submarí Girona 500, de forma que el model creatserveixi d’eina per apoder estudiar qualsevol tipus de propulsor que es vulgui fer servir.Es crearà un model de simulació amb CFD d’ANSYS per tal de poder recrear qualsevol situació ambqualsevol model de propulsor que es vulgui emprar, estalviant en costos de compra o fabricació, a mésd’evitar un muntatge experimental que pot no ser del tot fiable.A partir de geometries de propulsors de pales comercials existents es realitzarà una simulació amb elprograma de dinàmica de fluids computacional (CFD) d’ANSYS.La informació proporcionada per l’eina de simulació es compararan amb els resultats obtinguts de formaempírica a les instal•lacions del Parc Científic i Tecnològic de la Universitat de Girona i amb el model teòric.D’aquesta forma, es comprovarà la bondat de la simulació i es validarà el model numèric utilitzat