A flexible navigation system for autonomous robots operating in heterogeneous environments

Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e de Computadores

Programación de un robot autónomo: diseño, construcción, programación e integración de un robot autónomo en el entorno Pyro

En aquest projecte, s'ha dissenyat, construït i programat un robot autònom, dotat de sistema de locomoció i sensors que li permeten navegar sense impactar en un entorn controlat. Per assolir aquests objectius s'ha dissenyat i programat una unitat de control que gestiona el hardware de baix volum de dades amb diferents modes d'operació, abstraient-lo en una única interfície. Posteriorment s'ha integrat aquest sistema en l'entorn de robòtica Pyro. Aquest entorn permet usar i adaptar, segons es necessiti, eines d'intel·ligència artificial ja desenvolupades.

Anàlisi, disseny i construcció d'un robot educatiu basat en microcontrolador AVR-ATMEGA

Aquest projecte presenta el disseny, construcció i programació d’un robot autònom, com a base per una proposta educativa. Per aconseguir aquest objectiu s’ha dotat el robot d’una unitat de procés, un sistema de locomoció i un seguit de sensors que proporcionaran a la unitat informació respecte l’entorn. Per gestionar totes aquestes funcionalitats, s’ha fet servir un sistema operatiu en temps real capaç de gestionar amb efectivitat les tasques que puguin ser executades pel robot. Finalment, s’ha exposat una detallada descripció dels costos per una producció de volum mig i de caire merament educatiu.

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Development of a Control System for a Tele Operated Mobile Robot

Mobile robots are capable of performing spatial displacement motions in different environments. This motions can be calculated based on sensorial data (autonomous robot) or given by an operator (tele operated robot). This thesis is focused on the latter providing the control architecture which bridges the tele operator and the robot’s locomotion system and end effectors. Such a task might prove overwhelming in cases where the robot comprises a wide variety of sensors and actuators hence a relatively new option was selected: Robot Operating System (ROS). The control system of a new robot will be sketched and tested in a simulation model using ROS together with Gazebo in order to determine the viability of such a system. The simulated model will be based on the projected shape and main features of the real machine. A stability analysis will be performed first theoretically and afterwards using the developed model. This thesis concluded that both the physical properties and the control architecture are feasible and stable settling up the ground for further work with the same robot.

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Machine learning and small robot navigation

Developing successful navigation and mapping strategies is an essential part of autonomous robot research. However, hardware limitations often make for inaccurate systems. This project serves to investigate efficient alternatives to mapping an environment, by first creating a mobile robot, and then applying machine learning to the robot and controlling systems to increase the robustness of the robot system. My mapping system consists of a semi-autonomous robot drone in communication with a stationary Linux computer system. There are learning systems running on both the robot and the more powerful Linux system. The first stage of this project was devoted to designing and building an inexpensive robot. Utilizing my prior experience from independent studies in robotics, I designed a small mobile robot that was well suited for simple navigation and mapping research. When the major components of the robot base were designed, I began to implement my design. This involved physically constructing the base of the robot, as well as researching and acquiring components such as sensors. Implementing the more complex sensors became a time-consuming task, involving much research and assistance from a variety of sources. A concurrent stage of the project involved researching and experimenting with different types of machine learning systems. I finally settled on using neural networks as the machine learning system to incorporate into my project. Neural nets can be thought of as a structure of interconnected nodes, through which information filters. The type of neural net that I chose to use is a type that requires a known set of data that serves to train the net to produce the desired output. Neural nets are particularly well suited for use with robotic systems as they can handle cases that lie at the extreme edges of the training set, such as may be produced by "noisy" sensor data. Through experimenting with available neural net code, I became familiar with the code and its function, and modified it to be more generic and reusable for multiple applications of neural nets.

Petri nets and autonomous systems: A case study

This work presents the Petri net-based modeling of an autonomous robot's navigation system used for the application of supplies in agriculture. The model was developed theoretically and implemented through the CPNTools software. It simulates the behavior of the robot, capturing environmental characteristics by means of sensors, making appropriate decisions, and forwarding them to the corresponding actuators. By exciting the model using CPNTools it is possible to simulate situations that the robot might undergo, without the need to expose it to real potentially dangerous situations. ©2009 IEEE.

VladBot. Robot para posicionamiento de medida

VladBot es un robot autónomo diseñado para posicionar en interiores un micrófono de medida. Este prototipo puede valorar la idea de automatizar medidas acústicas en interiores mediante un robot autónomo. Posee dos ruedas motrices y una rueda loca. Ésta rueda loca aporta maniobrabilidad al robot. Un soporte extensible hecho de aluminio sostiene el micrófono de medida. VladBot ha sido diseñado con tecnologías de bajo coste y bajo una plataforma abierta, Arduino. Arduino es una plataforma electrónica libre. Esto quiere decir que los usuarios tienen libre acceso a toda la información referente a los micro-controladores (hardware) y referente al software. Ofrece un IDE (Integrated Development Environment, en español, Entorno de Desarrollo Integrado) de forma gratuita y con un sencillo lenguaje de programación, con el que se pueden realizar proyectos de cualquier tipo. Además, los usuarios disponen de un foro donde encontrar ayuda, “Arduino Forum”. VladBot se comunica con el usuario a través de Bluetooth, creando un enlace fiable y con un alcance suficiente (aproximadamente 100 metros) para que controlar a VladBot desde una sala contigua. Hoy en día, Bluetooth es una tecnología implantada en casi todos los ordenadores, por lo que no necesario ningún sistema adicional para crear dicho enlace. Esta comunicación utiliza un protocolo de comunicaciones, JSON (JavaScript Object Notation). JSON hace que la comunicación sea más fiable, ya que sólo un tipo de mensajes preestablecidos son reconocidos. Gracias a este protocolo es posible la comunicación con otro software, permitiendo crear itinerarios en otro programa externo. El diseño de VladBot favorece su evolución hasta un sistema más preciso ya que el usuario puede realizar modificaciones en el robot. El código que se proporciona puede ser modificado, aumentando las funcionalidades de VladBot o mejorándolas. Sus componentes pueden ser cambiados también (incluso añadir nuevos dispositivos) para aumentar sus capacidades. Vladbot es por tanto, un sistema de transporte (de bajo coste) para un micrófono de medida que se puede comunicar inalámbricamente con el usuario de manera fiable. ABSTRACT. VladBot is an autonomous robot designed to indoor positioning of a measurement microphone. This prototype can value the idea of making automatic acoustic measurements indoor with an autonomous robot. It has two drive wheels and a caster ball. This caster ball provides manoeuvrability to the robot. An extendible stand made in aluminium holds the measurement microphone. VladBot has been designed with low cost technologies and under an open-source platform, Arduino. Arduino is a freeFsource electronics platform. This means that users have free access to all the information about micro-controllers (hardware) and about the software. Arduino offers a free IDE (Integrated Development Environment) with an easy programming language, which any kind of project can be made with. Besides, users have a forum where find help, “Arduino Forum”. VladBot communicates with the user by Bluetooth, creating a reliable link with enough range (100 meters approximately) for controlling VladBot in the next room. Nowadays, Bluetooth is a technology embedded in almost laptops, so it is not necessary any additional system for create this link. This communication uses a communication protocol, JSON (JavaScript Object Notation). JSON makes the communication more reliable, since only a preFestablished kind of messages are recognised. Thanks to this protocol is possible the communication with another software, allowing to create routes in an external program. VladBot´s design favours its evolution to an accurate system since the user can make modifications in the robot. The code given can be changed, increasing VladBot´s uses or improving these uses. Their components can be changed too (even new devices can be added) for increasing its abilities. So, VladBot is a (low cost) transport system for a measurement microphone, which can communicate with the user in a reliable way.

A modified particle filter for simultaneous robot localization and landmark tracking in an indoor environment

This paper presents the implementation of a modified particle filter for vision-based simultaneous localization and mapping of an autonomous robot in a structured indoor environment. Through this method, artificial landmarks such as multi-coloured cylinders can be tracked with a camera mounted on the robot, and the position of the robot can be estimated at the same time. Experimental results in simulation and in real environments show that this approach has advantages over the extended Kalman filter with ambiguous data association and various levels of odometric noise.

Towards a Mobile Three-dimensional Modelling System for Underground Structures

Robotica 2012: 12th International Conference on Autonomous Robot Systems and Competitions April 11, 2012, Guimarães, Portugal