A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Model-based objects recognition in industrial environments for autonomous vehicles control

Behavior-based navigation of autonomous vehicles requires the recognition of the navigable areas and the potential obstacles. In this paper we describe a model-based objects recognition system which is part of an image interpretation system intended to assist the navigation of autonomous vehicles that operate in industrial environments. The recognition system integrates color, shape and texture information together with the location of the vanishing point. The recognition process starts from some prior scene knowledge, that is, a generic model of the expected scene and the potential objects. The recognition system constitutes an approach where different low-level vision techniques extract a multitude of image descriptors which are then analyzed using a rule-based reasoning system to interpret the image content. This system has been implemented using a rule-based cooperative expert system

Local model predictive control experiences with differential driven wheeled mobile robots

This work extends a previously developed research concerning about the use of local model predictive control in differential driven mobile robots. Hence, experimental results are presented as a way to improve the methodology by considering aspects as trajectory accuracy and time performance. In this sense, the cost function and the prediction horizon are important aspects to be considered. The aim of the present work is to test the control method by measuring trajectory tracking accuracy and time performance. Moreover, strategies for the integration with perception system and path planning are briefly introduced. In this sense, monocular image data can be used to plan safety trajectories by using goal attraction potential fields

Environment mapping for mobile robots navigation using hierarchical neural network and omnivision

Autonomous robots must be able to learn and maintain models of their environments. In this context, the present work considers techniques for the classification and extraction of features from images in joined with artificial neural networks in order to use them in the system of mapping and localization of the mobile robot of Laboratory of Automation and Evolutive Computer (LACE). To do this, the robot uses a sensorial system composed for ultrasound sensors and a catadioptric vision system formed by a camera and a conical mirror. The mapping system is composed by three modules. Two of them will be presented in this paper: the classifier and the characterizer module. The first module uses a hierarchical neural network to do the classification; the second uses techiniques of extraction of attributes of images and recognition of invariant patterns extracted from the places images set. The neural network of the classifier module is structured in two layers, reason and intuition, and is trained to classify each place explored for the robot amongst four predefine classes. The final result of the exploration is the construction of a topological map of the explored environment. Results gotten through the simulation of the both modules of the mapping system will be presented in this paper. © 2008 IEEE.

Distributed orientation agreement in a group of robots

In this article, a method for the agreement of a set of robots on a common reference orientation based on a distributed consensus algorithm is described. It only needs that robots detect the relative positions of their neighbors and communicate with them. Two different consensus algorithms based on the exchange of information are proposed, tested and analyzed. Systematic experiments were carried out in simulation and with real robots in order to test the method. Experimental results show that the robots are able to agree on the reference orientation under certain conditions. Scalability with an increasing number of robots was tested successfully in simulation with up to 49 robots. Experiments with real robots succeeded proving that the proposed method works in reality.

Distributed Orientation Agreement in a Group of Robots

In this article, a method for the agreement of a set of robots on a common reference orientation based on a distributed consensus algorithm is described. It only needs that robots detect the relative positions of their neighbors and communicate with them. Two different consensus algorithms based on the exchange of information are proposed, tested and analyzed. Systematic experiments were carried out in simulation and with real robots in order to test the method. Experimental results show that the robots are able to agree on the reference orientation under certain conditions. Scalability with an increasing number of robots was tested successfully in simulation with up to 49 robots. Experiments with real robots succeeded proving that the proposed method works in reality.

Control por modos deslizantes como estrategia de navegación en una flota de robots

En esta tesis se presenta el desarrollo de un esquema de cooperación entre vehículos terrestres (UGV) y aéreos (UAV) no tripulados, que sirve de base para conformar dos flotas de robots autónomos (denominadas FRACTAL y RoMA). Con el fin de comprobar, en diferentes escenarios y con diferente tareas, la validez de las estrategias de coordinación y cooperación propuestas en la tesis se utilizan los robots de la flota FRACTAL, que sirven como plataforma de prueba para tareas como el uso de vehículos aéreos y terrestres para apoyar labores de búsqueda y rescate en zonas de emergencia y la cooperación de una flota de robots para labores agrícolas. Se demuestra además, que el uso de la técnica de control no lineal conocida como Control por Modos Deslizantes puede ser aplicada no solo para conseguir la navegación autónoma individual de un robot aéreo o terrestre, sino también en tareas que requieren la navegación coordinada y sin colisiones de varios robots en un ambiente compartido. Para esto, se conceptualiza teóricamente el uso de la técnica de Control por Modos Deslizantes como estrategia de coordinación entre robots, extendiendo su aplicación a robots no-holonómicos en R2 y a robots aéreos en el espacio tridimensional. Después de dicha contextualización teórica, se analizan las condiciones necesarias para determinar la estabilidad del sistema multi-robot controlado y, finalmente, se comprueban las características de estabilidad y robustez ofrecidas por esta técnica de control. Tales comprobaciones se hacen simulando la navegación segura y eficiente de un grupo de UGVs para la detección de posibles riesgos ambientales, aprovechando la información aportada por un UAV. Para estas simulaciones se utilizan los modelos matemáticos de robots de la flota RoMA. Estas tareas coordinadas entre los robots se hacen posibles gracias a la efectividad, estabilidad y robustez de las estrategias de control que se desarrollan como núcleo fundamental de este trabajo de investigación. ABSTRACT This thesis presents the development of a cooperation scheme between unmanned ground (UGV) and aerial (UAV) vehicles. This scheme is the basis for forming two fleets of autonomous robots (called FRACTAL and RoMA). In order to assess, in different settings and on different tasks, the validity of the coordination and cooperation strategies proposed in the thesis, the FRACTAL fleet robots serves as a test bed for tasks like using coordinated aerial and ground vehicles to support search and rescue work in emergency scenarios or cooperation of a fleet of robots for agriculture. It is also shown that using the technique of nonlinear control known as Sliding Modes Control (SMC) can be applied not only for individual autonomous navigation of an aircraft or land robot, but also in tasks requiring the coordinated navigation of several robots, without collisions, in a shared environment. To this purpose, a strategy of coordination between robots using Sliding Mode Control technique is theoretically conceptualized, extending its application to non-holonomic robots in R2 and aerial robots in three-dimensional space. After this theoretical contextualization, the stability conditions of multi-robot system are analyzed, and finally, the stability and robustness characteristics are validated. Such validations are made with simulated experiments about the safe and efficient navigation of a group of UGV for the detection of possible environmental hazards, taking advantage of the information provided by a UAV. This simulations are made using mathematical models of RoMA fleet robots. These coordinated tasks of robots fleet are made possible thanks to the effectiveness, stability and robustness of the control strategies developed as core of this research.

A grammatical approach to the modeling of an autonomous robot

Virtual Worlds Generator is a grammatical model that is proposed to define virtual worlds. It integrates the diversity of sensors and interaction devices, multimodality and a virtual simulation system. Its grammar allows the definition and abstraction in symbols strings of the scenes of the virtual world, independently of the hardware that is used to represent the world or to interact with it. A case study is presented to explain how to use the proposed model to formalize a robot navigation system with multimodal perception and a hybrid control scheme of the robot. The result is an instance of the model grammar that implements the robotic system and is independent of the sensing devices used for perception and interaction. As a conclusion the Virtual Worlds Generator adds value in the simulation of virtual worlds since the definition can be done formally and independently of the peculiarities of the supporting devices.

La responsabilité civile dans les cas de dommages causés par les robots d’assistance au Québec

Le développement exponentiel de la technologie et le vieillissement de la population permettent d’introduire dans notre quotidien les robots d’assistance. La coexistence de l’homme avec ces robots intelligents et autonomes soulève une question fondamentale: dans l’éventualité où un robot provoquerait un accident causant un dommage à une personne ou à un bien qui serait le responsable? Aucune loi ne réglemente les activités de la robotique d’assistance dans le monde. Cette étude vise l’analyse de l’applicabilité des régimes de responsabilité du Code Civil du Québec aux cas de dommages causés par le robot d’assistance. L’analyse des régimes de responsabilité du Code civil du Québec permet de constater que deux régimes de responsabilité sont susceptibles d’être appliqués aux cas spécifiques de dommages causés par le robot d’assistance: le régime de responsabilité du fait des biens, énoncé à l’article 1465 C.c.Q., et le régime de responsabilité du fait des fabricants et vendeurs spécialisés, énoncé à l’article 1468 C.c.Q. Cela s’explique par la présence de critères et de conditions de mise en œuvre des régimes qui sont transposables aux différents aspects concernant la fabrication et l’utilisation du robot d’assistance.

La responsabilité civile dans les cas de dommages causés par les robots d’assistance au Québec

Le développement exponentiel de la technologie et le vieillissement de la population permettent d’introduire dans notre quotidien les robots d’assistance. La coexistence de l’homme avec ces robots intelligents et autonomes soulève une question fondamentale: dans l’éventualité où un robot provoquerait un accident causant un dommage à une personne ou à un bien qui serait le responsable? Aucune loi ne réglemente les activités de la robotique d’assistance dans le monde. Cette étude vise l’analyse de l’applicabilité des régimes de responsabilité du Code Civil du Québec aux cas de dommages causés par le robot d’assistance. L’analyse des régimes de responsabilité du Code civil du Québec permet de constater que deux régimes de responsabilité sont susceptibles d’être appliqués aux cas spécifiques de dommages causés par le robot d’assistance: le régime de responsabilité du fait des biens, énoncé à l’article 1465 C.c.Q., et le régime de responsabilité du fait des fabricants et vendeurs spécialisés, énoncé à l’article 1468 C.c.Q. Cela s’explique par la présence de critères et de conditions de mise en œuvre des régimes qui sont transposables aux différents aspects concernant la fabrication et l’utilisation du robot d’assistance.

Um algoritmo PSO híbrido para planejamento de caminhos em navegação de veículos utilizando A*

Utilizar robôs autônomos capazes de planejar o seu caminho é um desafio que atrai vários pesquisadores na área de navegação de robôs. Neste contexto, este trabalho tem como objetivo implementar um algoritmo PSO híbrido para o planejamento de caminhos em ambientes estáticos para veículos holonômicos e não holonômicos. O algoritmo proposto possui duas fases: a primeira utiliza o algoritmo A* para encontrar uma trajetória inicial viável que o algoritmo PSO otimiza na segunda fase. Por fim, uma fase de pós planejamento pode ser aplicada no caminho a fim de adaptá-lo às restrições cinemáticas do veículo não holonômico. O modelo Ackerman foi considerado para os experimentos. O ambiente de simulação de robótica CARMEN (Carnegie Mellon Robot Navigation Toolkit) foi utilizado para realização de todos os experimentos computacionais considerando cinco instâncias de mapas geradas artificialmente com obstáculos. O desempenho do algoritmo desenvolvido, A*PSO, foi comparado com os algoritmos A*, PSO convencional e A* Estado Híbrido. A análise dos resultados indicou que o algoritmo A*PSO híbrido desenvolvido superou em qualidade de solução o PSO convencional. Apesar de ter encontrado melhores soluções em 40% das instâncias quando comparado com o A*, o A*PSO apresentou trajetórias com menos pontos de guinada. Investigando os resultados obtidos para o modelo não holonômico, o A*PSO obteve caminhos maiores entretanto mais suaves e seguros.

Simulação e controlo de veículos autónomos terrestres

A implementação e venda de robôs autónomos tem sido um sector que nos últimos anos tem adquirido cada vez mais quota no mercado, nomeadamente no sector militar, agrícola e da vigilância. Como tal, tem sido também de grande importância a capacidade de implementar e testar robôs por parte das entidades que os fabricam. Uma das formas que tem garantido o sucesso do desenvolvimento de robôs é a simulação prévia dos mesmos antes que estes passem a fase de produção. Sendo assim, o LSA como entidade de desenvolvimento de robôs autónomos, tem necessidade de adquirir um sistema que simule os robôs em desenvolvimento. O trabalho desta tese consiste na realização de um sistema que simule robôs autónomos terrestres de forma que se possa observar o comportamento da cinemática, dinânica e hardware dos robôs em ambiente 3D. Esta aplicação de simulação pode mais tarde ser utilizada pelo laboratório para testar missões, validar alterações de estrutura, sensores, etc. Para além disso, com recurso ao simulador Player/Stage/Gazebo testar o robô LINCE e implementar algoritmos de controlo para o mesmo. Os algoritmos de controlo implementados baseiam-se em primitivas de controlo básico para serem utilizadas pelo sistema de navegação e gerar trajectórias complexas. Os algoritmos desenvolvidos nesta tese baseiam-se nas equações cinemáticas do veículo estudado. Estes algoritmos depois de testados no simulador, poderão ser colocados no Hardware do robô. Desta forma consegue-se desenvolver algoritmos para determinado robô sem que este esteja operacional.

Hybrid Maneuver for Gradient Search with Multiple Coordinated AUVs

This work presents a hybrid maneuver for gradient search with multiple AUV's. The mission consists in following a gradient field in order to locate the source of a hydrothermal vent or underwater freshwater source. The formation gradient search exploits the environment structuring by the phenomena to be studied. The ingredients for coordination are the payload data collected by each vehicle and their knowledge of the behaviour of other vehicles and detected formation distortions.