Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Model-based objects recognition in industrial environments for autonomous vehicles control

Behavior-based navigation of autonomous vehicles requires the recognition of the navigable areas and the potential obstacles. In this paper we describe a model-based objects recognition system which is part of an image interpretation system intended to assist the navigation of autonomous vehicles that operate in industrial environments. The recognition system integrates color, shape and texture information together with the location of the vanishing point. The recognition process starts from some prior scene knowledge, that is, a generic model of the expected scene and the potential objects. The recognition system constitutes an approach where different low-level vision techniques extract a multitude of image descriptors which are then analyzed using a rule-based reasoning system to interpret the image content. This system has been implemented using a rule-based cooperative expert system

Local model predictive control experiences with differential driven wheeled mobile robots

This work extends a previously developed research concerning about the use of local model predictive control in differential driven mobile robots. Hence, experimental results are presented as a way to improve the methodology by considering aspects as trajectory accuracy and time performance. In this sense, the cost function and the prediction horizon are important aspects to be considered. The aim of the present work is to test the control method by measuring trajectory tracking accuracy and time performance. Moreover, strategies for the integration with perception system and path planning are briefly introduced. In this sense, monocular image data can be used to plan safety trajectories by using goal attraction potential fields

Autonomous underwater vehicles in a hazardous non-deterministic environment

This research investigated the cooperation of multi underwater robots to perform a task. This combined engineering design, electronics and consensus control to create systems capable of achieving the task. Challenges such as underwater radio communications were researched and a simulation framework was created and tested on virtual and real systems.

Environment mapping for mobile robots navigation using hierarchical neural network and omnivision

Autonomous robots must be able to learn and maintain models of their environments. In this context, the present work considers techniques for the classification and extraction of features from images in joined with artificial neural networks in order to use them in the system of mapping and localization of the mobile robot of Laboratory of Automation and Evolutive Computer (LACE). To do this, the robot uses a sensorial system composed for ultrasound sensors and a catadioptric vision system formed by a camera and a conical mirror. The mapping system is composed by three modules. Two of them will be presented in this paper: the classifier and the characterizer module. The first module uses a hierarchical neural network to do the classification; the second uses techiniques of extraction of attributes of images and recognition of invariant patterns extracted from the places images set. The neural network of the classifier module is structured in two layers, reason and intuition, and is trained to classify each place explored for the robot amongst four predefine classes. The final result of the exploration is the construction of a topological map of the explored environment. Results gotten through the simulation of the both modules of the mapping system will be presented in this paper. © 2008 IEEE.

Distributed orientation agreement in a group of robots

In this article, a method for the agreement of a set of robots on a common reference orientation based on a distributed consensus algorithm is described. It only needs that robots detect the relative positions of their neighbors and communicate with them. Two different consensus algorithms based on the exchange of information are proposed, tested and analyzed. Systematic experiments were carried out in simulation and with real robots in order to test the method. Experimental results show that the robots are able to agree on the reference orientation under certain conditions. Scalability with an increasing number of robots was tested successfully in simulation with up to 49 robots. Experiments with real robots succeeded proving that the proposed method works in reality.

Distributed Orientation Agreement in a Group of Robots

In this article, a method for the agreement of a set of robots on a common reference orientation based on a distributed consensus algorithm is described. It only needs that robots detect the relative positions of their neighbors and communicate with them. Two different consensus algorithms based on the exchange of information are proposed, tested and analyzed. Systematic experiments were carried out in simulation and with real robots in order to test the method. Experimental results show that the robots are able to agree on the reference orientation under certain conditions. Scalability with an increasing number of robots was tested successfully in simulation with up to 49 robots. Experiments with real robots succeeded proving that the proposed method works in reality.

Control por modos deslizantes como estrategia de navegación en una flota de robots

En esta tesis se presenta el desarrollo de un esquema de cooperación entre vehículos terrestres (UGV) y aéreos (UAV) no tripulados, que sirve de base para conformar dos flotas de robots autónomos (denominadas FRACTAL y RoMA). Con el fin de comprobar, en diferentes escenarios y con diferente tareas, la validez de las estrategias de coordinación y cooperación propuestas en la tesis se utilizan los robots de la flota FRACTAL, que sirven como plataforma de prueba para tareas como el uso de vehículos aéreos y terrestres para apoyar labores de búsqueda y rescate en zonas de emergencia y la cooperación de una flota de robots para labores agrícolas. Se demuestra además, que el uso de la técnica de control no lineal conocida como Control por Modos Deslizantes puede ser aplicada no solo para conseguir la navegación autónoma individual de un robot aéreo o terrestre, sino también en tareas que requieren la navegación coordinada y sin colisiones de varios robots en un ambiente compartido. Para esto, se conceptualiza teóricamente el uso de la técnica de Control por Modos Deslizantes como estrategia de coordinación entre robots, extendiendo su aplicación a robots no-holonómicos en R2 y a robots aéreos en el espacio tridimensional. Después de dicha contextualización teórica, se analizan las condiciones necesarias para determinar la estabilidad del sistema multi-robot controlado y, finalmente, se comprueban las características de estabilidad y robustez ofrecidas por esta técnica de control. Tales comprobaciones se hacen simulando la navegación segura y eficiente de un grupo de UGVs para la detección de posibles riesgos ambientales, aprovechando la información aportada por un UAV. Para estas simulaciones se utilizan los modelos matemáticos de robots de la flota RoMA. Estas tareas coordinadas entre los robots se hacen posibles gracias a la efectividad, estabilidad y robustez de las estrategias de control que se desarrollan como núcleo fundamental de este trabajo de investigación. ABSTRACT This thesis presents the development of a cooperation scheme between unmanned ground (UGV) and aerial (UAV) vehicles. This scheme is the basis for forming two fleets of autonomous robots (called FRACTAL and RoMA). In order to assess, in different settings and on different tasks, the validity of the coordination and cooperation strategies proposed in the thesis, the FRACTAL fleet robots serves as a test bed for tasks like using coordinated aerial and ground vehicles to support search and rescue work in emergency scenarios or cooperation of a fleet of robots for agriculture. It is also shown that using the technique of nonlinear control known as Sliding Modes Control (SMC) can be applied not only for individual autonomous navigation of an aircraft or land robot, but also in tasks requiring the coordinated navigation of several robots, without collisions, in a shared environment. To this purpose, a strategy of coordination between robots using Sliding Mode Control technique is theoretically conceptualized, extending its application to non-holonomic robots in R2 and aerial robots in three-dimensional space. After this theoretical contextualization, the stability conditions of multi-robot system are analyzed, and finally, the stability and robustness characteristics are validated. Such validations are made with simulated experiments about the safe and efficient navigation of a group of UGV for the detection of possible environmental hazards, taking advantage of the information provided by a UAV. This simulations are made using mathematical models of RoMA fleet robots. These coordinated tasks of robots fleet are made possible thanks to the effectiveness, stability and robustness of the control strategies developed as core of this research.

A grammatical approach to the modeling of an autonomous robot

Virtual Worlds Generator is a grammatical model that is proposed to define virtual worlds. It integrates the diversity of sensors and interaction devices, multimodality and a virtual simulation system. Its grammar allows the definition and abstraction in symbols strings of the scenes of the virtual world, independently of the hardware that is used to represent the world or to interact with it. A case study is presented to explain how to use the proposed model to formalize a robot navigation system with multimodal perception and a hybrid control scheme of the robot. The result is an instance of the model grammar that implements the robotic system and is independent of the sensing devices used for perception and interaction. As a conclusion the Virtual Worlds Generator adds value in the simulation of virtual worlds since the definition can be done formally and independently of the peculiarities of the supporting devices.

Dynamic virtual workspace for teleoperation of heterogeneous robots in non-deterministic environments

Robots are ever increasing in a variety of different workplaces providing an array of benefits such alternative solutions to traditional human labor. While developing fully autonomous robots is the ultimate goal in many robotic applications the reality is that there still exist many situationswere robots require some level of teleoperation in order to achieve assigned goals especially when deployed in non-deterministic environments. For instance teleoperation is commonly used in areas such as search and rescue, bomb disposal and exploration of inaccessible or harsh terrain. This is due to a range of factors such as the lack of ability for robots to quickly and reliably navigate unknown environments or provide high-level decision making especially intime critical tasks. To provide an adequate solution for such situations human-in-the-loop control is required. When developing human-in-the-loop control it is important to take advantage of the complimentary skill-sets that both humans and robots share. For example robots can performrapid calculations, provide accurate measurements through hardware such as sensors and store large amounts of data while humans provide experience, intuition, risk management and complex decision making capabilities. Shared autonomy is the concept of building robotic systems that take advantage of these complementary skills-sets to provide a robust an efficient robotic solution. While the requirement of human-in-the-loop control exists Human Machine Interaction (HMI) remains an important research topic especially the area of User Interface (UI) design.In order to provide operators with an effective teleoperation system it is important that the interface is intuitive and dynamic while also achieving a high level of immersion. Recent advancements in virtual and augmented reality hardware is giving rise to innovative HMI systems. Interactive hardware such as Microsoft Kinect, leap motion, Oculus Rift, Samsung Gear VR and even CAVE Automatic Virtual Environments [1] are providing vast improvements over traditional user interface designs such as the experimental web browser JanusVR [2]. This combined with the introduction of standardized robot frameworks such as ROS and Webots [3] that now support a large number of different robots provides an opportunity to develop a universal UI for teleoperation control to improve operator efficiency while reducing teleoperation training.This research introduces the concept of a dynamic virtual workspace for teleoperation of heterogeneous robots in non-deterministic environments that require human-in-the-loop control. The system first identifies the connected robots through the use kinematic information then determines its network capabilities such as latency and bandwidth. Given the robot type and network capabilities the system can then provide the operator with available teleoperation modes such as pick and place control or waypoint navigation while also allowing them to manipulate the virtual workspace layout to provide information from onboard camera’s or sensors.

La responsabilité civile dans les cas de dommages causés par les robots d’assistance au Québec

Le développement exponentiel de la technologie et le vieillissement de la population permettent d’introduire dans notre quotidien les robots d’assistance. La coexistence de l’homme avec ces robots intelligents et autonomes soulève une question fondamentale: dans l’éventualité où un robot provoquerait un accident causant un dommage à une personne ou à un bien qui serait le responsable? Aucune loi ne réglemente les activités de la robotique d’assistance dans le monde. Cette étude vise l’analyse de l’applicabilité des régimes de responsabilité du Code Civil du Québec aux cas de dommages causés par le robot d’assistance. L’analyse des régimes de responsabilité du Code civil du Québec permet de constater que deux régimes de responsabilité sont susceptibles d’être appliqués aux cas spécifiques de dommages causés par le robot d’assistance: le régime de responsabilité du fait des biens, énoncé à l’article 1465 C.c.Q., et le régime de responsabilité du fait des fabricants et vendeurs spécialisés, énoncé à l’article 1468 C.c.Q. Cela s’explique par la présence de critères et de conditions de mise en œuvre des régimes qui sont transposables aux différents aspects concernant la fabrication et l’utilisation du robot d’assistance.

La responsabilité civile dans les cas de dommages causés par les robots d’assistance au Québec

Le développement exponentiel de la technologie et le vieillissement de la population permettent d’introduire dans notre quotidien les robots d’assistance. La coexistence de l’homme avec ces robots intelligents et autonomes soulève une question fondamentale: dans l’éventualité où un robot provoquerait un accident causant un dommage à une personne ou à un bien qui serait le responsable? Aucune loi ne réglemente les activités de la robotique d’assistance dans le monde. Cette étude vise l’analyse de l’applicabilité des régimes de responsabilité du Code Civil du Québec aux cas de dommages causés par le robot d’assistance. L’analyse des régimes de responsabilité du Code civil du Québec permet de constater que deux régimes de responsabilité sont susceptibles d’être appliqués aux cas spécifiques de dommages causés par le robot d’assistance: le régime de responsabilité du fait des biens, énoncé à l’article 1465 C.c.Q., et le régime de responsabilité du fait des fabricants et vendeurs spécialisés, énoncé à l’article 1468 C.c.Q. Cela s’explique par la présence de critères et de conditions de mise en œuvre des régimes qui sont transposables aux différents aspects concernant la fabrication et l’utilisation du robot d’assistance.