Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Model-based objects recognition in industrial environments for autonomous vehicles control

Behavior-based navigation of autonomous vehicles requires the recognition of the navigable areas and the potential obstacles. In this paper we describe a model-based objects recognition system which is part of an image interpretation system intended to assist the navigation of autonomous vehicles that operate in industrial environments. The recognition system integrates color, shape and texture information together with the location of the vanishing point. The recognition process starts from some prior scene knowledge, that is, a generic model of the expected scene and the potential objects. The recognition system constitutes an approach where different low-level vision techniques extract a multitude of image descriptors which are then analyzed using a rule-based reasoning system to interpret the image content. This system has been implemented using a rule-based cooperative expert system

Mission control system for dam inspection with an AUV

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

Using petri nets to specify and execute missions for autonomous underwater vehicles

This paper presents the design and implementation of a mission control system (MCS) for an autonomous underwater vehicle (AUV) based on Petri nets. In the proposed approach the Petri nets are used to specify as well as to execute the desired autonomous vehicle mission. The mission is easily described using an imperative programming language called mission control language (MCL) that formally describes the mission execution thread. A mission control language compiler (MCL-C) able to automatically translate the MCL into a Petri net is described and a real-time Petri net player that allows to execute the resulting Petri net onboard an AUV are also presented

An overview of the advantages and constraints of coded pattern projection techniques for autonomous navigation

The absolute necessity of obtaining 3D information of structured and unknown environments in autonomous navigation reduce considerably the set of sensors that can be used. The necessity to know, at each time, the position of the mobile robot with respect to the scene is indispensable. Furthermore, this information must be obtained in the least computing time. Stereo vision is an attractive and widely used method, but, it is rather limited to make fast 3D surface maps, due to the correspondence problem. The spatial and temporal correspondence among images can be alleviated using a method based on structured light. This relationship can be directly found codifying the projected light; then each imaged region of the projected pattern carries the needed information to solve the correspondence problem. We present the most significant techniques, used in recent years, concerning the coded structured light method

A Gain Scheduling Model Predictive Controller for Blood Glucose Control in Type 1 Diabetes

This paper presents a control strategy for blood glucose(BG) level regulation in type 1 diabetic patients. To design the controller, model-based predictive control scheme has been applied to a newly developed diabetic patient model. The controller is provided with a feedforward loop to improve meal compensation, a gain-scheduling scheme to account for different BG levels, and an asymmetric cost function to reduce hypoglycemic risk. A simulation environment that has been approved for testing of artificial pancreas control algorithms has been used to test the controller. The simulation results show a good controller performance in fasting conditions and meal disturbance rejection, and robustness against model–patient mismatch and errors in meal estimation

Social Networking Sites as Technology Enhanced Learning Communities towards autonomous language learning

The following contribution pretends to cope with the demands of a globalised, post-modern environment through the design and implementation of an online international project where an SNS is used in order to join English as Second Language (ESL) students from different parts of the world. The design of the project appears around the implementation of the Bologna process in the Faculty of Education from the University of Girona where the basic prerequisite of all students to acquire English at the level B1 of the Common European Portfolio makes English a compulsory competence for communication among its higher education candidates in order to develop in the world. Together with the University of Girona, there is the International Educational and Resources Network (iEARN) which promotes the participation of schools around the world in online international projects

Didactics–promoting collaborative work and personal learning environment. Critical, creative and autonomous roles in a competitive society

School has evolved from a place where knowledge is provided to a place where learners are helped to develop their professional and social skills. Consequently, education must evolve through big challenges in order to face the changes of society in the XXIst century

A mission control system for an autonomous underwater vehicle

The presented work focuses on the theoretical and practical aspects concerning the design and development of a formal method to build a mission control system for autonomous underwater vehicles bringing systematic design principles for the formal description of missions using Petri nets. The proposed methodology compounds Petri net building blocks within it to de_ne a mission plan for which it is proved that formal properties, such as reachability and reusability, hold as long as these same properties are also guaranteed by each Petri net building block. To simplify the de_nition of these Petri net blocks as well as their composition, a high level language called Mission Control Language has been developed. Moreover, a methodology to ensure coordination constraints for teams of multiple robots as well as the de_nition of an interface between the proposed system and an on-board planner able to plan/replan sequences of prede_ned mission plans is included as well. Results of experiments with several real underwater vehicles and simulations involving an autonomous surface craft and an autonomous underwater vehicles are presented to show the system's capabilities.

A Multi-agent Architecture with Distributed Coordination for an Autonomous Robot

Aquest treball proposa una nova arquitectura de control amb coordinació distribuïda per a un robot mòbil (ARMADiCo). La metodologia de coordinació distribuïda consisteix en dos passos: el primer determina quin és l'agent que guanya el recurs basat en el càlcul privat de la utilitat i el segon, com es fa el canvi del recurs per evitar comportaments abruptes del robot. Aquesta arquitectura ha estat concebuda per facilitar la introducció de nous components hardware i software, definint un patró de disseny d'agents que captura les característiques comunes dels agents. Aquest patró ha portat al desenvolupament d'una arquitectura modular dins l'agent que permet la separació dels diferents mètodes utilitzats per aconseguir els objectius, la col·laboració, la competició i la coordinació de recursos. ARMADiCo s'ha provat en un robot Pioneer 2DX de MobileRobots Inc.. S'han fet diversos experiments i els resultats han demostrat que s'han aconseguit les característiques proposades per l'arquitectura.

Simulation of delamination in composites under quasi-static and fatigue loading using cohesive zone models

Es desenvolupa una eina de disseny per l'anàlisi de la tolerància al dany en composites. L'eina pot predir el inici i la propagació de fisures interlaminars. També pot ser utilitzada per avaluar i planificar la necessitat de reparar o reemplaçar components durant la seva vida útil. El model desenvolupat pot ser utilitzat tan per simular càrregues estàtiques com de fatiga. El model proposat és un model de dany termodinàmicament consistent que permet simular la delaminació en composites sota càrregues variables. El model es formula dins el context de la Mecànica del Dany, fent ús dels models de zona cohesiva. Es presenta un metodologia per determinar els paràmetres del model constitutiu que permet utilitzar malles d'elements finits més bastes de les que es poden usar típicament. Finalment, el model és també capaç de simular la delaminació produïda per càrregues de fatiga.

Simulation of interlaminar and intralaminar damage in polymer-based composites for aeronautical applications under impact loading

La aplicación de materiales compuestos de matriz polimérica reforzados mediante fibras largas (FRP, Fiber Reinforced Plastic), está en gradual crecimiento debido a las buenas propiedades específicas y a la flexibilidad en el diseño. Uno de los mayores consumidores es la industria aeroespacial, dado que la aplicación de estos materiales tiene claros beneficios económicos y medioambientales. Cuando los materiales compuestos se aplican en componentes estructurales, se inicia un programa de diseño donde se combinan ensayos reales y técnicas de análisis. El desarrollo de herramientas de análisis fiables que permiten comprender el comportamiento mecánico de la estructura, así como reemplazar muchos, pero no todos, los ensayos reales, es de claro interés. Susceptibilidad al daño debido a cargas de impacto fuera del plano es uno de los aspectos de más importancia que se tienen en cuenta durante el proceso de diseño de estructuras de material compuesto. La falta de conocimiento de los efectos del impacto en estas estructuras es un factor que limita el uso de estos materiales. Por lo tanto, el desarrollo de modelos de ensayo virtual mecánico para analizar la resistencia a impacto de una estructura es de gran interés, pero aún más, la predicción de la resistencia residual después del impacto. En este sentido, el presente trabajo abarca un amplio rango de análisis de eventos de impacto a baja velocidad en placas laminadas de material compuesto, monolíticas, planas, rectangulares, y con secuencias de apilamiento convencionales. Teniendo en cuenta que el principal objetivo del presente trabajo es la predicción de la resistencia residual a compresión, diferentes tareas se llevan a cabo para favorecer el adecuado análisis del problema. Los temas que se desarrollan son: la descripción analítica del impacto, el diseño y la realización de un plan de ensayos experimentales, la formulación e implementación de modelos constitutivos para la descripción del comportamiento del material, y el desarrollo de ensayos virtuales basados en modelos de elementos finitos en los que se usan los modelos constitutivos implementados.

Selective submap joining SLAM for autonomous vehicles

Simultaneous Localization and Mapping (SLAM) do not result in consistent maps of large areas because of gradual increase of the uncertainty for long term missions. In addition, as the size of the map grows the computational cost increases, making SLAM solutions unsuitable for on-line applications. This thesis surveys SLAM approaches paying special attention to those approaches aimed to work on large scenarios. Special focus is given to existing underwater SLAM applications. A technique based on using independent local maps together with a global stochastic map is presented. This technique is called Selective Submap Joining SLAM (SSJS). A global map contains relative transformations between local maps, which are updated once a new loop is detected. Maps sharing several features are fused, maintaining the correlation between landmarks and vehicle's pose. The use of local maps reduces computational costs and improves map consistency as compared to state of the art techniques.