Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Diseño e implantación de un sistema SCADA para una planta de producción y envasado de líquidos

Este proyecto consiste en la implantación de un sistema SCADA en una planta piloto de fabricación y envasado de líquidos. En este documento se muestra toda la información que ha sido necesaria para realizar el sistema de automatización en los tres niveles propios de un SCADA: nivel de campo, nivel de control y nivel de supervisión. El trabajo de campo ha sido, típicamente de revisión, puesto que el Departament de Telecomunicació i d’Enginyeria de Sistemes ya poseía la planta totalmente equipada, mientras que el sistema de control y el sistema de supervisión se han implantado totalmente.

An hybrid methodology for RL-based behavior coordination in a target following mission with an AUV

Proposes a behavior-based scheme for high-level control of autonomous underwater vehicles (AUVs). Two main characteristics can be highlighted in the control scheme. Behavior coordination is done through a hybrid methodology, which takes in advantages of the robustness and modularity in competitive approaches, as well as optimized trajectories

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Intelligent task-level grasp mapping for robot control

In the future, robots will enter our everyday lives to help us with various tasks.For a complete integration and cooperation with humans, these robots needto be able to acquire new skills. Sensor capabilities for navigation in real humanenvironments and intelligent interaction with humans are some of the keychallenges.Learning by demonstration systems focus on the problem of human robotinteraction, and let the human teach the robot by demonstrating the task usinghis own hands. In this thesis, we present a solution to a subproblem within thelearning by demonstration field, namely human-robot grasp mapping. Robotgrasping of objects in a home or office environment is challenging problem.Programming by demonstration systems, can give important skills for aidingthe robot in the grasping task.The thesis presents two techniques for human-robot grasp mapping, directrobot imitation from human demonstrator and intelligent grasp imitation. Inintelligent grasp mapping, the robot takes the size and shape of the object intoconsideration, while for direct mapping, only the pose of the human hand isavailable.These are evaluated in a simulated environment on several robot platforms.The results show that knowing the object shape and size for a grasping taskimproves the robot precision and performance

Desarrollo y aplicación de estrategias de control avanzado en procesos biológicos de tratamiento de efluentes contaminados

En este proyecto se ha desarrollado estrategias de control avanzadas para plantas de depuración de aguas residuales urbanas que eliminan conjuntamente materia orgánica, nitrógeno y fósforo. Las estrategias se han basado en el estudio multivariable del comportamiento del sistema, que ha producido subsidios para la utilización de lazos de control feedforward, de control predictivo y de un control de costes que automáticamente enviaba las consignas más adecuadas para los controladores de proceso. Para el desarrollo de las estrategias, se ha creado un sistema virtual de simulación (simulador) de plantas de depuradoras, basado en datos de literatura. Para el caso de una planta real, se ha desarrollado un simulador de la planta de Manresa (Catalunya). Sin embargo, el sistema de Manresa se ha utilizado exclusivamente para auxiliar los ingenieros de la planta en la tomada de decisiones de cambio de configuración para que la eliminación de fósforo se dé por la ruta biológica y no por la ruta química. La implementación de los simuladores ha permitido hacer muchas pruebas que en una planta real demandarían mucho tiempo y consumirían muchos recursos energéticos y financieros. Las estrategias de control más elaboradas han podido ahorrar hasta 150.000,00 Euros por año en relación a la operación de la planta sin el control automático. Cuanto a los estudios del modelo de la planta real, se concluyó que la eliminación biológica de fósforo puede sustituir el actual proceso químico de eliminación de fósforo, bajando los costes operacionales (costes del agente precipitante).

Can You Home Again? Desertion and Control of Hometowns in Civil Wars

What allows an armed group in a civil war to prevent desertion? This paper addresses this question with a focus on control in the rearguard. Most past studies focus on motivations for desertion. They explain desertion in terms of where soldiers stand in relation to the macro themes of the war, or in terms of an inability to provide positive incentives to overcome the collective action problem. However, since individuals decide whether and how to participate in civil wars for multiple reasons, responding to a variety of local conditions in an environment of threat and violence, a focus only on macro-level motivations is incomplete. The opportunities side of the ledger deserves more attention. I therefore turn my attention to how control by an armed group eliminates soldiers’ opportunities to desert. In particular, I consider the control that an armed group maintains over soldiers’ hometowns, treating geographic terrain as an important exogenous indicator of the ease of control. Rough terrain at home affords soldiers and their families and friends advantages in ease of hiding, the difficulty of using force, and local knowledge. Based on an original dataset of soldiers from Santander Province in the Spanish Civil War, gathered from archival sources, I find statistical evidence that the rougher the terrain in a soldier’s home municipality, the more likely he is to desert. I find complementary qualitative evidence indicating that soldiers from rough-terrain communities took active advantage of their greater opportunities for evasion. This finding has important implications for the way observers interpret different soldiers’ decisions to desert or remain fighting, for the prospect that structural factors may shape the cohesion of armed groups, and for the possibility that local knowledge may be a double-edged sword, making soldiers simultaneously good at fighting and good at deserting.

Supervisió i control de la lectura: anàlisi de procediments, judicis metacognitius i resultats de comprensió en estudiants d'Educació Secundària Obligatòria

Els processos de comprensió lectora són complexes, i requereixen de l’ús autorregulat, conscient i deliberat de la metacognició en les seves dues vessants: supervisió i control. L’objectiu d’aquest projecte és analitzar les relacions entre aquests dos components, i la seva repercussió en la comprensió lectora que finalment s’assoleix. Per tal de tractar aquesta qüestió s’han analitzat els processos de lectura seguits per 60 alumnes d’Educació Secundària Obligatòria. La presència i el tipus de relectura ha estat considerada com a mesura de control (estratègia emprada per millorar la pròpia comprensió), mentre que els judicis metacognitius són considerats com a mesura de supervisió. També es va elaborar una prova de comprensió lectora incloent preguntes que requerissin diferents nivells de processament cognitiu, de més superficial a més profund. Els nostres resultats preliminars mostren una relació entre l’ajust dels judicis metacognitius i la comprensió lectora assolida; els lectors que millor entenen el text tendeixen a ser els que millor supervisen la seva comprensió. La relectura també s’ha mostrat útil per millorar la comprensió, ja que els participants obtenen millors resultats després de rellegir. Per últim, s’ha trobat una relació significativa entre l’ajust dels judicis metacognitius i la relectura. D’una banda, aquests resultats subratllen la importància de la supervisió i el control en la comprensió lectora, i ofereixen evidències per explicar les complexes relacions entre supervisió, control, i resultats de comprensió. A més, els processos de lectura i els objectius amb què hom rellegeix apareixen com a temes rellevants que requereixen ser investigats per comprendre millor la interacció entre els processos de supervisió i control en la comprensió lectora. D’altra banda, els resultats tenen valuoses implicacions educatives, ja que proporcionen evidències de la importància d’ensenyar processos metacognitius des d’un punt de vista estratègic, en contraposició a les “tècniques”, per tal de millorar la comprensió lectora.

El control de la informació a Catalunya : regulació i autoregulació dels periodistes i mitjans de comunicació

S'hi estudia el paper dels mitjans de comunicació en el sistema democràtic i la manera com la deontologia pot ajudar a pal·liar alguns dels abusos o desviacions de l'ètica que hauria de presidir el comportament de periodistes i mitjans. També s'hi analitza el control que cal establir sobre aquests protagonistes davant l'actual concentració d'empreses de comunicació.

Predicting random level and seasonality of hotel prices. A structural equation growth curve approach

This article examines the effect on price of different characteristics of holiday hotels in the sun-and-beach segment, under the hedonic function perspective. Monthly prices of the majority of hotels in the Spanish continental Mediterranean coast are gathered from May to October 1999 from the tour operator catalogues. Hedonic functions are specified as random-effect models and parametrized as structural equation models with two latent variables, a random peak season price and a random width of seasonal fluctuations. Characteristics of the hotel and the region where they are located are used as predictors of both latent variables. Besides hotel category, region, distance to the beach, availability of parking place and room equipment have an effect on peak price and also on seasonality. 3- star hotels have the highest seasonality and hotels located in the southern regions the lowest, which could be explained by a warmer climate in autumn

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Experimental Modeling and Control Strategies on an Open Mobile Robot Platform PRIM

This research work deals with the problem of modeling and design of low level speed controller for the mobile robot PRIM. The main objective is to develop an effective educational tool. On one hand, the interests in using the open mobile platform PRIM consist in integrating several highly related subjects to the automatic control theory in an educational context, by embracing the subjects of communications, signal processing, sensor fusion and hardware design, amongst others. On the other hand, the idea is to implement useful navigation strategies such that the robot can be served as a mobile multimedia information point. It is in this context, when navigation strategies are oriented to goal achievement, that a local model predictive control is attained. Hence, such studies are presented as a very interesting control strategy in order to develop the future capabilities of the system

Mission control system for dam inspection with an AUV

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

The quest for a message: budding yeast, a model organism to study the control of pre-mRNA splicing

Removal of introns during pre-mRNA splicing is a critical process in gene expression, and understanding its control at both single-gene and genomic levels is one of the great challenges in Biology. Splicing takes place in a dynamic, large ribonucleoprotein complex known as the spliceosome. Combining Genetics and Biochemistry, Saccharomyces cerevisiae provides insights into its mechanisms, including its regulation by RNA-protein interactions. Recent genome-wide analyses indicate that regulated splicing is broad and biologically relevant even in organisms with a relatively simple intronic structure, such as yeast. Furthermore, the possibility of coordination in splicing regulation at genomic level is becoming clear in this model organism. This should provide a valuable system to approach the complex problem of the role of regulated splicing in genomic expression.

Mobile robot experimental modeling and control strategies using sensor fusion

This research work deals with the problem of modeling and design of low level speed controller for the mobile robot PRIM. The main objective is to develop an effective educational, and research tool. On one hand, the interests in using the open mobile platform PRIM consist in integrating several highly related subjects to the automatic control theory in an educational context, by embracing the subjects of communications, signal processing, sensor fusion and hardware design, amongst others. On the other hand, the idea is to implement useful navigation strategies such that the robot can be served as a mobile multimedia information point. It is in this context, when navigation strategies are oriented to goal achievement, that a local model predictive control is attained. Hence, such studies are presented as a very interesting control strategy in order to develop the future capabilities of the system. In this context the research developed includes the visual information as a meaningful source that allows detecting the obstacle position coordinates as well as planning the free obstacle trajectory that should be reached by the robot