Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Digitalización del sistema de control de nivel en calentadores y tanques de drenaje de una Central Nuclear

Contexto Una central nuclear, al igual que cualquier otro tipo de central generadora de energía eléctrica, mediante turbinas de vapor, está basada en un proceso termodinámico. El rendimiento de las mismas es función del salto entálpico del vapor, para mejorarlo las centrales están constituidas por un ciclo compound formado por turbina de alta presión y turbinas de baja presión, y un ciclo regenerativo consistente en calentar el agua de alimentación antes de su introducción a los generadores de vapor. Un ciclo regenerativo está basado en etapas de calentadores o cambiadores de calor para aprovechar al máximo la energía térmica del vapor, este proyecto está basado en la mejora y optimización del proceso de control de estos para contribuir a mejorar el rendimiento de la central. Objetivo Implementar un sistema de control que nos permita modernizar los clásicos sistemas basados en controles locales y comunicaciones analógicas. Mejorar el rendimiento del ciclo regenerativo de la central, aprovechando las mejoras tecnológicas que ofrece el mercado, tanto en el hardware como en el software de los sistemas de instrumentación y control. Optimizar el rendimiento de los lazos de control de cada uno de los elementos del ciclo regenerativo mediante estrategias de control. Procedimiento Desarrollo de un sistema de control actualizado considerando, como premisa principal, la fiabilidad del sistema, el análisis de fallos y la jerarquización del riesgo. Análisis y cálculo de los lazos de control considerando las premisas establecidas. Configuración de los lazos mediante estrategias de control que nos permitan optimizar y minimizar los efectos del fallo. Para ello se han utilizado parámetros y datos extraídos de la Central Nuclear de Ascó. Conclusiones Se ha modernizado y optimizado el sistema de control mejorando el rendimiento del ciclo regenerativo. Se ha conseguido un sistema más fiable, reduciendo el riesgo del fallo y disminuyendo los efectos de los mismos. El coste de un proyecto de estas características es inferior al de un sistema convencional y ofrece más posibilidades. Es un sistema abierto que permite utilizar e interconectar equipos de diferentes fabricantes, lo que favorece tanto el mantenimiento como las posibles ampliaciones futuras del sistema.

Gestión de centros deportivos: Virtual Trainer Web Ed.

Virtual Trainer Web Edition proporciona un sistema de gestión, planificación y control de un centro deportivo, ofreciendo interactividad entre técnicos, socios y personal de gerencia y facilitando todo el trabajo de un mantenimiento de información tradicional. A través de cualquier PC con conexión a Internet y sin instalación de software adicional, se accede a una completa aplicación con contenidos multimedia.

Diseño e implantación de un sistema SCADA para una planta de producción y envasado de líquidos

Este proyecto consiste en la implantación de un sistema SCADA en una planta piloto de fabricación y envasado de líquidos. En este documento se muestra toda la información que ha sido necesaria para realizar el sistema de automatización en los tres niveles propios de un SCADA: nivel de campo, nivel de control y nivel de supervisión. El trabajo de campo ha sido, típicamente de revisión, puesto que el Departament de Telecomunicació i d’Enginyeria de Sistemes ya poseía la planta totalmente equipada, mientras que el sistema de control y el sistema de supervisión se han implantado totalmente.

NXT: el robot de Lego

Treball de recerca realitzat per un alumne d'ensenyament secundari i guardonat amb un Premi CIRIT per fomentar l'esperit cientí­fic del Jovent l'any 2009. L'NXT és un robot creat per l'empresa Lego que disposa d'un controlador, de diversos servo motors i de sensors (tacte, llum, ultrasons, so...). Es programa mitjançant un programa especial, pensat per nois i noies de catorze anys, anomenat Lego Mindstorms. S'estudia el funcionament d'aquest programa i les parts del sistema de control del robot. L'estudi engloba el controlador, quatre sensors i els servomotors.

Extensió de les capacitats de monitoratge de JFFNM en entorns Wlan

Aquest TFC pretén recollir un conjunt de paràmetres de radi d'un dispositiu Wireless i poder representar-los en l'aplicació de monitoratge Just For Fun Network Management System (JFFNMS).

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Graphical simulators for AUV development

A long development time is needed from the design to the implementation of an AUV. During the first steps, simulation plays an important role, since it allows for the development of preliminary versions of the control system to be integrated. Once the robot is ready, the control systems are implemented, tuned and tested. The use of a real-time simulator can help closing the gap between off-line simulation and real testing using the already implemented robot. When properly interfaced with the robot hardware, a real-time graphical simulation with a "hardware in the loop" configuration, can allow for the testing of the implemented control system running in the actual robot hardware. Hence, the development time is drastically reduced. These paper overviews the field of graphical simulators used for AUV development proposing a classification. It also presents NEPTUNE, a multi-vehicle, real-time, graphical simulator based on OpenGL that allows hardware in the loop simulations

I-SS: integrated supervision systems approach based on interactive components

Process supervision is the activity focused on monitoring the process operation in order to deduce conditions to maintain the normality including when faults are present Depending on the number/distribution/heterogeneity of variables, behaviour situations, sub-processes, etc. from processes, human operators and engineers do not easily manipulate the information. This leads to the necessity of automation of supervision activities. Nevertheless, the difficulty to deal with the information complicates the design and development of software applications. We present an approach called "integrated supervision systems". It proposes multiple supervisors coordination to supervise multiple sub-processes whose interactions permit one to supervise the global process

Interacting software-agents to support a kind of expert supervision systems

Expert supervision systems are software applications specially designed to automate process monitoring. The goal is to reduce the dependency on human operators to assure the correct operation of a process including faulty situations. Construction of this kind of application involves an important task of design and development in order to represent and to manipulate process data and behaviour at different degrees of abstraction for interfacing with data acquisition systems connected to the process. This is an open problem that becomes more complex with the number of variables, parameters and relations to account for the complexity of the process. Multiple specialised modules tuned to solve simpler tasks that operate under a co-ordination provide a solution. A modular architecture based on concepts of software agents, taking advantage of the integration of diverse knowledge-based techniques, is proposed for this purpose. The components (software agents, communication mechanisms and perception/action mechanisms) are based on ICa (Intelligent Control architecture), software middleware supporting the build-up of applications with software agent features

Application for fault location in electrical power distribution systems

Fault location has been studied deeply for transmission lines due to its importance in power systems. Nowadays the problem of fault location on distribution systems is receiving special attention mainly because of the power quality regulations. In this context, this paper presents an application software developed in Matlabtrade that automatically calculates the location of a fault in a distribution power system, starting from voltages and currents measured at the line terminal and the model of the distribution power system data. The application is based on a N-ary tree structure, which is suitable to be used in this application due to the highly branched and the non- homogeneity nature of the distribution systems, and has been developed for single-phase, two-phase, two-phase-to-ground, and three-phase faults. The implemented application is tested by using fault data in a real electrical distribution power system

Controladores PI con acción de reset

El control automàtic exerceix un paper important en molts processos de la industria. Cada un dels sistemes de control requereix d’un controlador, la majoria dels quals són del tipus PI. L’objectiu d’aquest projecte es investigar tècniques que permetin superar les limitacions que tenen els controladors PI lineals. En la resposta d’un sistema de control es poden distingir dues tasques diferents: El seguiment a un canvi d’entrada o consigna correspon a la tasca de servo, mentre que el rebuig a pertorbacions correspon a la tasca de regulatori. Al típic esquema de control realimentat, aquestes dues tasques estan enfrontades, és a dir, una millora a la tasca de servo implica un empitjorament a la tasca de regulatori i a l’inversa. Això suposa un problema al rendiment del sistema, així com la necessitat d’establir un cert compromís entre les dues tasques. El que es pretén en aquest projecte es implementar senzilles regles de control no lineal amb la finalitat de millorar el rendiment del sistema i evitar la necessitat d’establir un compromís entre les dues tasques. Així, es pretén superar les limitacions que aquest té, obtenint controladors PI alternatius fàcilment sintetitzables.

Two methods for voltage sag source location

This paper presents and compares two approaches to estimate the origin (upstream or downstream) of voltage sag registered in distribution substations. The first approach is based on the application of a single rule dealing with features extracted from the impedances during the fault whereas the second method exploit the variability of waveforms from an statistical point of view. Both approaches have been tested with voltage sags registered in distribution substations and advantages, drawbacks and comparative results are presented

A survey on voltage sag events in power systems

This paper aims to survey the techniques and methods described in literature to analyse and characterise voltage sags and the corresponding objectives of these works. The study has been performed from a data mining point of view