Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Mission control system for dam inspection with an AUV

This paper presents a complete control architecture that has been designed to fulfill predefined missions with an autonomous underwater vehicle (AUV). The control architecture has three levels of control: mission level, task level and vehicle level. The novelty of the work resides in the mission level, which is built with a Petri network that defines the sequence of tasks that are executed depending on the unpredictable situations that may occur. The task control system is composed of a set of active behaviours and a coordinator that selects the most appropriate vehicle action at each moment. The paper focuses on the design of the mission controller and its interaction with the task controller. Simulations, inspired on an industrial underwater inspection of a dam grate, show the effectiveness of the control architecture

On AUV control architecture

This paper surveys control architectures proposed in the literature and describes a control architecture that is being developed for a semi-autonomous underwater vehicle for intervention missions (SAUVIM) at the University of Hawaii. Conceived as hybrid, this architecture has been organized in three layers: planning, control and execution. The mission is planned with a sequence of subgoals. Each subgoal has a related task supervisor responsible for arranging a set of pre-programmed task modules in order to achieve the subgoal. Task modules are the key concept of the architecture. They are the main building blocks and can be dynamically re-arranged by the task supervisor. In our architecture, deliberation takes place at the planning layer while reaction is dealt through the parallel execution of the task modules. Hence, the system presents both a hierarchical and an heterarchical decomposition, being able to show a predictable response while keeping rapid reactivity to the dynamic environment

Identification and Semiactive Control of Smart Structures Equipped with Magnetorheological Actuators

This paper deals with the problem of identification and semiactive control of smart structures subject to unknown external disturbances such as earthquake, wind, etc. The experimental setup used is a 6-story test structure equipped with shear-mode semiactive magnetorheological actuators being installed in WUSCEEL. The experimental results obtained have verified the effectiveness of the proposed control algorithms

Predictive motion control of a mirosot mobile robot

This paper discusses predictive motion control of a MiRoSoT robot. The dynamic model of the robot is deduced by taking into account the whole process - robot, vision, control and transmission systems. Based on the obtained dynamic model, an integrated predictive control algorithm is proposed to position precisely with either stationary or moving obstacle avoidance. This objective is achieved automatically by introducing distant constraints into the open-loop optimization of control inputs. Simulation results demonstrate the feasibility of such control strategy for the deduced dynamic model

Millora experimental de les lleis de control predictiu aplicades sobre la plataforma PRIM I, per el seguiment de trajectòries

Aquest projecte s’aplica sobre el robot PRIM (Plataforma Robotitzada d’Informació Multimèdia), un robot autònom no humanoide creat el 2004 per Ateneu Informàtic (AI) que permet realitzar trajectòries 2D gràcies a un sistema de tracció format per dues rodes motrius propulsades independentment. La plataforma PRIM és controlada a partir del control predictiu, aquest control es va implementar en un projecte anterior, creat per l’Alexandre Blasco Gutierrez i titulat “Implementació de tècniques MPC (Model Predictiu Control) sobre la plataforma PRIM I”. El que es pretén en aquest projecte és millorar els resultats obtinguts en el passat projecte reformulant la llei de control i analitzar les discrepàncies obtingudes en les metodologies que s’utilitzen per minimitzar la funció de costos a partir de simulacions de trajectòries

New delay-dependent stability criteria for uncertain neutral systems with mixed time-varying delays and nonlinear perturbations

The problem of stability analysis for a class of neutral systems with mixed time-varying neutral, discrete and distributed delays and nonlinear parameter perturbations is addressed. By introducing a novel Lyapunov-Krasovskii functional and combining the descriptor model transformation, the Leibniz-Newton formula, some free-weighting matrices, and a suitable change of variables, new sufficient conditions are established for the stability of the considered system, which are neutral-delay-dependent, discrete-delay-range dependent, and distributeddelay-dependent. The conditions are presented in terms of linear matrix inequalities (LMIs) and can be efficiently solved using convex programming techniques. Two numerical examples are given to illustrate the efficiency of the proposed method

An LMI approach to H∞ synchronization of second-order neutral master-slave systems

The H∞ synchronization problem of the master and slave structure of a second-order neutral master-slave systems with time-varying delays is presented in this paper. Delay-dependent sufficient conditions for the design of a delayed output-feedback control are given by Lyapunov-Krasovskii method in terms of a linear matrix inequality (LMI). A controller, which guarantees H∞ synchronization of the master and slave structure using some free weighting matrices, is then developed. A numerical example has been given to show the effectiveness of the method

Robust control of structures subject to uncertain disturbances and actuator dynamics

The objective the present research is try to find some control design strategies, which must be effective and closed to the real operation conditions. As a novel contribution to structural control strategies, the theories of Interval Modal Arithmetic, Backstepping Control and QFT (Qualitative Feedback Theory) will be studied. The steps to follow are to develop first new controllers based on the above theories and then to implement the proposed control strategies to different kind of structures. The report is organized as follows. The Chapter 2 presents the state-of-the-art on structural control systems. The chapter 3 presents the most important open problems found in field of structural control. The exploratory work made by the author, research proposal and working plan are given in the Chapter 4

Integració de la posició GPS per fer el control del moviment d'un robot

L’objectiu d’aquest treball és fer un estudi dels diferents tipus de sistemes de posicionament global que hi ha en el mercat, elegir un mòdul receptor assequible per poder analitzar-lo i veure si disposa de les característiques adequades per integrar-lo en un robot autònom d’exploració del projecte Sant Bernardo. S’hauran de fer les anàlisis de la precisió del mòdul en les diferents direccions cardinals, es a dir Nord, Sud, Est, Oest i altura i veure la diferència d’error que hi ha en cada una, veure si la precisió varia molt en diferents situacions, com en cel obert, en sotabosc, costat i interior edificis. A més a més s’haurà de mirar la repetibilitat, la diferencia d’error amb diferentnombre de satèl•lits connectats i si disposa de suficient velocitat de processat per a podar corregir la posició del robot en moviment. Un cop analitzades les característiques del mòdul receptor elegit, es decidirà si aquest ésadequat per fer les correccions de posició del robot, o s’haurà d’adquirir un mòdul de característiques superiors i per tant molt més car per a dura a terme adequadament la correcció de la posició

Manipulator control board

The Mechatronics Research Centre (MRC) owns a small scale robot manipulator called aMini-Mover 5. This robot arm is a microprocessor-controlled, six-jointed mechanical armdesigned to provide an unusual combination of dexterity and low cost.The Mini-Mover-5 is operated by a number of stepper motors and is controlled by a PCparallel port via a discrete logic board. The manipulator also has an impoverished array ofsensors.This project requires that a new control board and suitable software be designed to allow themanipulator to be controlled from a PC. The control board will also provide a mechanism forthe values measured using some sensors to be returned to the PC.On this project I will consider: stepper motor control requirements, sensor technologies,power requirements, USB protocols, USB hardware and software development and controlrequirements (e.g. sample rates).In this report we will have a look at robots history and background, as well as we willconcentrate how stepper motors and parallel port work

Control de glicèmia en pacients crítics mitjançant CGMS

L’estudi que es realitza en aquest projecte/treball final de carrera queda englobat dins del grup de recerca MICE (Modal Intervals Control and Engeneering), el qual realitzainvestigacions entorn al control de glucèmia. Aquest grup de recerca vinculat a la Universitat de Girona col•labora amb l’Hospital Universitari Dr. Josep Trueta de Girona. La temàtica principal tractarà de realitzar el control de glucèmia en pacients crítics, que es troben ingressats en la unitat de cures intensives de qualsevol hospital. Com a conseqüència d’aquesta problemàtica, s’ha implementat en un entorn virtual, un pacient el qual simula la situació d’un pacient real en la unitat de cures intensives. El model emprat per a la obtenció del model de pacient virtual és el desenvolupat per Chase et al. (2005), el qual mitjançant variables com l’alimentació enteral i la sensibilitat insulínica, es podien realitzar assajos reals per a validar protocols de control ‘in silico’ per posteriorment realitzar assajos amb població real

Control de posició del robot wifibot

Aquest projecte pretén presentar de forma clara i detallada l’estructura i el funcionament del robot així com dels components que el conformen. Aquesta informació és de vital importància a l’hora de desenvolupar aplicacions per al robot. Un cop descrites les característiques del robot s’analitzaran les eines necessàries i/o disponibles per poder desenvolupar programari per cada nivell de la forma més senzilla i eficient possible. Posteriorment s’analitzaran els diferents nivells de programació i se’n contrastaran els avantatges i els inconvenients de cada un. Aquest anàlisi es començarà fent pel nivell més alt i anirà baixant amb la intenció de no entrar en nivells més baixos del necessari. Baixar un nivell en la programació suposa haver de crear aplicacions sempre compatibles amb els nivells superiors de forma que com més es baixa més augmenta la complexitat. A partir d’aquest anàlisi s’ha arribat a la conclusió que per tal d’aprofitar totes les prestacions del robot és precís arribar a programar en el nivell més baix del robot. Finalment l’objectiu és obtenir una sèrie de programes per cada nivell que permetin controlar el robot i fer-lo seguir senzilles trajectòries

Local model predictive control experiences with differential driven wheeled mobile robots

This work extends a previously developed research concerning about the use of local model predictive control in differential driven mobile robots. Hence, experimental results are presented as a way to improve the methodology by considering aspects as trajectory accuracy and time performance. In this sense, the cost function and the prediction horizon are important aspects to be considered. The aim of the present work is to test the control method by measuring trajectory tracking accuracy and time performance. Moreover, strategies for the integration with perception system and path planning are briefly introduced. In this sense, monocular image data can be used to plan safety trajectories by using goal attraction potential fields