Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent's local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

Modelado y control de estación de célula de manipulación

[ES]El objetivo de este proyecto es el diseño e implementación del modelo de la estación FMS 201 (alimentación de la base) y el diseño e implementación del control de la estación. Esta estación pertenece a la serie FMS 200 (sistema didáctico modular de ensamblaje flexible) distribuido por la empresa SMC. Se dispone uno en el laboratorio de investigación del departamento de Ingeniería de Sistemas y Automática de la Escuela Superior de Ingeniería de Bilbao (EHU/UPV). Para el desarrollo e implementación del modelo se usará la herramienta informática Automation Studio. Para el control del modelo se usará el PLC. Para el intercambio de información entre modelo y controlador se utilizará la comunicación OPC Para el control de la estación se usa un PLC S7-300 de la marca SIEMENS. Se finaliza el documento realizando las pruebas de validación del modelo desarrollado, ejecutándose el programa de control en el PLC y corriendo el modelo desarrollado en el PC.

About the Stabilization of a Nonlinear Perturbed Difference Equation

This paper investigates the local asymptotic stabilization of a very general class of instable autonomous nonlinear difference equations which are subject to perturbed dynamics which can have a different order than that of the nominal difference equation. In the general case, the controller consists of two combined parts, namely, the feedback nominal controller which stabilizes the nominal (i.e., perturbation-free) difference equation plus an incremental controller which completes the stabilization in the presence of perturbed or unmodeled dynamics in the uncontrolled difference equation. A stabilization variant consists of using a single controller to stabilize both the nominal difference equation and also the perturbed one under a small-type characterization of the perturbed dynamics. The study is based on Banach fixed point principle, and it is also valid with slight modification for the stabilization of unstable oscillatory solutions.

Adaptive Scalable SVD Unit for Fast Processing of Large LSE Problems

Singular Value Decomposition (SVD) is a key linear algebraic operation in many scientific and engineering applications. In particular, many computational intelligence systems rely on machine learning methods involving high dimensionality datasets that have to be fast processed for real-time adaptability. In this paper we describe a practical FPGA (Field Programmable Gate Array) implementation of a SVD processor for accelerating the solution of large LSE problems. The design approach has been comprehensive, from the algorithmic refinement to the numerical analysis to the customization for an efficient hardware realization. The processing scheme rests on an adaptive vector rotation evaluator for error regularization that enhances convergence speed with no penalty on the solution accuracy. The proposed architecture, which follows a data transfer scheme, is scalable and based on the interconnection of simple rotations units, which allows for a trade-off between occupied area and processing acceleration in the final implementation. This permits the SVD processor to be implemented both on low-cost and highend FPGAs, according to the final application requirements.

Properties of convergence of a class of iterative processes generated by sequences of self-mappings with applications to switched dynamic systems

This article investigates the convergence properties of iterative processes involving sequences of self-mappings of metric or Banach spaces. Such sequences are built from a set of primary self-mappings which are either expansive or non-expansive self-mappings and some of the non-expansive ones can be contractive including the case of strict contractions. The sequences are built subject to switching laws which select each active self-mapping on a certain activation interval in such a way that essential properties of boundedness and convergence of distances and iterated sequences are guaranteed. Applications to the important problem of stability of dynamic switched systems are also given.

Results on proximal and generalized weak proximal contractions including the case of iteration-dependent range sets

This paper presents some further results on proximal and asymptotic proximal contractions and on a class of generalized weak proximal contractions in metric spaces. The generalizations are stated for non-self-mappings of the forms for and , or , subject to and , such that converges uniformly to T, and the distances are iteration-dependent, where , , and are non-empty subsets of X, for , where is a metric space, provided that the set-theoretic limit of the sequences of closed sets and exist as and that the countable infinite unions of the closed sets are closed. The convergence of the sequences in the domain and the image sets of the non-self-mapping, as well as the existence and uniqueness of the best proximity points, are also investigated if the metric space is complete. Two application examples are also given, being concerned, respectively, with the solutions through pseudo-inverses of both compatible and incompatible linear algebraic systems and with the parametrical

An adaptive sliding mode control law for the power maximization of the wind turbine system

POWERENG 2011

Stability and Limit Oscillations of a Control Event-Based Sampling Criterion

This paper investigates the presence of limit oscillations in an adaptive sampling system. The basic sampling criterion operates in the sense that each next sampling occurs when the absolute difference of the signal amplitude with respect to its currently sampled signal equalizes a prescribed threshold amplitude. The sampling criterion is extended involving a prescribed set of amplitudes. The limit oscillations might be interpreted through the equivalence of the adaptive sampling and hold device with a nonlinear one consisting of a relay with multiple hysteresis whose parameterization is, in general, dependent on the initial conditions of the dynamic system. The performed study is performed on the time domain.

Performance of an Ocean Energy Conversion System with DFIG Sensorless Control

The 2009/28/EC Directive requires Member States of the European Union to adopt a National Action Plan for Renewable Energy. In this context, the Basque Energy Board, EVE, is committed to research activities such as the Mutriku Oscillating Water Column plant, OWC. This is an experimental facility whose concept consists of a turbine located in a pneumatic energy collection chamber and a doubly fed induction generator that converts energy extracted by the turbine into a form that can be returned to the network. The turbo-generator control requires a precise knowledge of system parameters and of the rotor angular velocity in particular. Thus, to remove the rotor speed sensor implies a simplification of the hardware that is always convenient in rough working conditions. In this particular case, a Luenberger based observer is considered and the effectiveness of the proposed control is shown by numerical simulations. Comparing these results with those obtained using a traditional speed sensor, it is shown that the proposed solution provides better performance since it increases power extraction in the sense that it allows a more reliable and robust performance of the plant, which is even more relevant in a hostile environment as the ocean.

Adaptive variable structure control law for a variable speed wind turbine

Presentado en el 13th WSEAS International Conference on Automatic Control, Modelling and Simulation, ACMOS'11

A Data Dropout Compensation Algorithm Based on the Iterative Learning Control Methodology for Discrete-Time Systems

This paper deals with the convergence of a remote iterative learning control system subject to data dropouts. The system is composed by a set of discrete-time multiple input-multiple output linear models, each one with its corresponding actuator device and its sensor. Each actuator applies the input signals vector to its corresponding model at the sampling instants and the sensor measures the output signals vector. The iterative learning law is processed in a controller located far away of the models so the control signals vector has to be transmitted from the controller to the actuators through transmission channels. Such a law uses the measurements of each model to generate the input vector to be applied to its subsequent model so the measurements of the models have to be transmitted from the sensors to the controller. All transmissions are subject to failures which are described as a binary sequence taking value 1 or 0. A compensation dropout technique is used to replace the lost data in the transmission processes. The convergence to zero of the errors between the output signals vector and a reference one is achieved as the number of models tends to infinity.

Robust Sliding Control of SEIR Epidemic Models

This paper is aimed at designing a robust vaccination strategy capable of eradicating an infectious disease from a population regardless of the potential uncertainty in the parameters defining the disease. For this purpose, a control theoretic approach based on a sliding-mode control law is used. Initially, the controller is designed assuming certain knowledge of an upper-bound of the uncertainty signal. Afterwards, this condition is removed while an adaptive sliding control system is designed. The closed-loop properties are proved mathematically in the nonadaptive and adaptive cases. Furthermore, the usual sign function appearing in the sliding-mode control is substituted by the saturation function in order to prevent chattering. In addition, the properties achieved by the closed-loop system under this variation are also stated and proved analytically. The closed-loop system is able to attain the control objective regardless of the parametric uncertainties of the model and the lack of a priori knowledge on the system.