Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent's local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

Stability and Limit Oscillations of a Control Event-Based Sampling Criterion

This paper investigates the presence of limit oscillations in an adaptive sampling system. The basic sampling criterion operates in the sense that each next sampling occurs when the absolute difference of the signal amplitude with respect to its currently sampled signal equalizes a prescribed threshold amplitude. The sampling criterion is extended involving a prescribed set of amplitudes. The limit oscillations might be interpreted through the equivalence of the adaptive sampling and hold device with a nonlinear one consisting of a relay with multiple hysteresis whose parameterization is, in general, dependent on the initial conditions of the dynamic system. The performed study is performed on the time domain.

Stability of Switched Feedback Time-Varying Dynamic Systems Based on the Properties of the Gap Metric for Operators

The stabilization of dynamic switched control systems is focused on and based on an operator-based formulation. It is assumed that the controlled object and the controller are described by sequences of closed operator pairs (L, C) on a Hilbert space H of the input and output spaces and it is related to the existence of the inverse of the resulting input-output operator being admissible and bounded. The technical mechanism addressed to get the results is the appropriate use of the fact that closed operators being sufficiently close to bounded operators, in terms of the gap metric, are also bounded. That philosophy is followed for the operators describing the input-output relations in switched feedback control systems so as to guarantee the closed-loop stabilization.

Robust Sliding Mode Control for Tokamaks

Nuclear fusion has arisen as an alternative energy to avoid carbon dioxide emissions, being the tokamak a promising nuclear fusion reactor that uses a magnetic field to confine plasma in the shape of a torus. However, different kinds of magnetohydrodynamic instabilities may affect tokamak plasma equilibrium, causing severe reduction of particle confinement and leading to plasma disruptions. In this sense, numerous efforts and resources have been devoted to seeking solutions for the different plasma control problems so as to avoid energy confinement time decrements in these devices. In particular, since the growth rate of the vertical instability increases with the internal inductance, lowering the internal inductance is a fundamental issue to address for the elongated plasmas employed within the advanced tokamaks currently under development. In this sense, this paper introduces a lumped parameter numerical model of the tokamak in order to design a novel robust sliding mode controller for the internal inductance using the transformer primary coil as actuator.

Advances towards behaviour-based indoor robotic exploration

215 p.