Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent's local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

An approach version of fuzzy metric spaces including an ad hoc fixed point theorem

In this paper, inspired by two very different, successful metric theories such us the real view-point of Lowen's approach spaces and the probabilistic field of Kramosil and Michalek's fuzzymetric spaces, we present a family of spaces, called fuzzy approach spaces, that are appropriate to handle, at the same time, both measure conceptions. To do that, we study the underlying metric interrelationships between the above mentioned theories, obtaining six postulates that allow us to consider such kind of spaces in a unique category. As a result, the natural way in which metric spaces can be embedded in both classes leads to a commutative categorical scheme. Each postulate is interpreted in the context of the study of the evolution of fuzzy systems. First properties of fuzzy approach spaces are introduced, including a topology. Finally, we describe a fixed point theorem in the setting of fuzzy approach spaces that can be particularized to the previous existing measure spaces.