About stabilization of non-minimum phase systems by output feedback

This thesis work has been motivated by an internal benchmark dealing with the output regulation problem of a nonlinear non-minimum phase system in the case of full-state feedback. The system under consideration structurally suffers from finite escape time, and this condition makes the output regulation problem very hard even for very simple steady-state evolution or exosystem dynamics, such as a simple integrator. This situation leads to studying the approaches developed for controlling Non-minimum phase systems and how they affect feedback performances. Despite a lot of frequency domain results, only a few works have been proposed for describing the performance limitations in a state space system representation. In particular, in our opinion, the most relevant research thread exploits the so-called Inner-Outer Decomposition. Such decomposition allows splitting the Non-minimum phase system under consideration into a cascade of two subsystems: a minimum phase system (the outer) that contains all poles of the original system and an all-pass Non-minimum phase system (the inner) that contains all the unavoidable pathologies of the unstable zero dynamics. Such a cascade decomposition was inspiring to start working on functional observers for linear and nonlinear systems. In particular, the idea of a functional observer is to exploit only the measured signals from the system to asymptotically reconstruct a certain function of the system states, without necessarily reconstructing the whole state vector. The feature of asymptotically reconstructing a certain state functional plays an important role in the design of a feedback controller able to stabilize the Non-minimum phase system.

Distributed optimization and games over networks: a system theoretical perspective

Several decision and control tasks involve networks of cyber-physical systems that need to be coordinated and controlled according to a fully-distributed paradigm involving only local communications without any central unit. This thesis focuses on distributed optimization and games over networks from a system theoretical perspective. In the addressed frameworks, we consider agents communicating only with neighbors and running distributed algorithms with optimization-oriented goals. The distinctive feature of this thesis is to interpret these algorithms as dynamical systems and, thus, to resort to powerful system theoretical tools for both their analysis and design. We first address the so-called consensus optimization setup. In this context, we provide an original system theoretical analysis of the well-known Gradient Tracking algorithm in the general case of nonconvex objective functions. Then, inspired by this method, we provide and study a series of extensions to improve the performance and to deal with more challenging settings like, e.g., the derivative-free framework or the online one. Subsequently, we tackle the recently emerged framework named distributed aggregative optimization. For this setup, we develop and analyze novel schemes to handle (i) online instances of the problem, (ii) ``personalized'' optimization frameworks, and (iii) feedback optimization settings. Finally, we adopt a system theoretical approach to address aggregative games over networks both in the presence or absence of linear coupling constraints among the decision variables of the players. In this context, we design and inspect novel fully-distributed algorithms, based on tracking mechanisms, that outperform state-of-the-art methods in finding the Nash equilibrium of the game.

Unsupervised reinforcement learning via state entropy maximization

Reinforcement Learning (RL) provides a powerful framework to address sequential decision-making problems in which the transition dynamics is unknown or too complex to be represented. The RL approach is based on speculating what is the best decision to make given sample estimates obtained from previous interactions, a recipe that led to several breakthroughs in various domains, ranging from game playing to robotics. Despite their success, current RL methods hardly generalize from one task to another, and achieving the kind of generalization obtained through unsupervised pre-training in non-sequential problems seems unthinkable. Unsupervised RL has recently emerged as a way to improve generalization of RL methods. Just as its non-sequential counterpart, the unsupervised RL framework comprises two phases: An unsupervised pre-training phase, in which the agent interacts with the environment without external feedback, and a supervised fine-tuning phase, in which the agent aims to efficiently solve a task in the same environment by exploiting the knowledge acquired during pre-training. In this thesis, we study unsupervised RL via state entropy maximization, in which the agent makes use of the unsupervised interactions to pre-train a policy that maximizes the entropy of its induced state distribution. First, we provide a theoretical characterization of the learning problem by considering a convex RL formulation that subsumes state entropy maximization. Our analysis shows that maximizing the state entropy in finite trials is inherently harder than RL. Then, we study the state entropy maximization problem from an optimization perspective. Especially, we show that the primal formulation of the corresponding optimization problem can be (approximately) addressed through tractable linear programs. Finally, we provide the first practical methodologies for state entropy maximization in complex domains, both when the pre-training takes place in a single environment as well as multiple environments.