Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

Consensus in non-commutative spaces

Convergence analysis of consensus algorithms is revisited in the light of the Hilbert distance. The Lyapunov function used in the early analysis by Tsitsiklis is shown to be the Hilbert distance to consensus in log coordinates. Birkhoff theorem, which proves contraction of the Hilbert metric for any positive homogeneous monotone map, provides an early yet general convergence result for consensus algorithms. Because Birkhoff theorem holds in arbitrary cones, we extend consensus algorithms to the cone of positive definite matrices. The proposed generalization finds applications in the convergence analysis of quantum stochastic maps, which are a generalization of stochastic maps to non-commutative probability spaces. ©2010 IEEE.

Spatial models of bistability in biological collectives

We explore collective behavior in biological systems using a cooperative control framework. In particular, we study a hysteresis phenomenon in which a collective switches from circular to parallel motion under slow variation of the neighborhood size in which individuals tend to align with one another. In the case that the neighborhood radius is less than the circular motion radius, both circular and parallel motion can occur. We provide Lyapunov-based analysis of bistability of circular and parallel motion in a closed-loop system of self-propelled particles with coupled-oscillator dynamics. ©2007 IEEE.

Stabilization of planar collective motion: All-to-all communication

This paper proposes a design methodology to stabilize isolated relative equilibria in a model of all-to-all coupled identical particles moving in the plane at unit speed. Isolated relative equilibria correspond to either parallel motion of all particles with fixed relative spacing or to circular motion of all particles with fixed relative phases. The stabilizing feedbacks derive from Lyapunov functions that prove exponential stability and suggest almost global convergence properties. The results of the paper provide a low-order parametric family of stabilizable collectives that offer a set of primitives for the design of higher-level tasks at the group level. © 2007 IEEE.

Oscillator models and collective motion: Splay state stabilization of self-propelled particles

This paper presents a Lyapunov design for the stabilization of collective motion in a planar kinematic model of N particles moving at constant speed. We derive a control law that achieves asymptotic stability of the splay state formation, characterized by uniform rotation of N evenly spaced particles on a circle. In designing the control law, the particle headings are treated as a system of coupled phase oscillators. The coupling function which exponentially stabilizes the splay state of particle phases is combined with a decentralized beacon control law that stabilizes circular motion of the particles. © 2005 IEEE.

Stabilization and disturbance attenuation of nonlinear systems using dissipativity theory

In this paper, we survey some recent results on stabilization and disturbance attenuation for nonlinear systems using a dissipativity approach. After reviewing the basic dissipativity concept, we stress the connections between Lyapunov designs and the problem of achieving passivity by feedback. Focusing on physical models, we then illustrate how the design of stabilizing feedback can take advantage of the natural energy balance equation of the system. Here stabilization is viewed as the task of shaping the energy of the system to enforce a minimum at the desired equilibrium. Finally, we show the implications of dissipativity theory as an appropriate framework to study the nonlinear H∞ control problem. © 2002 EUCA.

CLF based designs with robustness to dynamic input uncertainties

The problem of robust stabilization of nonlinear systems in the presence of input uncertainties is of great importance in practical implementation. Stabilizing control laws may not be robust to this type of uncertainty, especially if cancellation of nonlinearities is used in the design. By exploiting a connection between robustness and optimality, "domination redesign" of the control Lyapunov function (CLF) based Sontag's formula has been shown to possess robustness to static and dynamic input uncertainties. In this paper we provide a sufficient condition for the domination redesign to apply. This condition relies on properties of local homogeneous approximations of the system and of the CLF. We show that an inverse optimal control law may not exist when these conditions are violated and illustrate how these conditions may guide the choice of a CLF which is suitable for domination redesign. © 1999 Elsevier Science B.V. All rights reserved.

Asymptotic stability for time-variant systems and observability: Uniform and nonuniform criteria

This paper presents some new criteria for uniform and nonuniform asymptotic stability of equilibria for time-variant differential equations and this within a Lyapunov approach. The stability criteria are formulated in terms of certain observability conditions with the output derived from the Lyapunov function. For some classes of systems, this system theoretic interpretation proves to be fruitful since - after establishing the invariance of observability under output injection - this enables us to check the stability criteria on a simpler system. This procedure is illustrated for some classical examples.

Constructive Nonlinear Control

In this book several streams of nonlinear control theory are merged and di- rected towards a constructive solution of the feedback stabilization problem. Analytic, geometric and asymptotic concepts are assembled as design tools for a wide variety of nonlinear phenomena and structures. Di®erential-geometric concepts reveal important structural properties of nonlinear systems, but al- low no margin for modeling errors. To overcome this de¯ciency, we combine them with analytic concepts of passivity, optimality and Lyapunov stability. In this way geometry serves as a guide for construction of design procedures, while analysis provides robustness tools which geometry lacks.

Adaptive combustion instability control with saturation

The control of a class of combustion systems, suceptible to damage from self-excited combustion oscillations, is considered. An adaptive stable controller, called Self-Tuning Regulator (STR), has recently been developed, which meets the apparently contradictory challenge of relying as little as possible on a particular combustion model while providing some guarantee that the controller will cause no harm. The controller injects some fuel unsteadily into the burning region, thereby altering the heat release, in response to an input signal detecting the oscillation. This paper focuses on an extension of the STR design, when, due to stringent emission requirements and to the danger of flame extension, the amount of fuel used for control is limited in amplitude. A Lyapunov stability analysis is used to prove the stability of the modified STR when the saturation constraint is imposed. The practical implementation of the modified STR remains straightforward, and simulation results, based on the nonlinear premixed flame model developed by Dowling, show that in the presence of a saturation constraint, the self-excited oscillations are damped more rapidly with the modified STR than with the original STR. © 2001 by S. Evesque. Published by the American Institute of Aeronautics and Astronautics, Inc.