Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

Lyapunov's stability and operational margin of grid-connected photovoltaic modules

Analyses of photovoltaic power generation based on Lyapunov's theorems are presented. The characteristics of the photovoltaic module and the power conditioning unit are analyzed in order to establish energy functions that assess the stability of solutions and define safe regions of operation. Furthermore, it is shown that grid-connected photovoltaic modules driven at maximum power may become unstable under normal grid transients. In such cases, stability can be maintained by allowing an operational margin defined as the energy difference between the stable and the unstable solutions of the system. Simulations show that modules cope well with grid transients when a sufficiently large margin is used.

Achieving Csiszár's source-channel coding exponent with product distributions

We derive a random-coding upper bound on the average probability of error of joint source-channel coding that recovers Csiszár's error exponent when used with product distributions over the channel inputs. Our proof technique for the error probability analysis employs a code construction for which source messages are assigned to subsets and codewords are generated with a distribution that depends on the subset. © 2012 IEEE.

Lyapunov-like conditions for the existence of zeno behavior in hybrid and lagrangian hybrid systems

Lyapunov-like conditions that utilize generalizations of energy and barrier functions certifying Zeno behavior near Zeno equilibria are presented. To better illustrate these conditions, we will study them in the context of Lagrangian hybrid systems. Through the observation that Lagrangian hybrid systems with isolated Zeno equilibria must have a onedimensional configuration space, we utilize our Lyapunov-like conditions to obtain easily verifiable necessary and sufficient conditions for the existence of Zeno behavior in systems of this form. © 2007 IEEE.

Lyapunov theory for zeno stability

Zeno behavior is a dynamic phenomenon unique to hybrid systems in which an infinite number of discrete transitions occurs in a finite amount of time. This behavior commonly arises in mechanical systems undergoing impacts and optimal control problems, but its characterization for general hybrid systems is not completely understood. The goal of this paper is to develop a stability theory for Zeno hybrid systems that parallels classical Lyapunov theory; that is, we present Lyapunov-like sufficient conditions for Zeno behavior obtained by mapping solutions of complex hybrid systems to solutions of simpler Zeno hybrid systems defined on the first quadrant of the plane. These conditions are applied to Lagrangian hybrid systems, which model mechanical systems undergoing impacts, yielding simple sufficient conditions for Zeno behavior. Finally, the results are applied to robotic bipedal walking. © 2012 IEEE.