Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

圆柱振荡绕流中涡旋运动模式(pattern)的数值模拟

提出并发展的一种基于区域分解思想,综合了解 N—S方程的有限差分法及涡法各自优点的新数值方法,计算了各种 Keulegan-Carpenter 数下(Kc=2～24)振荡流绕圆柱的流动。系统地研究了振荡流中涡旋运动模式随 Kc 数变化的规律,模拟了不对称区、单对涡区(或模向区)、双对涡区(或对角区)和三对涡区四种不同的涡旋运动模式。将计算所得的阻力系数 C_D、惯性系数 C_M与国外近期发表的计算结果进行了比较。

Laser-induced wavy pattern formation in metal thin films

Laser-induced well-ordered and controllable wavy patterns are constructed in the deposited metal thin film. The micrometer-sized structure and orientation of the wavy patterns can be controlled via scanning a different size of rectangle laser spot on the films. Ordered patterns such as aligned, crossed, and whirled wave structures were designed over large areas. This patterning technique may find applications in both exploring the reliability and mechanical properties of thin films, and fabricating microfluidic devices. (C) 2004 American Institute of Physics.

An overview of sequential Monte Carlo methods for parameter estimation in general state-space models

Nonlinear non-Gaussian state-space models arise in numerous applications in control and signal processing. Sequential Monte Carlo (SMC) methods, also known as Particle Filters, are numerical techniques based on Importance Sampling for solving the optimal state estimation problem. The task of calibrating the state-space model is an important problem frequently faced by practitioners and the observed data may be used to estimate the parameters of the model. The aim of this paper is to present a comprehensive overview of SMC methods that have been proposed for this task accompanied with a discussion of their advantages and limitations.

Sequential Monte Carlo computation of the score and observed information matrix in state-space models with application to parameter estimation

Sequential Monte Carlo (SMC) methods are popular computational tools for Bayesian inference in non-linear non-Gaussian state-space models. For this class of models, we propose SMC algorithms to compute the score vector and observed information matrix recursively in time. We propose two different SMC implementations, one with computational complexity $\mathcal{O}(N)$ and the other with complexity $\mathcal{O}(N^{2})$ where $N$ is the number of importance sampling draws. Although cheaper, the performance of the $\mathcal{O}(N)$ method degrades quickly in time as it inherently relies on the SMC approximation of a sequence of probability distributions whose dimension is increasing linearly with time. In particular, even under strong \textit{mixing} assumptions, the variance of the estimates computed with the $\mathcal{O}(N)$ method increases at least quadratically in time. The $\mathcal{O}(N^{2})$ is a non-standard SMC implementation that does not suffer from this rapid degrade. We then show how both methods can be used to perform batch and recursive parameter estimation.

Wave pattern characteristics of a two-phase nozzle flow by shock propagation

In this paper, the wave pattern characteristics of shock-induced two-phase nozzle Hows with the quiescent or moving dusty gas ahead of the incident-shock front has been investigated by using high-resolution numerical method. As compared with the corresponding results in single-phase nozzle flows of the pure gas, obvious differences between these two kinds of flows can be obtained.