A backward particle interpretation of Feynman-Kac formulae

We design a particle interpretation of Feynman-Kac measures on path spaces based on a backward Markovian representation combined with a traditional mean field particle interpretation of the flow of their final time marginals. In contrast to traditional genealogical tree based models, these new particle algorithms can be used to compute normalized additive functionals "on-the-fly" as well as their limiting occupation measures with a given precision degree that does not depend on the final time horizon. We provide uniform convergence results with respect to the time horizon parameter as well as functional central limit theorems and exponential concentration estimates. Our results have important consequences for online parameter estimation for non-linear non-Gaussian state-space models. We show how the forward filtering backward smoothing estimates of additive functionals can be computed using a forward only recursion.

Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

Pressure data analysis of formula one racing engines

The measured time-history of the cylinder pressure is the principal diagnostic in the analysis of processes within the combustion chamber. This paper defines, implements and tests a pressure analysis algorithm for a Formula One racing engine in MATLAB1. Evaluation of the software on real data is presented. The sensitivity of the model to the variability of burn parameter estimates is also discussed. Copyright © 1997 Society of Automotive Engineers, Inc.

Practical matching of principal stress field geometries in composite components

A simple composite design methodology has been developed from the basic principles of composite component failure. This design approach applies the principles of stress field matching to develop suitable reinforcement patterns around three-dimensional details such as lugs in mechanical components. The resulting patterns are essentially curvilinear orthogonal meshes, adjusted to meet the restrictions imposed by geometric restraints and the intended manufacturing process. Whilst the principles behind the design methodology can be applied to components produced by differing manufacturing processes, the results found from looking at simple generic example problems suggest a realistic and practical generic manufacturing approach. The underlying principles of the design methodology are described and simple analyses are used to help illustrate both the methodology and how such components behave. These analyses suggest it is possible to replace high-strength steel lugs with composite components whose strength-to-weight ratio is some 4-5 times better. © 1998 Elsevier Science Ltd. All rights reserved.

Bayesian model selection for time series using Markov chain Monte Carlo

We present a stochastic simulation technique for subset selection in time series models, based on the use of indicator variables with the Gibbs sampler within a hierarchical Bayesian framework. As an example, the method is applied to the selection of subset linear AR models, in which only significant lags are included. Joint sampling of the indicators and parameters is found to speed convergence. We discuss the possibility of model mixing where the model is not well determined by the data, and the extension of the approach to include non-linear model terms.

Nucleated polymerization with secondary pathways. I. Time evolution of the principal moments.

Self-assembly processes resulting in linear structures are often observed in molecular biology, and include the formation of functional filaments such as actin and tubulin, as well as generally dysfunctional ones such as amyloid aggregates. Although the basic kinetic equations describing these phenomena are well-established, it has proved to be challenging, due to their non-linear nature, to derive solutions to these equations except for special cases. The availability of general analytical solutions provides a route for determining the rates of molecular level processes from the analysis of macroscopic experimental measurements of the growth kinetics, in addition to the phenomenological parameters, such as lag times and maximal growth rates that are already obtainable from standard fitting procedures. We describe here an analytical approach based on fixed-point analysis, which provides self-consistent solutions for the growth of filamentous structures that can, in addition to elongation, undergo internal fracturing and monomer-dependent nucleation as mechanisms for generating new free ends acting as growth sites. Our results generalise the analytical expression for sigmoidal growth kinetics from the Oosawa theory for nucleated polymerisation to the case of fragmenting filaments. We determine the corresponding growth laws in closed form and derive from first principles a number of relationships which have been empirically established for the kinetics of the self-assembly of amyloid fibrils.

Parameter Estimation for Hidden Markov Models with Intractable Likelihoods

Approximate Bayesian computation (ABC) is a popular technique for analysing data for complex models where the likelihood function is intractable. It involves using simulation from the model to approximate the likelihood, with this approximate likelihood then being used to construct an approximate posterior. In this paper, we consider methods that estimate the parameters by maximizing the approximate likelihood used in ABC. We give a theoretical analysis of the asymptotic properties of the resulting estimator. In particular, we derive results analogous to those of consistency and asymptotic normality for standard maximum likelihood estimation. We also discuss how sequential Monte Carlo methods provide a natural method for implementing our likelihood-based ABC procedures.

Bayesian learning of noisy Markov decision processes

This work addresses the problem of estimating the optimal value function in a Markov Decision Process from observed state-action pairs. We adopt a Bayesian approach to inference, which allows both the model to be estimated and predictions about actions to be made in a unified framework, providing a principled approach to mimicry of a controller on the basis of observed data. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from theposterior distribution over the optimal value function. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.