Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

Supporting excavations in clay - From analysis to decision-making

Finite Element Analysis (FEA) is used to calibrate a decision-making tool based on an extension of the Mobilized Strength Design (MSD) method which permits the designer an extremely simple method of predicting ground displacements during construction. This newly extended MSD approach accommodates a number of issues which are important in underground construction between in-situ walls, including: alternative base heave mechanisms suitable either for wide excavations in relatively shallow soft clay strata, or narrow excavations in relatively deep soft strata; the influence of support system stiffness in relation to the sequence of propping of the wall; and the capability of dealing with stratified ground. These developments should make it possible for a design engineer to take informed decisions on the relationship between prop spacing and ground movements, or the influence of wall stiffness, or on the need for and influence of a jet-grouted base slab, for example, without having to conduct project-specific FEA. © 2009 Taylor & Francis Group.

Proposal of a method to clarify and enhance decision-making at the front-end of innovation

Decision-making at the front-end of innovation is critical for the success of companies. This paper presents a simple visual method, called DMCA (Decision-Making Criteria Assessment), which was created to clarify and improve decision-making at the front-end of innovation. The method maps the uncertainty of project information and importance of decision criteria, compiling a measure that indicates whether the decision is highly uncertain, what information interferes with it, and what criteria are actually being considered. The DMCA method was tested in two projects that faced decision-making issues, and the results confirm the benefits of using this method in decision-making at the front-end. © 2012 IEEE.