Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

RANS-URANS in axial compressor, a design methodology

Turbomachinery flows are inherently unsteady. Until now during the design process, unsteadiness has been neglected, with resort merely to steady numerical simulations. Despite the assumption involved, the results obtained with steady simulations have been used with success. One of the questions arising in recent years is can unsteady simulations be used to improve the design of turbomachines? In this work the numerical simulation of a multi-stage axial compressor is carried out. Comparison of Reynolds averaged Navier-Stokes (RANS) and unsteady Reynolds averaged Navier-Stokes (URANS) calculation shows that the unsteadiness affects pressure losses and the prediction of stall limit. The unsteady inflow due to the wake passing mainly modifies the losses and whirl angle near the endwalls. The computational cost of the fully unsteady compared with a steady simulation is about four times in terms of mesh dimension and two orders of magnitude as number of iterations. A mixed RANS-URANS solution has been proposed to give the designer the possibility to simulate an unsteady stage embedded in a steady-state simulation. This method has been applied to the simulation of a four-stage axial compressor rig. The mixed RANS-URANS approach has been developed using sliding and mixing planes as interface conditions. The rotor-stator interaction has been captured physically while reducing the computational time and mesh size.

Beyond 2D-grids: A dependence maximization view on image browsing

Ideally, one would like to perform image search using an intuitive and friendly approach. Many existing image search engines, however, present users with sets of images arranged in some default order on the screen, typically the relevance to a query, only. While this certainly has its advantages, arguably, a more flexible and intuitive way would be to sort images into arbitrary structures such as grids, hierarchies, or spheres so that images that are visually or semantically alike are placed together. This paper focuses on designing such a navigation system for image browsers. This is a challenging task because arbitrary layout structure makes it difficult - if not impossible - to compute cross-similarities between images and structure coordinates, the main ingredient of traditional layouting approaches. For this reason, we resort to a recently developed machine learning technique: kernelized sorting. It is a general technique for matching pairs of objects from different domains without requiring cross-domain similarity measures and hence elegantly allows sorting images into arbitrary structures. Moreover, we extend it so that some images can be preselected for instance forming the tip of the hierarchy allowing to subsequently navigate through the search results in the lower levels in an intuitive way. Copyright 2010 ACM.