Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

Space experiments of thermocapillary convection in two-liquid layers

In 1999, the space experiments on the Marangoni convection and thermocapillary convection in a system of two immiscible liquid layers in microgravity environment were conducted on board the Chinese scientific satellite SJ-5. A new system of two-layer liquids such as FC-70 liquid and paraffin was used successfully, with the paraffin melted in the space. Two different test-cells are subjected to a temperature gradient perpendicular or parallel to the interface to study the Marangoni convection and thermocapillary convection, respectively. The experimental data obtained in the first Chinese space experiment of fluid are presented. Two-dimensional numerical simulations of thermocapillary convections are carried out using SIMPLEC method A reasonable agreement between the experimental investigation and the numerical results is obtained.

An overview of sequential Monte Carlo methods for parameter estimation in general state-space models

Nonlinear non-Gaussian state-space models arise in numerous applications in control and signal processing. Sequential Monte Carlo (SMC) methods, also known as Particle Filters, are numerical techniques based on Importance Sampling for solving the optimal state estimation problem. The task of calibrating the state-space model is an important problem frequently faced by practitioners and the observed data may be used to estimate the parameters of the model. The aim of this paper is to present a comprehensive overview of SMC methods that have been proposed for this task accompanied with a discussion of their advantages and limitations.

Finding space to mental health : promoting mental health in adolescents (12 to 14 year olds): preliminary results of the intervention's effectiveness

Resumen: La salud mental y el bienestar son fundamentales para nuestra capacidad colectiva y individual como seres humanos de pensar, de exteriorizar los sentimientos, de establecer y mantener relaciones, para estudiar, para perseguir las actividades de ocio, para tomar decisiones diarias y para disfrutar de una vida plena. Una adolescencia saludable es un prerrequisito para una vida adulta saludable. Sin embargo, la realidad actual presenta un panorama preocupante. La formación del capital mental individual y colectivo - especialmente en las primeras etapas de la vida - está siendo retenida por una serie de riesgos evitables para la salud mental (World Health Organization [WHO], 2013). Los adolescentes del sur de Europa (región que ha sido más severamente afectada por la crisis financiera; e.g., Portugal) son señalados como un grupo extremadamente vulnerable, ya que su salud mental fácilmente podría ser influenciada por las dificultades económicas de sus padres y la escasez de solidaridad social (European Parliament, 2012). La promoción de la salud mental de los adolescentes es considerada como una preocupación fundamental (WHO, 2005a, 2013). En este ámbito, las intervenciones centradas en la promoción de la literacía de la salud mental han revelado importantes ventajas en la prevención, reconocimiento, intervención precoz y la reducción del estigma (Pinfold, Stuart, Thornicroft & Arboleda-Florez, 2005; Pinfold, Toulmin, Thornicroft, Huxley, Farmer & Graham, 2003; Schulze, Richter-Werling, Matschinger & Angermeyer, 2003; Stuart, 2006). En consonancia con los marcos de promoción de la salud mentales propuestos por la Organización Mundial de la Salud (2005a), tenemos que involucrar a jóvenes en los ambientes donde interactúan (Burns, 2011). Las escuelas son implícitamente uno de los locales más importantes para la promoción de la salud mental de los adolescentes (Barry, Clarke, Jenkins & Patel, 2013; WHO, 2001). El proyecto “Abrir Espacio para la Salud Mental – Promoción de la salud mental en adolescentes (12-14 años)” tiene como objetivo incrementar literacía de la salud mental en los jóvenes. En el primer año se ha desarrollado un instrumento de evaluación - Mental Health Literacy questionnaire (MHLq) - y la intervención para la promoción de la salud mental. La intervención consiste en 2 sesiones, 90 minutos cada una, implementadas con intervalo de una semana. Siguen una metodología interactiva, utilizando dinámicas de grupo, videos, música y discusión. El estudio de la eficacia de la intervención se lleva a cabo mediante un análisis pre y pos-test con el MHLq, utilizando un grupo experimental y un grupo de control. Este artículo presenta los resultados preliminares de la eficacia de la intervención de promoción de la salud mental en una muestra de 100 adolescentes portugueses (12-14 años). El pos-test mostró un incremento de los niveles de conocimientos de salud mental y estrategias de autoayuda. Los resultados sugieren que la intervención desarrollada parece ser adecuada al objetivo propuesto y refuerzan la creencia de que intervenciones escolares, sistemáticas y sostenibles, para la promoción de la salud mental con jóvenes, es un enfoque prometedor para la promoción de la literacía de la salud mental (Schulze et al., 2003; Rickwood et al., 2005; Corrigan et al., 2007; WHO, 2010).

Sequential Monte Carlo computation of the score and observed information matrix in state-space models with application to parameter estimation

Sequential Monte Carlo (SMC) methods are popular computational tools for Bayesian inference in non-linear non-Gaussian state-space models. For this class of models, we propose SMC algorithms to compute the score vector and observed information matrix recursively in time. We propose two different SMC implementations, one with computational complexity $\mathcal{O}(N)$ and the other with complexity $\mathcal{O}(N^{2})$ where $N$ is the number of importance sampling draws. Although cheaper, the performance of the $\mathcal{O}(N)$ method degrades quickly in time as it inherently relies on the SMC approximation of a sequence of probability distributions whose dimension is increasing linearly with time. In particular, even under strong \textit{mixing} assumptions, the variance of the estimates computed with the $\mathcal{O}(N)$ method increases at least quadratically in time. The $\mathcal{O}(N^{2})$ is a non-standard SMC implementation that does not suffer from this rapid degrade. We then show how both methods can be used to perform batch and recursive parameter estimation.