Probabilistic inference for fast learning in control

A novel framework is provided for very fast model-based reinforcement learning in continuous state and action spaces. It requires probabilistic models that explicitly characterize their levels of condence. Within the framework, exible, non-parametric models are used to describe the world based on previously collected experience. It demonstrates learning on the cart-pole problem in a setting where very limited prior knowledge about the task has been provided. Learning progressed rapidly, and a good policy found after only a small number of iterations.

Bayesian non-parametrics and the probabilistic approach to modelling.

Modelling is fundamental to many fields of science and engineering. A model can be thought of as a representation of possible data one could predict from a system. The probabilistic approach to modelling uses probability theory to express all aspects of uncertainty in the model. The probabilistic approach is synonymous with Bayesian modelling, which simply uses the rules of probability theory in order to make predictions, compare alternative models, and learn model parameters and structure from data. This simple and elegant framework is most powerful when coupled with flexible probabilistic models. Flexibility is achieved through the use of Bayesian non-parametrics. This article provides an overview of probabilistic modelling and an accessible survey of some of the main tools in Bayesian non-parametrics. The survey covers the use of Bayesian non-parametrics for modelling unknown functions, density estimation, clustering, time-series modelling, and representing sparsity, hierarchies, and covariance structure. More specifically, it gives brief non-technical overviews of Gaussian processes, Dirichlet processes, infinite hidden Markov models, Indian buffet processes, Kingman's coalescent, Dirichlet diffusion trees and Wishart processes.

Modelling within-host spatiotemporal dynamics of invasive bacterial disease

Mechanistic determinants of bacterial growth, death, and spread within mammalian hosts cannot be fully resolved studying a single bacterial population. They are also currently poorly understood. Here, we report on the application of sophisticated experimental approaches to map spatiotemporal population dynamics of bacteria during an infection. We analyzed heterogeneous traits of simultaneous infections with tagged Salmonella enterica populations (wild-type isogenic tagged strains [WITS]) in wild-type and gene-targeted mice. WITS are phenotypically identical but can be distinguished and enumerated by quantitative PCR, making it possible, using probabilistic models, to estimate bacterial death rate based on the disappearance of strains through time. This multidisciplinary approach allowed us to establish the timing, relative occurrence, and immune control of key infection parameters in a true host-pathogen combination. Our analyses support a model in which shortly after infection, concomitant death and rapid bacterial replication lead to the establishment of independent bacterial subpopulations in different organs, a process controlled by host antimicrobial mechanisms. Later, decreased microbial mortality leads to an exponential increase in the number of bacteria that spread locally, with subsequent mixing of bacteria between organs via bacteraemia and further stochastic selection. This approach provides us with an unprecedented outlook on the pathogenesis of S. enterica infections, illustrating the complex spatial and stochastic effects that drive an infectious disease. The application of the novel method that we present in appropriate and diverse host-pathogen combinations, together with modelling of the data that result, will facilitate a comprehensive view of the spatial and stochastic nature of within-host dynamics. © 2008 Grant et al.

Elliptical slice sampling

Many probabilistic models introduce strong dependencies between variables using a latent multivariate Gaussian distribution or a Gaussian process. We present a new Markov chain Monte Carlo algorithm for performing inference in models with multivariate Gaussian priors. Its key properties are: 1) it has simple, generic code applicable to many models, 2) it has no free parameters, 3) it works well for a variety of Gaussian process based models. These properties make our method ideal for use while model building, removing the need to spend time deriving and tuning updates for more complex algorithms.

Bayesian Active Learning for Classification and Preference Learning

Information theoretic active learning has been widely studied for probabilistic models. For simple regression an optimal myopic policy is easily tractable. However, for other tasks and with more complex models, such as classification with nonparametric models, the optimal solution is harder to compute. Current approaches make approximations to achieve tractability. We propose an approach that expresses information gain in terms of predictive entropies, and apply this method to the Gaussian Process Classifier (GPC). Our approach makes minimal approximations to the full information theoretic objective. Our experimental performance compares favourably to many popular active learning algorithms, and has equal or lower computational complexity. We compare well to decision theoretic approaches also, which are privy to more information and require much more computational time. Secondly, by developing further a reformulation of binary preference learning to a classification problem, we extend our algorithm to Gaussian Process preference learning.

Confidence in beliefs about pain predicts expectancy effects on pain perception and anticipatory processing in right anterior insula.

Psychological factors play a major role in exacerbating chronic pain. Effective self-management of pain is often hindered by inaccurate beliefs about the nature of pain which lead to a high degree of emotional reactivity. Probabilistic models of perception state that greater confidence (certainty) in beliefs increases their influence on perception and behavior. In this study, we treat confidence as a metacognitive process dissociable from the content of belief. We hypothesized that confidence is associated with anticipatory activation of areas of the pain matrix involved with top-down modulation of pain. Healthy volunteers rated their beliefs about the emotional distress that experimental pain would cause, and separately rated their level of confidence in this belief. Confidence predicted the influence of anticipation cues on experienced pain. We measured brain activity during anticipation of pain using high-density EEG and used electromagnetic tomography to determine neural substrates of this effect. Confidence correlated with activity in right anterior insula, posterior midcingulate and inferior parietal cortices during the anticipation of pain. Activity in the right anterior insula predicted a greater influence of anticipation cues on pain perception, whereas activity in right inferior parietal cortex predicted a decreased influence of anticipatory cues. The results support probabilistic models of pain perception and suggest that confidence in beliefs is an important determinant of expectancy effects on pain perception.

Probabilistic graphical models for semi-supervised traffic classification

Traffic classification using machine learning continues to be an active research area. The majority of work in this area uses off-the-shelf machine learning tools and treats them as black-box classifiers. This approach turns all the modelling complexity into a feature selection problem. In this paper, we build a problem-specific solution to the traffic classification problem by designing a custom probabilistic graphical model. Graphical models are a modular framework to design classifiers which incorporate domain-specific knowledge. More specifically, our solution introduces semi-supervised learning which means we learn from both labelled and unlabelled traffic flows. We show that our solution performs competitively compared to previous approaches while using less data and simpler features. Copyright © 2010 ACM.

Handling uncertainty in housing stock models

Housing stock models can be useful tools in helping to assess the environmental and socio-economic impacts of retrofits to residential buildings; however, existing housing stock models are not able to quantify the uncertainties that arise in the modelling process from various sources, thus limiting the role that they can play in helping decision makers. This paper examines the different sources of uncertainty involved in housing stock models and proposes a framework for handling these uncertainties. This framework involves integrating probabilistic sensitivity analysis with a Bayesian calibration process in order to quantify uncertain parameters more accurately. The proposed framework is tested on a case study building, and suggestions are made on how to expand the framework for retrofit analysis at an urban-scale. © 2011 Elsevier Ltd.

Thermal building simulation using the UKCP09 probabilistic climate projections

This article investigates how to use UK probabilistic climate-change projections (UKCP09) in rigorous building energy analysis. Two office buildings (deep plan and shallow plan) are used as case studies to demonstrate the application of UKCP09. Three different methods for reducing the computational demands are explored: statistical reduction (Finkelstein-Schafer [F-S] statistics), simplification using degree-day theory and the use of metamodels. The first method, which is based on an established technique, can be used as reference because it provides the most accurate information. However, it is necessary to automatically choose weather files based on F-S statistic by using computer programming language because thousands of weather files created from UKCP09 weather generator need to be processed. A combination of the second (degree-day theory) and third method (metamodels) requires only a relatively small number of simulation runs, but still provides valuable information to further implement the uncertainty and sensitivity analyses. The article also demonstrates how grid computing can be used to speed up the calculation for many independent EnergyPlus models by harnessing the processing power of idle desktop computers. © 2011 International Building Performance Simulation Association (IBPSA).

Probabilistic amplitude and frequency demodulation

A number of recent scientific and engineering problems require signals to be decomposed into a product of a slowly varying positive envelope and a quickly varying carrier whose instantaneous frequency also varies slowly over time. Although signal processing provides algorithms for so-called amplitude-and frequency-demodulation (AFD), there are well known problems with all of the existing methods. Motivated by the fact that AFD is ill-posed, we approach the problem using probabilistic inference. The new approach, called probabilistic amplitude and frequency demodulation (PAFD), models instantaneous frequency using an auto-regressive generalization of the von Mises distribution, and the envelopes using Gaussian auto-regressive dynamics with a positivity constraint. A novel form of expectation propagation is used for inference. We demonstrate that although PAFD is computationally demanding, it outperforms previous approaches on synthetic and real signals in clean, noisy and missing data settings.