Modelling and control of nonlinear systems using Gaussian processes with partial model information

Gaussian processes are gaining increasing popularity among the control community, in particular for the modelling of discrete time state space systems. However, it has not been clear how to incorporate model information, in the form of known state relationships, when using a Gaussian process as a predictive model. An obvious example of known prior information is position and velocity related states. Incorporation of such information would be beneficial both computationally and for faster dynamics learning. This paper introduces a method of achieving this, yielding faster dynamics learning and a reduction in computational effort from O(Dn2) to O((D - F)n2) in the prediction stage for a system with D states, F known state relationships and n observations. The effectiveness of the method is demonstrated through its inclusion in the PILCO learning algorithm with application to the swing-up and balance of a torque-limited pendulum and the balancing of a robotic unicycle in simulation. © 2012 IEEE.

Gaussian Processes for Machine Learning

The code provided here originally demonstrated the main algorithms from Rasmussen and Williams: Gaussian Processes for Machine Learning. It has since grown to allow more likelihood functions, further inference methods and a flexible framework for specifying GPs.

Modulation of pain ratings by expectation and uncertainty: Behavioral characteristics and anticipatory neural correlates.

Expectations about the magnitude of impending pain exert a substantial effect on subsequent perception. However, the neural mechanisms that underlie the predictive processes that modulate pain are poorly understood. In a combined behavioral and high-density electrophysiological study we measured anticipatory neural responses to heat stimuli to determine how predictions of pain intensity, and certainty about those predictions, modulate brain activity and subjective pain ratings. Prior to receiving randomized laser heat stimuli at different intensities (low, medium or high) subjects (n=15) viewed cues that either accurately informed them of forthcoming intensity (certain expectation) or not (uncertain expectation). Pain ratings were biased towards prior expectations of either high or low intensity. Anticipatory neural responses increased with expectations of painful vs. non-painful heat intensity, suggesting the presence of neural responses that represent predicted heat stimulus intensity. These anticipatory responses also correlated with the amplitude of the Laser-Evoked Potential (LEP) response to painful stimuli when the intensity was predictable. Source analysis (LORETA) revealed that uncertainty about expected heat intensity involves an anticipatory cortical network commonly associated with attention (left dorsolateral prefrontal, posterior cingulate and bilateral inferior parietal cortices). Relative certainty, however, involves cortical areas previously associated with semantic and prospective memory (left inferior frontal and inferior temporal cortex, and right anterior prefrontal cortex). This suggests that biasing of pain reports and LEPs by expectation involves temporally precise activity in specific cortical networks.

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

Anxiety reduction through detachment: subjective, physiological, and neural effects.

The ability to volitionally regulate emotions helps to adapt behavior to changing environmental demands and can alleviate subjective distress. We show that a cognitive strategy of detachment attenuates subjective and physiological measures of anticipatory anxiety for pain and reduces reactivity to receipt of pain itself. Using functional magnetic resonance imaging, we locate the potential site and source of this modulation of anticipatory anxiety in the medial prefrontal/anterior cingulate and anterolateral prefrontal cortex, respectively.

Foresight vehicle programme - Customer understanding processes in design

Customer feedback is normally fed into product design and engineering via quality surveys and therefore mainly comprises negative comments: complaints about things gone wrong. Whilst eradication of such problems will result in a feeling of satisfaction in existing customers, it will not instil the sense of delight required to attract conquest buyers. CUPID's aim is to conceive and evaluate ideas to stimulate product desirability through the provision of delightful features and execution. By definition, surprise and delight features cannot be foreseen, so we have to understand sensory appeal and, therefore, the "hidden" voice of the customer. Copyright © 2002 Society of Automotive Engineers, Inc.

Bayesian Learning of Noisy Markov Decision Processes

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Change prediction using interface data

Managing change can be challenging due to the high levels of interdependency in concurrent engineering processes. A key activity in engineering change management is propagation analysis, which can be supported using the change prediction method. In common with most other change prediction approaches, the change prediction method has three important limitations: L1: it depends on highly subjective input data; L2: it is capable of modelling 'generalised cases' only and cannot be; customised to assess specific changes; and L3: the input data are static, and thus, guidance does not reflect changes in the design. This article contributes to resolving these limitations by incorporating interface information into the change prediction method. The enhanced method is illustrated using an example based on a flight simulator. © The Author(s) 2013.

Bayesian Structured Prediction Using Gaussian Processes

We introduce a conceptually novel structured prediction model, GPstruct, which is kernelized, non-parametric and Bayesian, by design. We motivate the model with respect to existing approaches, among others, conditional random fields (CRFs), maximum margin Markov networks (M3N), and structured support vector machines (SVMstruct), which embody only a subset of its properties. We present an inference procedure based on Markov Chain Monte Carlo. The framework can be instantiated for a wide range of structured objects such as linear chains, trees, grids, and other general graphs. As a proof of concept, the model is benchmarked on several natural language processing tasks and a video gesture segmentation task involving a linear chain structure. We show prediction accuracies for GPstruct which are comparable to or exceeding those of CRFs and SVMstruct.

Nonequilibrium dynamics of photoexcited electrons in graphene: Collinear scattering, Auger processes, and the impact of screening

We present a combined analytical and numerical study of the early stages (sub-100-fs) of the nonequilibrium dynamics of photoexcited electrons in graphene. We employ the semiclassical Boltzmann equation with a collision integral that includes contributions from electron-electron (e-e) and electron-optical phonon interactions. Taking advantage of circular symmetry and employing the massless Dirac fermion (MDF) Hamiltonian, we are able to perform an essentially analytical study of the e-e contribution to the collision integral. This allows us to take particular care of subtle collinear scattering processes - processes in which incoming and outgoing momenta of the scattering particles lie on the same line - including carrier multiplication (CM) and Auger recombination (AR). These processes have a vanishing phase space for two-dimensional MDF bare bands. However, we argue that electron-lifetime effects, seen in experiments based on angle-resolved photoemission spectroscopy, provide a natural pathway to regularize this pathology, yielding a finite contribution due to CM and AR to the Coulomb collision integral. Finally, we discuss in detail the role of physics beyond the Fermi golden rule by including screening in the matrix element of the Coulomb interaction at the level of the random phase approximation (RPA), focusing in particular on the consequences of various approximations including static RPA screening, which maximizes the impact of CM and AR processes, and dynamical RPA screening, which completely suppresses them. © 2013 American Physical Society.

Determinantal Clustering Processes - A Nonparametric Bayesian Approach to Kernel Based Semi-Supervised Clustering

Semi-supervised clustering is the task of clustering data points into clusters where only a fraction of the points are labelled. The true number of clusters in the data is often unknown and most models require this parameter as an input. Dirichlet process mixture models are appealing as they can infer the number of clusters from the data. However, these models do not deal with high dimensional data well and can encounter difficulties in inference. We present a novel nonparameteric Bayesian kernel based method to cluster data points without the need to prespecify the number of clusters or to model complicated densities from which data points are assumed to be generated from. The key insight is to use determinants of submatrices of a kernel matrix as a measure of how close together a set of points are. We explore some theoretical properties of the model and derive a natural Gibbs based algorithm with MCMC hyperparameter learning. The model is implemented on a variety of synthetic and real world data sets.