Partially Observable Markov Decision Processes with continuous observations for dialogue management

This work shows how a dialogue model can be represented as a Partially Observable Markov Decision Process (POMDP) with observations composed of a discrete and continuous component. The continuous component enables the model to directly incorporate a confidence score for automated planning. Using a testbed simulated dialogue management problem, we show how recent optimization techniques are able to find a policy for this continuous POMDP which outperforms a traditional MDP approach. Further, we present a method for automatically improving handcrafted dialogue managers by incorporating POMDP belief state monitoring, including confidence score information. Experiments on the testbed system show significant improvements for several example handcrafted dialogue managers across a range of operating conditions.

A high performance oxygen storage material for chemical looping processes with CO2 capture

Chemical looping combustion (CLC) is a novel combustion technology that involves cyclic reduction and oxidation of oxygen storage materials to provide oxygen for the combustion of fuels to CO2 and H2O, whilst giving a pure stream of CO2 suitable for sequestration or utilisation. Here, we report a method for preparing of oxygen storage materials from layered double hydroxides (LDHs) precursors and demonstrate their applications in the CLC process. The LDHs precursor enables homogeneous mixing of elements at the molecular level, giving a high degree of dispersion and high-loading of active metal oxide in the support after calcination. Using a Cu-Al LDH precursor as a prototype, we demonstrate that rational design of oxygen storage materials by material chemistry significantly improved the reactivity and stability in the high temperature redox cycles. We discovered that the presence of sodium-containing species were effective in inhibiting the formation of copper aluminates (CuAl2O4 or CuAlO 2) and stabilising the copper phase in an amorphous support over multiple redox cycles. A representative nanostructured Cu-based oxygen storage material derived from the LDH precursor showed stable gaseous O2 release capacity (∼5 wt%), stable oxygen storage capacity (∼12 wt%), and stable reaction rates during reversible phase changes between CuO-Cu 2O-Cu at high temperatures (800-1000 °C). We anticipate that the strategy can be extended to manufacture a variety of metal oxide composites for applications in novel high temperature looping cycles for clean energy production and CO2 capture. © The Royal Society of Chemistry 2013.

Er3+ excitation and up-conversion processes in y 2-xErxSi2O7 films for planar optical amplifiers

Structural and optical properties of Y2-xErxSi 2O7 thin films have been studied. For higher Er content mechanisms related to Er-Er interactions increase optical efficiency. Moreover the influence of up-conversion has been estimated. ©2009 IEEE.

Modeling natural sounds with modulation cascade processes

Natural sounds are structured on many time-scales. A typical segment of speech, for example, contains features that span four orders of magnitude: Sentences ($\sim1$s); phonemes ($\sim10$−$1$ s); glottal pulses ($\sim 10$−$2$s); and formants ($\sim 10$−$3$s). The auditory system uses information from each of these time-scales to solve complicated tasks such as auditory scene analysis [1]. One route toward understanding how auditory processing accomplishes this analysis is to build neuroscience-inspired algorithms which solve similar tasks and to compare the properties of these algorithms with properties of auditory processing. There is however a discord: Current machine-audition algorithms largely concentrate on the shorter time-scale structures in sounds, and the longer structures are ignored. The reason for this is two-fold. Firstly, it is a difficult technical problem to construct an algorithm that utilises both sorts of information. Secondly, it is computationally demanding to simultaneously process data both at high resolution (to extract short temporal information) and for long duration (to extract long temporal information). The contribution of this work is to develop a new statistical model for natural sounds that captures structure across a wide range of time-scales, and to provide efficient learning and inference algorithms. We demonstrate the success of this approach on a missing data task.

Gaussian Process Vine Copulas for Multivariate Dependence

Copulas allow to learn marginal distributions separately from the multivariate dependence structure (copula) that links them together into a density function. Vine factorizations ease the learning of high-dimensional copulas by constructing a hierarchy of conditional bivariate copulas. However, to simplify inference, it is common to assume that each of these conditional bivariate copulas is independent from its conditioning variables. In this paper, we relax this assumption by discovering the latent functions that specify the shape of a conditional copula given its conditioning variables We learn these functions by following a Bayesian approach based on sparse Gaussian processes with expectation propagation for scalable, approximate inference. Experiments on real-world datasets show that, when modeling all conditional dependencies, we obtain better estimates of the underlying copula of the data.

Modelling and control of nonlinear systems using Gaussian processes with partial model information

Gaussian processes are gaining increasing popularity among the control community, in particular for the modelling of discrete time state space systems. However, it has not been clear how to incorporate model information, in the form of known state relationships, when using a Gaussian process as a predictive model. An obvious example of known prior information is position and velocity related states. Incorporation of such information would be beneficial both computationally and for faster dynamics learning. This paper introduces a method of achieving this, yielding faster dynamics learning and a reduction in computational effort from O(Dn2) to O((D - F)n2) in the prediction stage for a system with D states, F known state relationships and n observations. The effectiveness of the method is demonstrated through its inclusion in the PILCO learning algorithm with application to the swing-up and balance of a torque-limited pendulum and the balancing of a robotic unicycle in simulation. © 2012 IEEE.

Gaussian Processes for Machine Learning

The code provided here originally demonstrated the main algorithms from Rasmussen and Williams: Gaussian Processes for Machine Learning. It has since grown to allow more likelihood functions, further inference methods and a flexible framework for specifying GPs.

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

Foresight vehicle programme - Customer understanding processes in design

Customer feedback is normally fed into product design and engineering via quality surveys and therefore mainly comprises negative comments: complaints about things gone wrong. Whilst eradication of such problems will result in a feeling of satisfaction in existing customers, it will not instil the sense of delight required to attract conquest buyers. CUPID's aim is to conceive and evaluate ideas to stimulate product desirability through the provision of delightful features and execution. By definition, surprise and delight features cannot be foreseen, so we have to understand sensory appeal and, therefore, the "hidden" voice of the customer. Copyright © 2002 Society of Automotive Engineers, Inc.