Gaussian processes for fast policy optimisation of POMDP-based dialogue managers

Modelling dialogue as a Partially Observable Markov Decision Process (POMDP) enables a dialogue policy robust to speech understanding errors to be learnt. However, a major challenge in POMDP policy learning is to maintain tractability, so the use of approximation is inevitable. We propose applying Gaussian Processes in Reinforcement learning of optimal POMDP dialogue policies, in order (1) to make the learning process faster and (2) to obtain an estimate of the uncertainty of the approximation. We first demonstrate the idea on a simple voice mail dialogue task and then apply this method to a real-world tourist information dialogue task. © 2010 Association for Computational Linguistics.

Branching toughens fibrous networks.

Fibrous collagenous networks are not only stiff but also tough, due to their complex microstructures. This stiff yet tough behavior is desirable for both medical and military applications but it is difficult to reproduce in engineering materials. While the nonlinear hyperelastic behavior of fibrous networks has been extensively studied, the understanding of toughness is still incomplete. Here, we identify a microstructure mimicking the branched bundles of a natural type I collagen network, in which partially cross-linked long fibers give rise to novel combinations of stiffness and toughness. Finite element analysis shows that the stiffness of fully cross-linked fibrous networks is amplified by increasing the fibril length and cross-link density. However, a trade-off of such stiff networks is reduced toughness. By having partially cross-linked networks with long fibrils, the networks have comparable stiffness and improved toughness as compared to the fully cross-linked networks. Further, the partially cross-linked networks avoid the formation of kinks, which cause fibril rupture during deformation. As a result, the branching allows the networks to have stiff yet tough behavior.

Additive Gaussian processes

We introduce a Gaussian process model of functions which are additive. An additive function is one which decomposes into a sum of low-dimensional functions, each depending on only a subset of the input variables. Additive GPs generalize both Generalized Additive Models, and the standard GP models which use squared-exponential kernels. Hyperparameter learning in this model can be seen as Bayesian Hierarchical Kernel Learning (HKL). We introduce an expressive but tractable parameterization of the kernel function, which allows efficient evaluation of all input interaction terms, whose number is exponential in the input dimension. The additional structure discoverable by this model results in increased interpretability, as well as state-of-the-art predictive power in regression tasks.

Analysis of defect-related localized emission processes in InGaN/GaN-based LEDs

This paper reports an extensive analysis of the defect-related localized emission processes occurring in InGaN/GaN-based light-emitting diodes (LEDs) at low reverse- and forward-bias conditions. The analysis is based on combined electrical characterization and spectrally and spatially resolved electroluminescence (EL) measurements. Results of this analysis show that: (i) under reverse bias, LEDs can emit a weak luminescence signal, which is directly proportional to the injected reverse current. Reverse-bias emission is localized in submicrometer-size spots; the intensity of the signal is strongly correlated to the threading dislocation (TD) density, since TDs are preferential paths for leakage current conduction. (ii) Under low forward-bias conditions, the intensity of the EL signal is not uniform over the device area. Spectrally resolved EL analysis of green LEDs identifies the presence of localized spots emitting at 600 nm (i.e., in the yellow spectral region), whose origin is ascribed to localized tunneling occurring between the quantum wells and the barrier layers of the diodes, with subsequent defect-assisted radiative recombination. The role of defects in determining yellow luminescence is confirmed by the high activation energy of the thermal quenching of yellow emission (Ea =0.64&eV). © 2012 IEEE.

Paradoxical evidence integration in rapid decision processes

Decisions about noisy stimuli require evidence integration over time. Traditionally, evidence integration and decision making are described as a one-stage process: a decision is made when evidence for the presence of a stimulus crosses a threshold. Here, we show that one-stage models cannot explain psychophysical experiments on feature fusion, where two visual stimuli are presented in rapid succession. Paradoxically, the second stimulus biases decisions more strongly than the first one, contrary to predictions of one-stage models and intuition. We present a two-stage model where sensory information is integrated and buffered before it is fed into a drift diffusion process. The model is tested in a series of psychophysical experiments and explains both accuracy and reaction time distributions. © 2012 Rüter et al.

State-space inference and learning with Gaussian processes

State-space inference and learning with Gaussian processes (GPs) is an unsolved problem. We propose a new, general methodology for inference and learning in nonlinear state-space models that are described probabilistically by non-parametric GP models. We apply the expectation maximization algorithm to iterate between inference in the latent state-space and learning the parameters of the underlying GP dynamics model. Copyright 2010 by the authors.

Robust filtering and smoothing with gaussian processes

We propose a principled algorithm for robust Bayesian filtering and smoothing in nonlinear stochastic dynamic systems when both the transition function and the measurement function are described by non-parametric Gaussian process (GP) models. GPs are gaining increasing importance in signal processing, machine learning, robotics, and control for representing unknown system functions by posterior probability distributions. This modern way of system identification is more robust than finding point estimates of a parametric function representation. Our principled filtering/smoothing approach for GP dynamic systems is based on analytic moment matching in the context of the forward-backward algorithm. Our numerical evaluations demonstrate the robustness of the proposed approach in situations where other state-of-the-art Gaussian filters and smoothers can fail. © 2011 IEEE.

Partially Observable Markov Decision Processes with continuous observations for dialogue management

This work shows how a dialogue model can be represented as a Partially Observable Markov Decision Process (POMDP) with observations composed of a discrete and continuous component. The continuous component enables the model to directly incorporate a confidence score for automated planning. Using a testbed simulated dialogue management problem, we show how recent optimization techniques are able to find a policy for this continuous POMDP which outperforms a traditional MDP approach. Further, we present a method for automatically improving handcrafted dialogue managers by incorporating POMDP belief state monitoring, including confidence score information. Experiments on the testbed system show significant improvements for several example handcrafted dialogue managers across a range of operating conditions.

A high performance oxygen storage material for chemical looping processes with CO2 capture

Chemical looping combustion (CLC) is a novel combustion technology that involves cyclic reduction and oxidation of oxygen storage materials to provide oxygen for the combustion of fuels to CO2 and H2O, whilst giving a pure stream of CO2 suitable for sequestration or utilisation. Here, we report a method for preparing of oxygen storage materials from layered double hydroxides (LDHs) precursors and demonstrate their applications in the CLC process. The LDHs precursor enables homogeneous mixing of elements at the molecular level, giving a high degree of dispersion and high-loading of active metal oxide in the support after calcination. Using a Cu-Al LDH precursor as a prototype, we demonstrate that rational design of oxygen storage materials by material chemistry significantly improved the reactivity and stability in the high temperature redox cycles. We discovered that the presence of sodium-containing species were effective in inhibiting the formation of copper aluminates (CuAl2O4 or CuAlO 2) and stabilising the copper phase in an amorphous support over multiple redox cycles. A representative nanostructured Cu-based oxygen storage material derived from the LDH precursor showed stable gaseous O2 release capacity (∼5 wt%), stable oxygen storage capacity (∼12 wt%), and stable reaction rates during reversible phase changes between CuO-Cu 2O-Cu at high temperatures (800-1000 °C). We anticipate that the strategy can be extended to manufacture a variety of metal oxide composites for applications in novel high temperature looping cycles for clean energy production and CO2 capture. © The Royal Society of Chemistry 2013.

Er3+ excitation and up-conversion processes in y 2-xErxSi2O7 films for planar optical amplifiers

Structural and optical properties of Y2-xErxSi 2O7 thin films have been studied. For higher Er content mechanisms related to Er-Er interactions increase optical efficiency. Moreover the influence of up-conversion has been estimated. ©2009 IEEE.

Modeling natural sounds with modulation cascade processes

Natural sounds are structured on many time-scales. A typical segment of speech, for example, contains features that span four orders of magnitude: Sentences ($\sim1$s); phonemes ($\sim10$−$1$ s); glottal pulses ($\sim 10$−$2$s); and formants ($\sim 10$−$3$s). The auditory system uses information from each of these time-scales to solve complicated tasks such as auditory scene analysis [1]. One route toward understanding how auditory processing accomplishes this analysis is to build neuroscience-inspired algorithms which solve similar tasks and to compare the properties of these algorithms with properties of auditory processing. There is however a discord: Current machine-audition algorithms largely concentrate on the shorter time-scale structures in sounds, and the longer structures are ignored. The reason for this is two-fold. Firstly, it is a difficult technical problem to construct an algorithm that utilises both sorts of information. Secondly, it is computationally demanding to simultaneously process data both at high resolution (to extract short temporal information) and for long duration (to extract long temporal information). The contribution of this work is to develop a new statistical model for natural sounds that captures structure across a wide range of time-scales, and to provide efficient learning and inference algorithms. We demonstrate the success of this approach on a missing data task.