Gaussian processes for fast policy optimisation of POMDP-based dialogue managers

Modelling dialogue as a Partially Observable Markov Decision Process (POMDP) enables a dialogue policy robust to speech understanding errors to be learnt. However, a major challenge in POMDP policy learning is to maintain tractability, so the use of approximation is inevitable. We propose applying Gaussian Processes in Reinforcement learning of optimal POMDP dialogue policies, in order (1) to make the learning process faster and (2) to obtain an estimate of the uncertainty of the approximation. We first demonstrate the idea on a simple voice mail dialogue task and then apply this method to a real-world tourist information dialogue task. © 2010 Association for Computational Linguistics.

On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

Statistical dialogue models have required a large number of dialogues to optimise the dialogue policy, relying on the use of a simulated user. This results in a mismatch between training and live conditions, and significant development costs for the simulator thereby mitigating many of the claimed benefits of such models. Recent work on Gaussian process reinforcement learning, has shown that learning can be substantially accelerated. This paper reports on an experiment to learn a policy for a real-world task directly from human interaction using rewards provided by users. It shows that a usable policy can be learnt in just a few hundred dialogues without needing a user simulator and, using a learning strategy that reduces the risk of taking bad actions. The paper also investigates adaptation behaviour when the system continues learning for several thousand dialogues and highlights the need for robustness to noisy rewards. © 2011 IEEE.

Active control of high-frequency vibration: Optimisation using the hybrid modelling method

This work presents active control of high-frequency vibration using skyhook dampers. The choice of the damper gain and its optimal location is crucial for the effective implementation of active vibration control. In vibration control, certain sensor/actuator locations are preferable for reducing structural vibration while using minimum control effort. In order to perform optimisation on a general built-up structure to control vibration, it is necessary to have a good modelling technique to predict the performance of the controller. The present work exploits the hybrid modelling approach, which combines the finite element method (FEM) and statistical energy analysis (SEA) to provide efficient response predictions at medium to high frequencies. The hybrid method is implemented here for a general network of plates, coupled via springs, to allow study of a variety of generic control design problems. By combining the hybrid method with numerical optimisation using a genetic algorithm, optimal skyhook damper gains and locations are obtained. The optimal controller gain and location found from the hybrid method are compared with results from a deterministic modelling method. Good agreement between the results is observed, whereas results from the hybrid method are found in a significantly reduced amount of time. © 2012 Elsevier Ltd. All rights reserved.

Vehicle trajectory linearisation to enable efficient optimisation of the constant speed racing line

A driver model is presented capable of optimising the trajectory of a simple dynamic nonlinear vehicle, at constant forward speed, so that progression along a predefined track is maximised as a function of time. In doing so, the model is able to continually operate a vehicle at its lateral-handling limit, maximising vehicle performance. The technique used forms a part of the solution to the motor racing objective of minimising lap time. A new approach of formulating the minimum lap time problem is motivated by the need for a more computationally efficient and robust tool-set for understanding on-the-limit driving behaviour. This has been achieved through set point-dependent linearisation of the vehicle model and coupling the vehicle-track system using an intrinsic coordinate description. Through this, the geometric vehicle trajectory had been linearised relative to the track reference, leading to new path optimisation algorithm which can be formed as a computationally efficient convex quadratic programming problem. © 2012 Copyright Taylor and Francis Group, LLC.

Polarisation diversity of conformal arrays based on geometric algebra via convex optimisation

A simple and general design procedure is presented for the polarisation diversity of arbitrary conformal arrays; this procedure is based on the mathematical framework of geometric algebra and can be solved optimally using convex optimisation. Aside from being simpler and more direct than other derivations in the literature, this derivation is also entirely general in that it expresses the transformations in terms of rotors in geometric algebra which can easily be formulated for any arbitrary conformal array geometry. Convex optimisation has a number of advantages; solvers are widespread and freely available, the process generally requires a small number of iterations and a wide variety of constraints can be readily incorporated. The study outlines a two-step approach for addressing polarisation diversity in arbitrary conformal arrays: first, the authors obtain the array polarisation patterns using geometric algebra and secondly use a convex optimisation approach to find the optimal weights for the polarisation diversity problem. The versatility of this approach is illustrated via simulations of a 7×10 cylindrical conformal array. © 2012 The Institution of Engineering and Technology.

Uncertainty management for on-line optimisation of a POMDP-based large-scale spoken dialogue system

The optimization of dialogue policies using reinforcement learning (RL) is now an accepted part of the state of the art in spoken dialogue systems (SDS). Yet, it is still the case that the commonly used training algorithms for SDS require a large number of dialogues and hence most systems still rely on artificial data generated by a user simulator. Optimization is therefore performed off-line before releasing the system to real users. Gaussian Processes (GP) for RL have recently been applied to dialogue systems. One advantage of GP is that they compute an explicit measure of uncertainty in the value function estimates computed during learning. In this paper, a class of novel learning strategies is described which use uncertainty to control exploration on-line. Comparisons between several exploration schemes show that significant improvements to learning speed can be obtained and that rapid and safe online optimisation is possible, even on a complex task. Copyright © 2011 ISCA.

Making a shallow network deep: Conversion of a boosting classifier into a decision tree by boolean optimisation

This paper presents a novel way to speed up the evaluation time of a boosting classifier. We make a shallow (flat) network deep (hierarchical) by growing a tree from decision regions of a given boosting classifier. The tree provides many short paths for speeding up while preserving the reasonably smooth decision regions of the boosting classifier for good generalisation. For converting a boosting classifier into a decision tree, we formulate a Boolean optimization problem, which has been previously studied for circuit design but limited to a small number of binary variables. In this work, a novel optimisation method is proposed for, firstly, several tens of variables i.e. weak-learners of a boosting classifier, and then any larger number of weak-learners by using a two-stage cascade. Experiments on the synthetic and face image data sets show that the obtained tree achieves a significant speed up both over a standard boosting classifier and the Fast-exit-a previously described method for speeding-up boosting classification, at the same accuracy. The proposed method as a general meta-algorithm is also useful for a boosting cascade, where it speeds up individual stage classifiers by different gains. The proposed method is further demonstrated for fast-moving object tracking and segmentation problems. © 2011 Springer Science+Business Media, LLC.

Coherent design methodology using modelling, simulation and optimisation

There is an increasing demand for optimising complete systems and the devices within that system, including capturing the interactions between the various multi-disciplinary (MD) components involved. Furthermore confidence in robust solutions is esential. As a consequence the computational cost rapidly increases and in many cases becomes infeasible to perform such conceptual designs. A coherent design methodology is proposed, where the aim is to improve the design process by effectively exploiting the potential of computational synthesis, search and optimisation and conventional simulation, with a reduction of the computational cost. This optimization framework consists of a hybrid optimization algorithm to handles multi-fidelity simulations. Simultaneously and in order to handles uncertainty without recasting the model and at affordable computational cost, a stochastic modelling method known as non-intrusive polynomial chaos is introduced. The effectiveness of the design methodology is demonstrated with the optimisation of a submarine propulsion system.

Towards a fast electro-thermo-mechanical IGBT/Diode simulation framework for design of reliable power electronic converters

This paper describes a methodology that enables fast and reasonably accurate prediction of the reliability of power electronic modules featuring IGBTs and p-i-n diodes, by taking into account thermo-mechanical failure mechanisms of the devices and their associated packaging. In brief, the proposed simulation framework performs two main tasks which are tightly linked together: (i) the generation of the power devices' transient thermal response for realistic long load cycles and (ii) the prediction of the power modules' lifetime based on the obtained temperature profiles. In doing so the first task employs compact, physics-based device models, power losses lookup tables and polynomials and combined material-failure and thermal modelling, while the second task uses advanced reliability tests for failure mode and time-to-failure estimation. The proposed technique is intended to be utilised as a design/optimisation tool for reliable power electronic converters, since it allows easy and fast investigation of the effects that changes in circuit topology or devices' characteristics and packaging have on the reliability of the employed power electronic modules. © 2012 IEEE.

Policy optimisation of POMDP-based dialogue systems without state space compression

The partially observable Markov decision process (POMDP) has been proposed as a dialogue model that enables automatic improvement of the dialogue policy and robustness to speech understanding errors. It requires, however, a large number of dialogues to train the dialogue policy. Gaussian processes (GP) have recently been applied to POMDP dialogue management optimisation showing an ability to substantially increase the speed of learning. Here, we investigate this further using the Bayesian Update of Dialogue State dialogue manager. We show that it is possible to apply Gaussian processes directly to the belief state, removing the need for a parametric policy representation. In addition, the resulting policy learns significantly faster while maintaining operational performance. © 2012 IEEE.