On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

Statistical dialogue models have required a large number of dialogues to optimise the dialogue policy, relying on the use of a simulated user. This results in a mismatch between training and live conditions, and significant development costs for the simulator thereby mitigating many of the claimed benefits of such models. Recent work on Gaussian process reinforcement learning, has shown that learning can be substantially accelerated. This paper reports on an experiment to learn a policy for a real-world task directly from human interaction using rewards provided by users. It shows that a usable policy can be learnt in just a few hundred dialogues without needing a user simulator and, using a learning strategy that reduces the risk of taking bad actions. The paper also investigates adaptation behaviour when the system continues learning for several thousand dialogues and highlights the need for robustness to noisy rewards. © 2011 IEEE.

Active control of high-frequency vibration: Optimisation using the hybrid modelling method

This work presents active control of high-frequency vibration using skyhook dampers. The choice of the damper gain and its optimal location is crucial for the effective implementation of active vibration control. In vibration control, certain sensor/actuator locations are preferable for reducing structural vibration while using minimum control effort. In order to perform optimisation on a general built-up structure to control vibration, it is necessary to have a good modelling technique to predict the performance of the controller. The present work exploits the hybrid modelling approach, which combines the finite element method (FEM) and statistical energy analysis (SEA) to provide efficient response predictions at medium to high frequencies. The hybrid method is implemented here for a general network of plates, coupled via springs, to allow study of a variety of generic control design problems. By combining the hybrid method with numerical optimisation using a genetic algorithm, optimal skyhook damper gains and locations are obtained. The optimal controller gain and location found from the hybrid method are compared with results from a deterministic modelling method. Good agreement between the results is observed, whereas results from the hybrid method are found in a significantly reduced amount of time. © 2012 Elsevier Ltd. All rights reserved.

Vehicle trajectory linearisation to enable efficient optimisation of the constant speed racing line

A driver model is presented capable of optimising the trajectory of a simple dynamic nonlinear vehicle, at constant forward speed, so that progression along a predefined track is maximised as a function of time. In doing so, the model is able to continually operate a vehicle at its lateral-handling limit, maximising vehicle performance. The technique used forms a part of the solution to the motor racing objective of minimising lap time. A new approach of formulating the minimum lap time problem is motivated by the need for a more computationally efficient and robust tool-set for understanding on-the-limit driving behaviour. This has been achieved through set point-dependent linearisation of the vehicle model and coupling the vehicle-track system using an intrinsic coordinate description. Through this, the geometric vehicle trajectory had been linearised relative to the track reference, leading to new path optimisation algorithm which can be formed as a computationally efficient convex quadratic programming problem. © 2012 Copyright Taylor and Francis Group, LLC.

Polarisation diversity of conformal arrays based on geometric algebra via convex optimisation

A simple and general design procedure is presented for the polarisation diversity of arbitrary conformal arrays; this procedure is based on the mathematical framework of geometric algebra and can be solved optimally using convex optimisation. Aside from being simpler and more direct than other derivations in the literature, this derivation is also entirely general in that it expresses the transformations in terms of rotors in geometric algebra which can easily be formulated for any arbitrary conformal array geometry. Convex optimisation has a number of advantages; solvers are widespread and freely available, the process generally requires a small number of iterations and a wide variety of constraints can be readily incorporated. The study outlines a two-step approach for addressing polarisation diversity in arbitrary conformal arrays: first, the authors obtain the array polarisation patterns using geometric algebra and secondly use a convex optimisation approach to find the optimal weights for the polarisation diversity problem. The versatility of this approach is illustrated via simulations of a 7×10 cylindrical conformal array. © 2012 The Institution of Engineering and Technology.

Uncertainty management for on-line optimisation of a POMDP-based large-scale spoken dialogue system

The optimization of dialogue policies using reinforcement learning (RL) is now an accepted part of the state of the art in spoken dialogue systems (SDS). Yet, it is still the case that the commonly used training algorithms for SDS require a large number of dialogues and hence most systems still rely on artificial data generated by a user simulator. Optimization is therefore performed off-line before releasing the system to real users. Gaussian Processes (GP) for RL have recently been applied to dialogue systems. One advantage of GP is that they compute an explicit measure of uncertainty in the value function estimates computed during learning. In this paper, a class of novel learning strategies is described which use uncertainty to control exploration on-line. Comparisons between several exploration schemes show that significant improvements to learning speed can be obtained and that rapid and safe online optimisation is possible, even on a complex task. Copyright © 2011 ISCA.

Making a shallow network deep: Conversion of a boosting classifier into a decision tree by boolean optimisation

This paper presents a novel way to speed up the evaluation time of a boosting classifier. We make a shallow (flat) network deep (hierarchical) by growing a tree from decision regions of a given boosting classifier. The tree provides many short paths for speeding up while preserving the reasonably smooth decision regions of the boosting classifier for good generalisation. For converting a boosting classifier into a decision tree, we formulate a Boolean optimization problem, which has been previously studied for circuit design but limited to a small number of binary variables. In this work, a novel optimisation method is proposed for, firstly, several tens of variables i.e. weak-learners of a boosting classifier, and then any larger number of weak-learners by using a two-stage cascade. Experiments on the synthetic and face image data sets show that the obtained tree achieves a significant speed up both over a standard boosting classifier and the Fast-exit-a previously described method for speeding-up boosting classification, at the same accuracy. The proposed method as a general meta-algorithm is also useful for a boosting cascade, where it speeds up individual stage classifiers by different gains. The proposed method is further demonstrated for fast-moving object tracking and segmentation problems. © 2011 Springer Science+Business Media, LLC.

Coherent design methodology using modelling, simulation and optimisation

There is an increasing demand for optimising complete systems and the devices within that system, including capturing the interactions between the various multi-disciplinary (MD) components involved. Furthermore confidence in robust solutions is esential. As a consequence the computational cost rapidly increases and in many cases becomes infeasible to perform such conceptual designs. A coherent design methodology is proposed, where the aim is to improve the design process by effectively exploiting the potential of computational synthesis, search and optimisation and conventional simulation, with a reduction of the computational cost. This optimization framework consists of a hybrid optimization algorithm to handles multi-fidelity simulations. Simultaneously and in order to handles uncertainty without recasting the model and at affordable computational cost, a stochastic modelling method known as non-intrusive polynomial chaos is introduced. The effectiveness of the design methodology is demonstrated with the optimisation of a submarine propulsion system.

Policy optimisation of POMDP-based dialogue systems without state space compression

The partially observable Markov decision process (POMDP) has been proposed as a dialogue model that enables automatic improvement of the dialogue policy and robustness to speech understanding errors. It requires, however, a large number of dialogues to train the dialogue policy. Gaussian processes (GP) have recently been applied to POMDP dialogue management optimisation showing an ability to substantially increase the speed of learning. Here, we investigate this further using the Bayesian Update of Dialogue State dialogue manager. We show that it is possible to apply Gaussian processes directly to the belief state, removing the need for a parametric policy representation. In addition, the resulting policy learns significantly faster while maintaining operational performance. © 2012 IEEE.

Biobjective optimisation of preliminary aircraft trajectories

The protection of the environment against pollutants produced by aviation is of great concern in the 21st century. Among the multiplicity of proposed solutions, modifying flight profiles for existing aircraft is a promising approach. The aim is to deliver and understand the trade-off between environmental impact and operating costs. This work will illustrate the optimisation process of aircraft trajectories by minimising fuel consumption and flight time for the climb phase of an aircraft that belongs to A320 category. To achieve this purpose a new variant of a multi-objective Tabu Search optimiser was evolved and integrated within a computational framework, called GATAC, that simulates flight profiles based on altitude and speed. © 2013 Springer-Verlag.

Optimisation of passive and semi-active heavy vehicle suspensions

An investigation into the potential for reducing road damage by optimising the design of heavy vehicle suspensions is described. In the first part of the paper two simple mathematical models are used to study the optimisation of conventional passive suspensions. Simple modifications are made to the steel spring suspension of a tandem axle trailer and it is found experimentally that RMS dynamic tyre forces can be reduced by 15% and theoretical road damage by 5.2%. A mathematical model of an air-sprung articulated vehicle is validated, and its suspension is optimised according to the simple models. This vehicle generates about 9% less damage than the leaf-sprung vehicle in the unmodified state and it is predicted that, for the operating conditions examined, the road damage caused by this vehicle can be reduced by a further 5.4%. Finally, it is shown experimentally that computer-controlled semi-active dampers have the potential to reduce road damage by a further 5-6%, compared to an air suspension with optimum passive damping. © Copyright 1994 Society of Automotive Engineers, Inc.

An investigation of higher-order multi-objective optimisation for 3D aerodynamic shape design

We investigate the performance of different variants of a suitably tailored Tabu Search optimisation algorithm on a higher-order design problem. We consider four objective func- tions to describe the performance of a compressor stator row, subject to a number of equality and inequality constraints. The same design problem has been previously in- vestigated through single-, bi- and three-objective optimisation studies. However, in this study we explore the capabilities of enhanced variants of our Multi-objective Tabu Search (MOTS) optimisation algorithm in the context of detailed 3D aerodynamic shape design. It is shown that with these enhancements to the local search of the MOTS algorithm we can achieve a rapid exploration of complicated design spaces, but there is a trade-off be- tween speed and the quality of the trade-off surface found. Rapidly explored design spaces reveal the extremes of the objective functions, but the compromise optimum areas are not very well explored. However, there are ways to adapt the behaviour of the optimiser and maintain both a very efficient rate of progress towards the global optimum Pareto front and a healthy number of design configurations lying on the trade-off surface and exploring the compromise optimum regions. These compromise solutions almost always represent the best qualitative balance between the objectives under consideration. Such enhancements to the effectiveness of design space exploration make engineering design optimisation with multiple objectives and robustness criteria ever more practicable and attractive for modern advanced engineering design. Finally, new research questions are addressed that highlight the trade-offs between intelligence in optimisation algorithms and acquisition of qualita- tive information through computational engineering design processes that reveal patterns and relations between design parameters and objective functions, but also speed versus optimum quality. © 2012 AIAA.

Interactive multi-objective particle swarm optimisation using decision space interaction

The most common approach to decision making in multi-objective optimisation with metaheuristics is a posteriori preference articulation. Increased model complexity and a gradual increase of optimisation problems with three or more objectives have revived an interest in progressively interactive decision making, where a human decision maker interacts with the algorithm at regular intervals. This paper presents an interactive approach to multi-objective particle swarm optimisation (MOPSO) using a novel technique to preference articulation based on decision space interaction and visual preference articulation. The approach is tested on a 2D aerofoil design case study and comparisons are drawn to non-interactive MOPSO. © 2013 IEEE.