Gaussian processes for fast policy optimisation of POMDP-based dialogue managers

Modelling dialogue as a Partially Observable Markov Decision Process (POMDP) enables a dialogue policy robust to speech understanding errors to be learnt. However, a major challenge in POMDP policy learning is to maintain tractability, so the use of approximation is inevitable. We propose applying Gaussian Processes in Reinforcement learning of optimal POMDP dialogue policies, in order (1) to make the learning process faster and (2) to obtain an estimate of the uncertainty of the approximation. We first demonstrate the idea on a simple voice mail dialogue task and then apply this method to a real-world tourist information dialogue task. © 2010 Association for Computational Linguistics.

On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

Statistical dialogue models have required a large number of dialogues to optimise the dialogue policy, relying on the use of a simulated user. This results in a mismatch between training and live conditions, and significant development costs for the simulator thereby mitigating many of the claimed benefits of such models. Recent work on Gaussian process reinforcement learning, has shown that learning can be substantially accelerated. This paper reports on an experiment to learn a policy for a real-world task directly from human interaction using rewards provided by users. It shows that a usable policy can be learnt in just a few hundred dialogues without needing a user simulator and, using a learning strategy that reduces the risk of taking bad actions. The paper also investigates adaptation behaviour when the system continues learning for several thousand dialogues and highlights the need for robustness to noisy rewards. © 2011 IEEE.

Cost-effective 10 Gb/s polymer-based chip-to-chip optical interconnect

Board-level optical links are an attractive alternative to their electrical counterparts as they provide higher bandwidth and lower power consumption at high data rates. However, on-board optical technology has to be cost-effective to be commercially deployed. This study presents a chip-to-chip optical interconnect formed on an optoelectronic printed circuit board that uses a simple optical coupling scheme, cost-effective materials and is compatible with well-established manufacturing processes common to the electronics industry. Details of the link architecture, modelling studies of the link's frequency response, characterisation of optical coupling efficiencies and dynamic performance studies of this proof-of-concept chip-to-chip optical interconnect are reported. The fully assembled link exhibits a -3 dBe bandwidth of 9 GHz and -3 dBo tolerances to transverse component misalignments of ±25 and ±37 μm at the input and output waveguide interfaces, respectively. The link has a total insertion loss of 6 dBo and achieves error-free transmission at a 10 Gb/s data rate with a power margin of 11.6 dBo for a bit-error-rate of 10 -12. The proposed architecture demonstrates an integration approach for high-speed board-level chip-to-chip optical links that emphasises component simplicity and manufacturability crucial to the migration of such technology into real-world commercial systems. © 2012 The Institution of Engineering and Technology.

Color photometric stereo for multicolored surfaces

We present a multispectral photometric stereo method for capturing geometry of deforming surfaces. A novel photometric calibration technique allows calibration of scenes containing multiple piecewise constant chromaticities. This method estimates per-pixel photometric properties, then uses a RANSAC-based approach to estimate the dominant chromaticities in the scene. A likelihood term is developed linking surface normal, image intensity and photometric properties, which allows estimating the number of chromaticities present in a scene to be framed as a model estimation problem. The Bayesian Information Criterion is applied to automatically estimate the number of chromaticities present during calibration. A two-camera stereo system provides low resolution geometry, allowing the likelihood term to be used in segmenting new images into regions of constant chromaticity. This segmentation is carried out in a Markov Random Field framework and allows the correct photometric properties to be used at each pixel to estimate a dense normal map. Results are shown on several challenging real-world sequences, demonstrating state-of-the-art results using only two cameras and three light sources. Quantitative evaluation is provided against synthetic ground truth data. © 2011 IEEE.

An automatic speaker recognition system for intelligence applications

This paper presents an automatic speaker recognition system for intelligence applications. The system has to provide functionalities for a speaker skimming application in which databases of recorded conversations belonging to an ongoing investigation can be annotated and quickly browsed by an operator. The paper discusses the criticalities introduced by the characteristics of the audio signals under consideration - in particular background noise and channel/coding distortions - as well as the requirements and functionalities of the system under development. It is shown that the performance of state-of-the-art approaches degrades significantly in presence of moderately high background noise. Finally, a novel speaker recognizer based on phonetic features and an ensemble classifier is presented. Results show that the proposed approach improves performance on clean audio, and suggest that it can be employed towards improved real-world robustness. © EURASIP, 2009.

A validation of commercial flow solver performance on hybridised octree meshes

In recent years we have been developing a meshing system which is aimed at eliminating the bottleneck represented by building meshes for real-world, complex turbomachinery configurations. This system is based on a rapid octree meshing technology which is then made conformal to the bodies present. The objective of this paper is to demonstrate that this class of mesh is not only very fast to produce but also fit-for-purpose in the sense that simulations generated with third-party commercial flow solvers like Fluent have the same accuracy as those performed on more conventional meshes. A range of standard examples and test cases will be presented. Copyright © 2011 by ASME.

Gone in 0.6 seconds: the encoding of motor memories depends on recent sensorimotor states.

Real-world tasks often require movements that depend on a previous action or on changes in the state of the world. Here we investigate whether motor memories encode the current action in a manner that depends on previous sensorimotor states. Human subjects performed trials in which they made movements in a randomly selected clockwise or counterclockwise velocity-dependent curl force field. Movements during this adaptation phase were preceded by a contextual phase that determined which of the two fields would be experienced on any given trial. As expected from previous research, when static visual cues were presented in the contextual phase, strong interference (resulting in an inability to learn either field) was observed. In contrast, when the contextual phase involved subjects making a movement that was continuous with the adaptation-phase movement, a substantial reduction in interference was seen. As the time between the contextual and adaptation movement increased, so did the interference, reaching a level similar to that seen for static visual cues for delays >600 ms. This contextual effect generalized to purely visual motion, active movement without vision, passive movement, and isometric force generation. Our results show that sensorimotor states that differ in their recent temporal history can engage distinct representations in motor memory, but this effect decays progressively over time and is abolished by ∼600 ms. This suggests that motor memories are encoded not simply as a mapping from current state to motor command but are encoded in terms of the recent history of sensorimotor states.

Templates and anchors for antenna-based wall following in cockroaches and robots

The interplay between robotics and neuromechanics facilitates discoveries in both fields: nature provides roboticists with design ideas, while robotics research elucidates critical features that confer performance advantages to biological systems. Here, we explore a system particularly well suited to exploit the synergies between biology and robotics: high-speed antenna-based wall following of the American cockroach (Periplaneta americana). Our approach integrates mathematical and hardware modeling with behavioral and neurophysiological experiments. Specifically, we corroborate a prediction from a previously reported wall-following template - the simplest model that captures a behavior - that a cockroach antenna-based controller requires the rate of approach to a wall in addition to distance, e.g., in the form of a proportional-derivative (PD) controller. Neurophysiological experiments reveal that important features of the wall-following controller emerge at the earliest stages of sensory processing, namely in the antennal nerve. Furthermore, we embed the template in a robotic platform outfitted with a bio-inspired antenna. Using this system, we successfully test specific PD gains (up to a scale) fitted to the cockroach behavioral data in a "real-world" setting, lending further credence to the surprisingly simple notion that a cockroach might implement a PD controller for wall following. Finally, we embed the template in a simulated lateral-leg-spring (LLS) model using the center of pressure as the control input. Importantly, the same PD gains fitted to cockroach behavior also stabilize wall following for the LLS model. © 2008 IEEE.

Gaussian Process Vine Copulas for Multivariate Dependence

Copulas allow to learn marginal distributions separately from the multivariate dependence structure (copula) that links them together into a density function. Vine factorizations ease the learning of high-dimensional copulas by constructing a hierarchy of conditional bivariate copulas. However, to simplify inference, it is common to assume that each of these conditional bivariate copulas is independent from its conditioning variables. In this paper, we relax this assumption by discovering the latent functions that specify the shape of a conditional copula given its conditioning variables We learn these functions by following a Bayesian approach based on sparse Gaussian processes with expectation propagation for scalable, approximate inference. Experiments on real-world datasets show that, when modeling all conditional dependencies, we obtain better estimates of the underlying copula of the data.

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Multi-objective optimization of a fluid structure interaction benchmarking

The integration and application of a new multi-objective tabu search optimization algorithm for Fluid Structure Interaction (FSI) problems are presented. The aim is to enhance the computational design process for real world applications and to achieve higher performance of the whole system for the four considered objectives. The described system combines the optimizer with a well established FSI solver which is based on the fully implicit, monolithic formuFlation of the problem in the Arbitrary Lagrangian-Eulerian FEM approach. The proposed solver resolves the proposed uid-structure interaction benchmark which describes the self-induced elastic deformation of a beam attached to a cylinder in laminar channel ow. The optimized ow characteristics of the aforementioned geometrical arrangement illustrate the performance of the system in two dimensions. Special emphasis is given to the analysis of the simulation package, which is of high accuracy and is the core of application. The design process identifies the best combination of ow features for optimal system behavior and the most important objectives. In addition, the presented methodology has the potential to run in parallel, which will significantly speed-up the elapsed time. Finite Element Method (FEM), Fluid-Structure Interaction (FSI), Multi-Ojective Tabu search (MOTS2). Copyright © 2013 Tech Science Press.

Parallel coordinates in computational engineering design

Modern Engineering Design involves the deployment of many computational tools. Re- search on challenging real-world design problems is focused on developing improvements for the engineering design process through the integration and application of advanced com- putational search/optimization and analysis tools. Successful application of these methods generates vast quantities of data on potential optimum designs. To gain maximum value from the optimization process, designers need to visualise and interpret this information leading to better understanding of the complex and multimodal relations between param- eters, objectives and decision-making of multiple and strongly conflicting criteria. Initial work by the authors has identified that the Parallel Coordinates interactive visualisation method has considerable potential in this regard. This methodology involves significant levels of user-interaction, making the engineering designer central to the process, rather than the passive recipient of a deluge of pre-formatted information. In the present work we have applied and demonstrated this methodology in two differ- ent aerodynamic turbomachinery design cases; a detailed 3D shape design for compressor blades, and a preliminary mean-line design for the whole compressor core. The first case comprises 26 design parameters for the parameterisation of the blade geometry, and we analysed the data produced from a three-objective optimization study, thus describing a design space with 29 dimensions. The latter case comprises 45 design parameters and two objective functions, hence developing a design space with 47 dimensions. In both cases the dimensionality can be managed quite easily in Parallel Coordinates space, and most importantly, we are able to identify interesting and crucial aspects of the relationships between the design parameters and optimum level of the objective functions under con- sideration. These findings guide the human designer to find answers to questions that could not even be addressed before. In this way, understanding the design leads to more intelligent decision-making and design space exploration. © 2012 AIAA.

Assessing boost-assist options for turbocharged engines using 1-D engine simulation and model predictive control

Delivering acceptable low end torque and good transient response is a significant challenge for all turbocharged engines. As downsized gasoline engines and Diesel engines make up a larger and larger proportion of the light-duty engines entering the market, the issue takes on greater significance. Several schemes have been proposed to improve torque response in highly boosted engines, including the use of electrical assist turbochargers and compressed air assist. In this paper we examine these methods with respect to their effectiveness in improving transient response and their relative performance along with some of the practical considerations for real world application. Results shown in this paper are from 1-D simulations using the Ricardo WAVE software package. The simulation model is based on a production light-duty Diesel engine modified to allow the introduction of compressed air at various points in the air-path as well as direct torque application to the turbocharger shaft (such as might be available from an electrical assist turbocharger). Whilst the 1-D simulation software provides a suitable environment for investigating the various boost assistance options, the overall air path performance also depends upon the control system. The introduction of boost assistance complicates the control in two significant ways: the system may run into constraints (such as compressor surge) that are not encountered in normal operation and the assistance introduces an additional control input. Production engine controllers are usually based on gain-scheduled PID control and extensive calibration. For this study, the non-linear nature of the engine together with the multiple configurations considered and the slower than real-time execution of 1-D models makes such an approach time consuming. Moreover, an ad-hoc approach would leave some doubt as to the fairness of comparisons between the different boost-assist options. Model Predictive Control has been shown to offer a convenient approach to controlling the 1-D simulations in a close to optimal manner for a typical Diesel VGT-EGR air path configuration. We show that the same technique can be applied to all the considered assistance methods with only modest calibration effort required. Copyright © 2012 SAE International.

Enhancing and developing the practical optimisation capabilities and intelligence of automatic design software

In the modern engineering design cycle the use of computational tools becomes a neces- sity. The complexity of the engineering systems under consideration for design increases dramatically as the demands for advanced and innovative design concepts and engineering products is expanding. At the same time the advancements in the available technology in terms of computational resources and power, as well as the intelligence of the design software, accommodate these demands and make them a viable approach towards the chal- lenge of real-world engineering problems. This class of design optimisation problems is by nature multi-disciplinary. In the present work we establish enhanced optimisation capabil- ities within the Nimrod/O tool for massively distributed execution of computational tasks through cluster and computational grid resources, and develop the potential to combine and benefit from all the possible available technological advancements, both software and hardware. We develop the interface between a Free Form Deformation geometry manage- ment in-house code with the 2D airfoil aerodynamic efficiency evaluation tool XFoil, and the well established multi-objective heuristic optimisation algorithm NSGA-II. A simple airfoil design problem has been defined to demonstrate the functionality of the design sys- tem, but also to accommodate a framework for future developments and testing with other state-of-the-art optimisation algorithms such as the Multi-Objective Genetic Algorithm (MOGA) and the Multi-Objective Tabu Search (MOTS) techniques. Ultimately, heav- ily computationally expensive industrial design cases can be realised within the presented framework that could not be investigated before. © 2012 by the authors. Published by the American Institute of Aeronautics and Astronautics, Inc.

Enhancing and developing the practical optimisation capabilities and intelligence of automatic design software

In the modern engineering design cycle the use of computational tools becomes a necessity. The complexity of the engineering systems under consideration for design increases dramatically as the demands for advanced and innovative design concepts and engineering products is expanding. At the same time the advancements in the available technology in terms of computational resources and power, as well as the intelligence of the design software, accommodate these demands and make them a viable approach towards the challenge of real-world engineering problems. This class of design optimisation problems is by nature multi-disciplinary. In the present work we establish enhanced optimisation capabilities within the Nimrod/O tool for massively distributed execution of computational tasks through cluster and computational grid resources, and develop the potential to combine and benefit from all the possible available technological advancements, both software and hardware. We develop the interface between a Free Form Deformation geometry management in-house code with the 2D airfoil aerodynamic efficiency evaluation tool XFoil, and the well established multi-objective heuristic optimisation algorithm NSGA-II. A simple airfoil design problem has been defined to demonstrate the functionality of the design system, but also to accommodate a framework for future developments and testing with other state-of-the-art optimisation algorithms such as the Multi-Objective Genetic Algorithm (MOGA) and the Multi-Objective Tabu Search (MOTS) techniques. Ultimately, heavily computationally expensive industrial design cases can be realised within the presented framework that could not be investigated before. ©2012 AIAA.