On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

Statistical dialogue models have required a large number of dialogues to optimise the dialogue policy, relying on the use of a simulated user. This results in a mismatch between training and live conditions, and significant development costs for the simulator thereby mitigating many of the claimed benefits of such models. Recent work on Gaussian process reinforcement learning, has shown that learning can be substantially accelerated. This paper reports on an experiment to learn a policy for a real-world task directly from human interaction using rewards provided by users. It shows that a usable policy can be learnt in just a few hundred dialogues without needing a user simulator and, using a learning strategy that reduces the risk of taking bad actions. The paper also investigates adaptation behaviour when the system continues learning for several thousand dialogues and highlights the need for robustness to noisy rewards. © 2011 IEEE.

Identification of the steering control behaviour of five test subjects following a randomly curving path in a driving simulator

The paper is concerned with the identification of theoretical preview steering controllers using data obtained from five test subjects in a fixed-base driving simulator. An understanding of human steering control behaviour is relevant to the design of autonomous and semi-autonomous vehicle controls. The driving task involved steering a linear vehicle along a randomly curving path. The theoretical steering controllers identified from the data were based on optimal linear preview control. A direct-identification method was used, and the steering controllers were identified so that the predicted steering angle matched as closely as possible the measured steering angle of the test subjects. It was found that identification of the driver's time delay and noise is necessary to avoid bias in identification of the controller parameters. Most subjects' steering behaviour was predicted well by a theoretical controller based on the lateral/yaw dynamics of the vehicle. There was some evidence that an inexperienced driver's steering action was better represented by a controller based on a simpler model of the vehicle dynamics, perhaps reflecting incomplete learning by the driver. Copyright © 2014 Inderscience Enterprises Ltd.