On-line policy optimisation of Bayesian spoken dialogue systems via human interaction

A partially observable Markov decision process has been proposed as a dialogue model that enables robustness to speech recognition errors and automatic policy optimisation using reinforcement learning (RL). However, conventional RL algorithms require a very large number of dialogues, necessitating a user simulator. Recently, Gaussian processes have been shown to substantially speed up the optimisation, making it possible to learn directly from interaction with human users. However, early studies have been limited to very low dimensional spaces and the learning has exhibited convergence problems. Here we investigate learning from human interaction using the Bayesian Update of Dialogue State system. This dynamic Bayesian network based system has an optimisation space covering more than one hundred features, allowing a wide range of behaviours to be learned. Using an improved policy model and a more robust reward function, we show that stable learning can be achieved that significantly outperforms a simulator trained policy. © 2013 IEEE.

On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

Statistical dialogue models have required a large number of dialogues to optimise the dialogue policy, relying on the use of a simulated user. This results in a mismatch between training and live conditions, and significant development costs for the simulator thereby mitigating many of the claimed benefits of such models. Recent work on Gaussian process reinforcement learning, has shown that learning can be substantially accelerated. This paper reports on an experiment to learn a policy for a real-world task directly from human interaction using rewards provided by users. It shows that a usable policy can be learnt in just a few hundred dialogues without needing a user simulator and, using a learning strategy that reduces the risk of taking bad actions. The paper also investigates adaptation behaviour when the system continues learning for several thousand dialogues and highlights the need for robustness to noisy rewards. © 2011 IEEE.

Workspace comparisons of setup configurations for human-robot interaction

In virtual assembly verification or remote maintenance tasks, bimanual haptic interfaces play a crucial role in successful task completion. This paper proposes a method for objectively comparing how well a haptic interface covers the reachable workspace of human arms. Two system configurations are analyzed for a recently introduced haptic device that is based on two DLR-KUKA light weight robots: the standard configuration, where the device is opposite the human operator, and the ergonomic configuration, where the haptic device is mounted behind the human operator. The human operator directly controls the robotic arms using handles. The analysis is performed using a representation of the robot arm workspace. The merits of restricting the comparisons to the most significant regions of the human workspace are discussed. Using this method, a greater workspace correspondence for the ergonomic configuration was shown. ©2010 IEEE.

Human-humanoid interaction: Is a humanoid robot perceived as a human?

As humanoid robots become more commonplace in our society, it is important to understand the relation between humans and humanoid robots. In human face-to-face interaction, the observation of another individual performing an action facilitates the execution of a similar action, and interferes with the execution of different action. This phenomenon has been explained by the existence of shared internal representations for the execution and perception of actions, which would be automatically activated by the perception of another individual's action. In one interference experiment, null interference was reported when subjects observed a robotic arm perform the incongruent task, suggesting that this effect may be specific to interacting with other humans. This experimental paradigm, designed to investigate motor interference in human interactions, was adapted to investigate how similar the implicit perception of a humanoid robot is to a human agent. Subjects performed rhythmic arm movements while observing either a human agent or humanoid robot performing either congruent or incongruent movements. The variance of the executed movements was used as a measure of the amount of interference in the movements. Both the human and humanoid agents produced significant interference effect. These results suggest that observing the action of humanoid robot and human agent may rely on similar perceptual processes. Furthermore, the ratio of the variance in incongruent to congruent conditions varied between the human agent and humanoid robot. We speculate this ratio describes how the implicit perception of a robot is similar to that of a human, so that this paradigm could provide an objective measure of the reaction to different types of robots and be used to guide the design of humanoid robots interacting with humans. © 2004 IEEE.