Effects of the user model on simulation-based learning of dialogue strategies

Over the past decade, a variety of user models have been proposed for user simulation-based reinforcement-learning of dialogue strategies. However, the strategies learned with these models are rarely evaluated in actual user trials and it remains unclear how the choice of user model affects the quality of the learned strategy. In particular, the degree to which strategies learned with a user model generalise to real user populations has not be investigated. This paper presents a series of experiments that qualitatively and quantitatively examine the effect of the user model on the learned strategy. Our results show that the performance and characteristics of the strategy are in fact highly dependent on the user model. Furthermore, a policy trained with a poor user model may appear to perform well when tested with the same model, but fail when tested with a more sophisticated user model. This raises significant doubts about the current practice of learning and evaluating strategies with the same user model. The paper further investigates a new technique for testing and comparing strategies directly on real human-machine dialogues, thereby avoiding any evaluation bias introduced by the user model. © 2005 IEEE.

A systematic approach for improving the quality of official statistics: Case of Korean information and telecommunication industry

With the rapid growth of information and communication technology (ICT) in Korea, there was a need to improve the quality of official ICT statistics. In order to do this, various factors had to be considered, such as the quality of surveying, processing, and output as well as the reputation of the statistical agency. We used PLS estimation to determine how these factors might influence customer satisfaction. Furthermore, through a comparison of associated satisfaction indices, we provided feedback to the responsible statistics agency. It appears that our model can be used as a tool for improving the quality of official ICT statistics. © 2008 Elsevier B.V. All rights reserved.