Real user evaluation of spoken dialogue systems using Amazon Mechanical Turk

This paper describes a framework for evaluation of spoken dialogue systems. Typically, evaluation of dialogue systems is performed in a controlled test environment with carefully selected and instructed users. However, this approach is very demanding. An alternative is to recruit a large group of users who evaluate the dialogue systems in a remote setting under virtually no supervision. Crowdsourcing technology, for example Amazon Mechanical Turk (AMT), provides an efficient way of recruiting subjects. This paper describes an evaluation framework for spoken dialogue systems using AMT users and compares the obtained results with a recent trial in which the systems were tested by locally recruited users. The results suggest that the use of crowdsourcing technology is feasible and it can provide reliable results. Copyright © 2011 ISCA.

Uncertainty management for on-line optimisation of a POMDP-based large-scale spoken dialogue system

The optimization of dialogue policies using reinforcement learning (RL) is now an accepted part of the state of the art in spoken dialogue systems (SDS). Yet, it is still the case that the commonly used training algorithms for SDS require a large number of dialogues and hence most systems still rely on artificial data generated by a user simulator. Optimization is therefore performed off-line before releasing the system to real users. Gaussian Processes (GP) for RL have recently been applied to dialogue systems. One advantage of GP is that they compute an explicit measure of uncertainty in the value function estimates computed during learning. In this paper, a class of novel learning strategies is described which use uncertainty to control exploration on-line. Comparisons between several exploration schemes show that significant improvements to learning speed can be obtained and that rapid and safe online optimisation is possible, even on a complex task. Copyright © 2011 ISCA.

Computer generated holograms for carbon nanotube arrays.

Multiwalled carbon nanotubes are highly diffractive structures in the optical regime. Their metallic character and large scattering cross-section allow their usage as diffractive elements in Fraunhofer holograms. This work elaborates some important features of the far field diffraction patterns produced from periodic arrays of nanotubes. A theoretical approach for the interaction of arrays of nanotubes with light is presented and a computer generated hologram is calculated by means of periodical patterns. Based on the results, fabrication of carbon nanotube arrays (in holographic patterns) was performed. Experimentally measured diffraction patterns were in good agreement with the calculations.

Designing intrinsically motivating user interfaces for the ageing population

Older people often find it difficult to learn to use new technology. Although they may want to adopt it, they can find the learning process challenging and frustrating and subsequently lose motivation. This paper looks at how psychological theories of intrinsic motivation could be applied to make the ICT learning process more engaging for older users and describes an experiment set up to test the applicability of these theories to user interface (UI) design. The results of the experiment confirmed that intrinsic motivation theory is a valid lens through which to look at current ICT design and also uncovered significant gender differences in reaction to different kinds of learning tasks. © 2013 Springer-Verlag Berlin Heidelberg.

User target intention recognition from cursor position using kalman filter

This paper discusses user target intention recognition algorithms for pointing - clicking tasks to reduce users' pointing time and difficulty. Predicting targets by comparing the bearing angles to targets proposed as one of the first algorithms [1] is compared with a Kalman Filter prediction algorithm. Accuracy and sensitivity of prediction are used as performance criteria. The outcomes of a standard point and click experiment are used for performance comparison, collected from both able-bodied and impaired users. © 2013 Springer-Verlag Berlin Heidelberg.