Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

This article presents a novel algorithm for learning parameters in statistical dialogue systems which are modeled as Partially Observable Markov Decision Processes (POMDPs). The three main components of a POMDP dialogue manager are a dialogue model representing dialogue state information; a policy that selects the system's responses based on the inferred state; and a reward function that specifies the desired behavior of the system. Ideally both the model parameters and the policy would be designed to maximize the cumulative reward. However, while there are many techniques available for learning the optimal policy, no good ways of learning the optimal model parameters that scale to real-world dialogue systems have been found yet. The presented algorithm, called the Natural Actor and Belief Critic (NABC), is a policy gradient method that offers a solution to this problem. Based on observed rewards, the algorithm estimates the natural gradient of the expected cumulative reward. The resulting gradient is then used to adapt both the prior distribution of the dialogue model parameters and the policy parameters. In addition, the article presents a variant of the NABC algorithm, called the Natural Belief Critic (NBC), which assumes that the policy is fixed and only the model parameters need to be estimated. The algorithms are evaluated on a spoken dialogue system in the tourist information domain. The experiments show that model parameters estimated to maximize the expected cumulative reward result in significantly improved performance compared to the baseline hand-crafted model parameters. The algorithms are also compared to optimization techniques using plain gradients and state-of-the-art random search algorithms. In all cases, the algorithms based on the natural gradient work significantly better. © 2011 ACM.

User responses to the learning demands of consumer products

Designers are typically male, under 35 years old and unimpaired. Users can be of any age and currently over 15% will have some form of impairment. As a result a vast array of consumer products suit youthful males and in many cases exclude other demographics (e.g. Keates and Clarkson, 2004). In studying the way a range of users learn how to use new products, key cognitive difficulties are revealed and linked back to the areas of the product causing the problems. The trials were structured so each user had to complete a specific set of tasks and were consistent across the user spectrum. The tasks set aimed to represent both everyday usage and less familiar functions. Whilst the knowledge gained could provide designers with valuable guidelines for the specific products examined, a more general abstraction provides knowledge of the pitfalls to avoid in the design of other product families.

Learning the dynamics of the external world: Brain inspired learning for robotic applications

Humans have exceptional abilities to learn new skills, manipulate tools and objects, and interact with our environment. In order to be successful at these tasks, our brain has become exceptionally well adapted to learning to deal not only with the complex dynamics of our own limbs but also with novel dynamics in the external world. While learning of these dynamics includes learning the complex time-varying forces at the end of limbs through the updating of internal models, it must also include learning the appropriate mechanical impedance in order to stabilize both the limb and any objects contacted in the environment. This article reviews the field of human learning by examining recent experimental evidence about adaptation to novel unstable dynamics and explores how this knowledge about the brain and neuro-muscular system can expand the learning capabilities of robotics and prosthetics. © 2006.

Statistics anxiety, trait anxiety, learning behavior, and academic performance

The present study investigated the relationship between statistics anxiety, individual characteristics (e.g., trait anxiety and learning strategies), and academic performance. Students enrolled in a statistics course in psychology (N=147) filled in a questionnaire on statistics anxiety, trait anxiety, interest in statistics, mathematical selfconcept, learning strategies, and procrastination. Additionally, their performance in the examination was recorded. The structural equation model showed that statistics anxiety held a crucial role as the strongest direct predictor of performance. Students with higher statistics anxiety achieved less in the examination and showed higher procrastination scores. Statistics anxiety was related indirectly to spending less effort and time on learning. Trait anxiety was related positively to statistics anxiety and, counterintuitively, to academic performance. This result can be explained by the heterogeneity of the measure of trait anxiety. The part of trait anxiety that is unrelated to the specific part of statistics anxiety correlated positively with performance.