Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

This article presents a novel algorithm for learning parameters in statistical dialogue systems which are modeled as Partially Observable Markov Decision Processes (POMDPs). The three main components of a POMDP dialogue manager are a dialogue model representing dialogue state information; a policy that selects the system's responses based on the inferred state; and a reward function that specifies the desired behavior of the system. Ideally both the model parameters and the policy would be designed to maximize the cumulative reward. However, while there are many techniques available for learning the optimal policy, no good ways of learning the optimal model parameters that scale to real-world dialogue systems have been found yet. The presented algorithm, called the Natural Actor and Belief Critic (NABC), is a policy gradient method that offers a solution to this problem. Based on observed rewards, the algorithm estimates the natural gradient of the expected cumulative reward. The resulting gradient is then used to adapt both the prior distribution of the dialogue model parameters and the policy parameters. In addition, the article presents a variant of the NABC algorithm, called the Natural Belief Critic (NBC), which assumes that the policy is fixed and only the model parameters need to be estimated. The algorithms are evaluated on a spoken dialogue system in the tourist information domain. The experiments show that model parameters estimated to maximize the expected cumulative reward result in significantly improved performance compared to the baseline hand-crafted model parameters. The algorithms are also compared to optimization techniques using plain gradients and state-of-the-art random search algorithms. In all cases, the algorithms based on the natural gradient work significantly better. © 2011 ACM.

Modelling the evolution of uncertainty levels during design

Design work involves uncertainty that arises from, and influences, the progressive development of solutions. This paper analyses the influences of evolving uncertainty levels on the design process. We focus on uncertainties associated with choosing the values of design parameters, and do not consider in detail the issues that arise when parameters must first be identified. Aspects of uncertainty and its evolution are discussed, and a new task-based model is introduced to describe process behaviour in terms of changing uncertainty levels. The model is applied to study two process configuration problems based on aircraft wing design: one using an analytical solution and one using Monte-Carlo simulation. The applications show that modelling uncertainty levels during design can help assess management policies, such as how many concepts should be considered during design and to what level of accuracy. © 2011 Springer-Verlag.

Min-Max Moving Horizon Estimation for a Class of Hybrid Systems

In this paper, a strategy for min-max Moving Horizon Estimation (MHE) of a class of uncertain hybrid systems is proposed. The class of hybrid systems being considered are Piecewise Affine systems (PWA) with both continuous valued and logic components. Furthermore, we consider the case when there is a (possibly structured) norm bounded uncertainty in each subsystem. Sufficient conditions on the time horizon and the penalties on the state at the beginning of the estimation horizon to guarantee convergence of the MHE scheme will be provided. The MHE scheme will be implemented as a mixed integer semidefinite optimisation for which an efficient algorithm was recently introduced.

Guidelines for evaluating the ease of reconfiguration of manufacturing systems

This paper presents guidelines for evaluating the ease of reconfiguration of manufacturing systems. Based reconfigurability measurement tools proposed in the past [1], [2]and a reconfiguration process model, the paper proposes a method that can be used to assess different system characteristics. After the guideline is presented, an example of how the method can be applied to a batch processing system is given. It is found that the proposed method can be applied to batch processing systems based on ISA S88 batch server commercial software. ©2008 IEEE.

The evolution of information while building cross-domain models of a design: A video experiment

Models capturing the connectivity between different domains of a design, e.g. between components and functions, can provide a tool for tracing and analysing aspects of that design. In this paper, video experiments are used to explore the role of cross-domain modelling in building up information about a design. The experiments highlight that cross-domain modelling can be a useful tool to create and structure design information. Findings suggest that consideration of multiple domains encourages discussion during modelling, helps identify design aspects that might otherwise be overlooked, and can help promote consideration of alternative design options. Copyright © 2002-2012 The Design Society. All rights reserved.