Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

This article presents a novel algorithm for learning parameters in statistical dialogue systems which are modeled as Partially Observable Markov Decision Processes (POMDPs). The three main components of a POMDP dialogue manager are a dialogue model representing dialogue state information; a policy that selects the system's responses based on the inferred state; and a reward function that specifies the desired behavior of the system. Ideally both the model parameters and the policy would be designed to maximize the cumulative reward. However, while there are many techniques available for learning the optimal policy, no good ways of learning the optimal model parameters that scale to real-world dialogue systems have been found yet. The presented algorithm, called the Natural Actor and Belief Critic (NABC), is a policy gradient method that offers a solution to this problem. Based on observed rewards, the algorithm estimates the natural gradient of the expected cumulative reward. The resulting gradient is then used to adapt both the prior distribution of the dialogue model parameters and the policy parameters. In addition, the article presents a variant of the NABC algorithm, called the Natural Belief Critic (NBC), which assumes that the policy is fixed and only the model parameters need to be estimated. The algorithms are evaluated on a spoken dialogue system in the tourist information domain. The experiments show that model parameters estimated to maximize the expected cumulative reward result in significantly improved performance compared to the baseline hand-crafted model parameters. The algorithms are also compared to optimization techniques using plain gradients and state-of-the-art random search algorithms. In all cases, the algorithms based on the natural gradient work significantly better. © 2011 ACM.

New approaches to materials education for students of engineering

A novel approach to the teaching of materials to engineering students is outlined. It starts from the overview of the "world" of materials made possible by material property charts, and develops both an understanding of material properties and skills in selecting materials and processes to meet design specifications. It is supported by extensive computer-based methods and tools, and is well adapted both for elementary and for advanced courses.

Load-displacement behavior during sharp indentation of viscous-elastic-plastic materials

A constitutive equation is developed for geometrically-similar sharp indentation of a material capable of elastic, viscous, and plastic deformation. The equation is based on a series of elements consisting of a quadratic (reversible) spring, a quadratic (time-dependent, reversible) dashpot, and a quadratic (time-independent, irreversible) slider-essentially modifying a model for an elastic-perfectly plastic material by incorporating a creeping component. Load-displacement solutions to the constitutive equation are obtained for load-controlled indentation during constant loading-rate testing. A characteristic of the responses is the appearance of a forward-displacing "nose" during unloading of load-controlled systems (e.g., magnetic-coil-driven "nanoindentation" systems). Even in the absence of this nose, and the associated initial negative unloading tangent, load-displacement traces (and hence inferred modulus and hardness values) are significantly perturbed on the addition of the viscous component. The viscous-elastic-plastic (VEP) model shows promise for obtaining material properties (elastic modulus, hardness, time-dependence) of time-dependent materials during indentation experiments.

Design and process selection for joints in flexible materials

The application of high performance textiles has grown significantly in the last 10 to 15 years. Various research groups throughout the United Kingdom, such as the Department of Trade and Industry, have identified technical textiles as a field for future development. There is little design guidance for joining of flexible materials or general property models that can be applied to theses materials. This lack is due to the large diversity of properties, structures and resulting behaviours of the materials that are classified as "Flexible Materials". This dissertation explores the issues that are involved in characterising the materials at the fibre, bulk and textile levels. Different units of measurement are used for each stage of the manufacturing process of flexible materials and this disparity creates problems when trying to make general comparisons (e.g. comparing textiles to polymer films). Thus, a possible solution to this is to create selection charts that allow designers to compare the strength of materials for a given mass per unit area. A design tool was created using the Cambridge Engineering Selector (CES) software to enable the selection of joining processes for material. The tool is effective in selecting a reduced number of viable joining processes. Through case studies it was shown that designers are required to examine the selected processes (identified by the software) in greater detail - in particular the economics and geometry of the joint - in order to identify the optimum joining process.