Reinforcement learning for parameter estimation in statistical spoken dialogue systems

Reinforcement techniques have been successfully used to maximise the expected cumulative reward of statistical dialogue systems. Typically, reinforcement learning is used to estimate the parameters of a dialogue policy which selects the system's responses based on the inferred dialogue state. However, the inference of the dialogue state itself depends on a dialogue model which describes the expected behaviour of a user when interacting with the system. Ideally the parameters of this dialogue model should be also optimised to maximise the expected cumulative reward. This article presents two novel reinforcement algorithms for learning the parameters of a dialogue model. First, the Natural Belief Critic algorithm is designed to optimise the model parameters while the policy is kept fixed. This algorithm is suitable, for example, in systems using a handcrafted policy, perhaps prescribed by other design considerations. Second, the Natural Actor and Belief Critic algorithm jointly optimises both the model and the policy parameters. The algorithms are evaluated on a statistical dialogue system modelled as a Partially Observable Markov Decision Process in a tourist information domain. The evaluation is performed with a user simulator and with real users. The experiments indicate that model parameters estimated to maximise the expected reward function provide improved performance compared to the baseline handcrafted parameters. © 2011 Elsevier Ltd. All rights reserved.

Parameter estimation for agenda-based user simulation

This paper presents an agenda-based user simulator which has been extended to be trainable on real data with the aim of more closely modelling the complex rational behaviour exhibited by real users. The train-able part is formed by a set of random decision points that may be encountered during the process of receiving a system act and responding with a user act. A sample-based method is presented for using real user data to estimate the parameters that control these decisions. Evaluation results are given both in terms of statistics of generated user behaviour and the quality of policies trained with different simulators. Compared to a handcrafted simulator, the trained system provides a much better fit to corpus data and evaluations suggest that this better fit should result in improved dialogue performance. © 2010 Association for Computational Linguistics.

Parameter optimization for automated concrete detection in image data

Several research studies have been recently initiated to investigate the use of construction site images for automated infrastructure inspection, progress monitoring, etc. In these studies, it is always necessary to extract material regions (concrete or steel) from the images. Existing methods made use of material's special color/texture ranges for material information retrieval, but they do not sufficiently discuss how to find these appropriate color/texture ranges. As a result, users have to define appropriate ones by themselves, which is difficult for those who do not have enough image processing background. This paper presents a novel method of identifying concrete material regions using machine learning techniques. Under the method, each construction site image is first divided into regions through image segmentation. Then, the visual features of each region are calculated and classified with a pre-trained classifier. The output value determines whether the region is composed of concrete or not. The method was implemented using C++ and tested over hundreds of construction site images. The results were compared with the manual classification ones to indicate the method's validity.

The parameter space of graphene chemical vapor deposition on polycrystalline Cu

A systematic study of the parameter space of graphene chemical vapor deposition (CVD) on polycrystalline Cu foils is presented, aiming at a more fundamental process rationale in particular regarding the choice of carbon precursor and mitigation of Cu sublimation. CH 4 as precursor requires H 2 dilution and temperatures ≥1000 °C to keep the Cu surface reduced and yield a high-quality, complete monolayer graphene coverage. The H 2 atmosphere etches as-grown graphene; hence, maintaining a balanced CH 4/H 2 ratio is critical. Such balance is more easily achieved at low-pressure conditions, at which however Cu sublimation reaches deleterious levels. In contrast, C 6H 6 as precursor requires no reactive diluent and consistently gives similar graphene quality at 100-150 °C lower temperatures. The lower process temperature and more robust processing conditions allow the problem of Cu sublimation to be effectively addressed. Graphene formation is not inherently self-limited to a monolayer for any of the precursors. Rather, the higher the supplied carbon chemical potential, the higher the likelihood of film inhomogeneity and primary and secondary multilayer graphene nucleation. For the latter, domain boundaries of the inherently polycrystalline CVD graphene offer pathways for a continued carbon supply to the catalyst. Graphene formation is significantly affected by the Cu crystallography; i.e., the evolution of microstructure and texture of the catalyst template form an integral part of the CVD process. © 2012 American Chemical Society.

Stable multivariate rational interpolation for parameter-dependent aerospace models

A multivariate, robust, rational interpolation method for propagating uncertainties in several dimensions is presented. The algorithm for selecting numerator and denominator polynomial orders is based on recent work that uses a singular value decomposition approach. In this paper we extend this algorithm to higher dimensions and demonstrate its efficacy in terms of convergence and accuracy, both as a method for response suface generation and interpolation. To obtain stable approximants for continuous functions, we use an L2 error norm indicator to rank optimal numerator and denominator solutions. For discontinous functions, a second criterion setting an upper limit on the approximant value is employed. Analytical examples demonstrate that, for the same stencil, rational methods can yield more rapid convergence compared to pseudospectral or collocation approaches for certain problems. © 2012 AIAA.

Between scylla and charybdis: Hydrophobic graphene-guided water diffusion on hydrophilic substrates

The structure of water confined in nanometer-sized cavities is important because, at this scale, a large fraction of hydrogen bonds can be perturbed by interaction with the confining walls. Unusual fluidity properties can thus be expected in the narrow pores, leading to new phenomena like the enhanced fluidity reported in carbon nanotubes. Crystalline mica and amorphous silicon dioxide are hydrophilic substrates that strongly adsorb water. Graphene, on the other hand, interacts weakly with water. This presents the question as to what determines the structure and diffusivity of water when intercalated between hydrophilic substrates and hydrophobic graphene. Using atomic force microscopy, we have found that while the hydrophilic substrates determine the structure of water near its surface, graphene guides its diffusion, favouring growth of intercalated water domains along the C-C bond zigzag direction. Molecular dynamics and density functional calculations are provided to help understand the highly anisotropic water stripe patterns observed.

Bayesian parameter identification for vibro-acoustic models with nonparametric uncertainty

Vibration and acoustic analysis at higher frequencies faces two challenges: computing the response without using an excessive number of degrees of freedom, and quantifying its uncertainty due to small spatial variations in geometry, material properties and boundary conditions. Efficient models make use of the observation that when the response of a decoupled vibro-acoustic subsystem is sufficiently sensitive to uncertainty in such spatial variations, the local statistics of its natural frequencies and mode shapes saturate to universal probability distributions. This holds irrespective of the causes that underly these spatial variations and thus leads to a nonparametric description of uncertainty. This work deals with the identification of uncertain parameters in such models by using experimental data. One of the difficulties is that both experimental errors and modeling errors, due to the nonparametric uncertainty that is inherent to the model type, are present. This is tackled by employing a Bayesian inference strategy. The prior probability distribution of the uncertain parameters is constructed using the maximum entropy principle. The likelihood function that is subsequently computed takes the experimental information, the experimental errors and the modeling errors into account. The posterior probability distribution, which is computed with the Markov Chain Monte Carlo method, provides a full uncertainty quantification of the identified parameters, and indicates how well their uncertainty is reduced, with respect to the prior information, by the experimental data. © 2013 Taylor & Francis Group, London.