Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

This article presents a novel algorithm for learning parameters in statistical dialogue systems which are modeled as Partially Observable Markov Decision Processes (POMDPs). The three main components of a POMDP dialogue manager are a dialogue model representing dialogue state information; a policy that selects the system's responses based on the inferred state; and a reward function that specifies the desired behavior of the system. Ideally both the model parameters and the policy would be designed to maximize the cumulative reward. However, while there are many techniques available for learning the optimal policy, no good ways of learning the optimal model parameters that scale to real-world dialogue systems have been found yet. The presented algorithm, called the Natural Actor and Belief Critic (NABC), is a policy gradient method that offers a solution to this problem. Based on observed rewards, the algorithm estimates the natural gradient of the expected cumulative reward. The resulting gradient is then used to adapt both the prior distribution of the dialogue model parameters and the policy parameters. In addition, the article presents a variant of the NABC algorithm, called the Natural Belief Critic (NBC), which assumes that the policy is fixed and only the model parameters need to be estimated. The algorithms are evaluated on a spoken dialogue system in the tourist information domain. The experiments show that model parameters estimated to maximize the expected cumulative reward result in significantly improved performance compared to the baseline hand-crafted model parameters. The algorithms are also compared to optimization techniques using plain gradients and state-of-the-art random search algorithms. In all cases, the algorithms based on the natural gradient work significantly better. © 2011 ACM.

Heat source modelling and natural ventilation efficiency

We compare natural ventilation flows established by a range of heat source distributions at floor level. Both evenly distributed and highly localised line and point source distributions are considered. We demonstrate that modelling the ventilation flow driven by a uniformly distributed heat source is equivalent to the flow driven by a large number of localised sources. A model is developed for the transient flow development in a room with a uniform heat distribution and is compared with existing models for localised buoyancy inputs. For large vent areas the flow driven by localised heat sources reaches a steady state more rapidly than the uniformly distributed case. For small vent areas there is little difference in the transient development times. Our transient model is then extended to consider the time taken to flush a neutrally buoyant pollutant from a naturally ventilated room. Again comparisons are drawn between uniform and localised (point and line) heat source geometries. It is demonstrated that for large vent areas a uniform heat distribution provides the fastest flushing. However, for smaller vent areas, localised heat sources produce the fastest flushing. These results are used to suggest a definition for the term 'natural ventilation efficiency', and a model is developed to estimate this efficiency as a function of the room and heat source geometries. © 2006 Elsevier Ltd. All rights reserved.

Pollutant flushing with natural displacement ventilation

We examine the time taken to flush pollutants from a naturally ventilated room. A simple theoretical model is developed to predict the time taken for neutrally-buoyant pollutants to be removed from a room by a flow driven by localised heat inputs; both line and point heat sources are considered. We show that the rate of flushing is a function of the room volume, vent areas ( A) and the distribution, number (n) and strength (B) of the heat sources. We also show that the entire problem can be reduced to a single parameter ( μ) that is a measure of the vent areas, and a dimensionless time ( τ) that is a function of B, V and μ. Small-scale salt-bath experiments were conducted to measure the flushing rates in order to validate our modelling assumptions and predictions. The predicted flushing times show good agreement with the experiments over a wide range of μ. We apply our model to a typical open plan office and lecture theatre and discuss some of the implications of our results. © 2005 Elsevier Ltd. All rights reserved.

On the applicability of the lognormal distribution in random dynamical systems

This paper is concerned with the probability density function of the energy of a random dynamical system subjected to harmonic excitation. It is shown that if the natural frequencies and mode shapes of the system conform to the Gaussian Orthogonal Ensemble, then under common types of loading the distribution of the energy of the response is approximately lognormal, providing the modal overlap factor is high (typically greater than two). In contrast, it is shown that the response of a system with Poisson natural frequencies is not approximately lognormal. Numerical simulations are conducted on a plate system to validate the theoretical findings and good agreement is obtained. Simulations are also conducted on a system made from two plates connected with rotational springs to demonstrate that the theoretical findings can be extended to a built-up system. The work provides a theoretical justification of the commonly used empirical practice of assuming that the energy response of a random system is lognormal.

Unified compression model for venice lagoon natural silts

Over the last 50 years, the city of Venice, Italy, has observed a significant increase in the frequency of flooding. Numerous engineering solutions have been proposed, including the use of movable gates located at the three lagoon inlets. A key element in the prediction of performance is the estimation of settlements of the foundation system of the gates. The soils of Venice Lagoon are characterized by very erratic depositional patterns of clayey silts, resulting in an extremely heterogeneous stratigraphy with discontinuous layering. The soils are also characterized by varying contents of coarse and fine-grained particles. In contrast, the mineralogical composition of these deposits is quite uniform, which allows us to separate the influence of mineralogy from that of grain size distribution. A comprehensive geotechnical testing program was performed to assess the one-dimensional compression of Venice soils and examine the factors affecting the response in the transition from one material type to another. The compressibility of these natural silty clayey soils can be described by a single set of constitutive laws incorporating the relative fraction of granular to cohesive material. © 2007 ASCE.