Bayesian learning of noisy Markov decision processes

This work addresses the problem of estimating the optimal value function in a Markov Decision Process from observed state-action pairs. We adopt a Bayesian approach to inference, which allows both the model to be estimated and predictions about actions to be made in a unified framework, providing a principled approach to mimicry of a controller on the basis of observed data. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from theposterior distribution over the optimal value function. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Conditional Moment Closure for Turbulent Premixed Flames

The conditional moment closure (CMC) method has been successfully applied to various non-premixed combustion systems in the past, but its application to premixed flames is not fully tested and validated. The main difficulty is associated with the modeling of conditional scalar dissipation rate of the conditioning scalar, the progress variable. A simple algebraic model for the conditional dissipation rate is validated using DNS results of a V-flame. This model along with the standard k- turbulence modeling is used in computations of stoichiometric pilot stabilized Bunsen flames using the RANS-CMC method. A first-order closure is used for the conditional mean reaction rate. The computed non reacting and reacting scalars are in reasonable agreement with the experimental measurements and are consistent with earlier computations using flamelets and transported PDF methods. Sensitivity to chemical kinetic mechanism is also assessed. The results suggest that the CMC may be applied across the regimes of premixed combustion.

Strategies for product recovery decisions

Product recovery is beset by uncertainty regarding the quality of end-of-life (EOL) products, and in order to ascertain the reusability of these products, they have to undergo expensive tests. This undermines the profitability of the recovery process. The key to improve the effectiveness of product recovery is to improve the quality of information available before testing. Emerging data capture technologies can significantly improve the availability of information. However, in order to maximise the potential of these technologies, appropriate decision-making algorithms that exploit such information must be developed. We model the recovery process using a decision-theoretic approach, and derive strategies to ascertain the reusability of EOL products, and also to decide when tests are beneficial. We show that improving the quality of information leads to increase in effectiveness of the recovery process by reducing the need for tests. Copyright © 2009 Inderscience Enterprises Ltd.

Conditional Moment Closure for Turbulent Premixed Flames

The conditional moment closure (CMC) method has been successfully applied to various non-premixed combustion systems in the past, but its application to premixed flames is not fully tested and validated. The main difficulty is associated with the modeling of conditional scalar dissipation rate of the conditioning scalar, the progress variable. A simple algebraic model for the conditional dissipation rate is validated using DNS results of a V-flame. This model along with the standard k- turbulence modeling is used in computations of stoichiometric pilot stabilized Bunsen flames using the RANS-CMC method. A first-order closure is used for the conditional mean reaction rate. The computed non reacting and reacting scalars are in reasonable agreement with the experimental measurements and are consistent with earlier computations using flamelets and transported PDF methods. Sensitivity to chemical kinetic mechanism is also assessed. The results suggest that the CMC may be applied across the regimes of premixed combustion.