Bayesian Learning of Noisy Markov Decision Processes

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Aerodynamic design of high end wall angle turbine stages- part II: Experimental verification

An experimental investigation of a turbine stage featuring very high end wall angles is presented. The initial turbine design did not achieve a satisfactory performance and the difference between the design predictions and the test results was traced to a large separated region on the rear suction-surface. To improve the agreement between computational fluid dynamics (CFD) and experiment, it was found necessary to modify the turbulence modeling employed. The modified CFD code was then used to redesign the vane, and the changes made are described. When tested, the performance of the redesigned vane was found to have much closer agreement with the predictions than the initial vane. Finally, the flowfield and performance of the redesigned stage are compared to a similar turbine, designed to perform the same duty, which lies in an annulus of moderate end wall angles. A reduction in stage efficiency of at least 2.4% was estimated for the very high end wall angle design. © 2014 by ASME.

Delayed-decision making in noisy bistable switches

Bistable switches are frequently encountered in biological systems. Typically, a bistable switch models a binary decision where each decision corresponds to one of the two stable equilibria. Recently, we showed that the global decision-making process in bistable switches strongly depends on a particular equilibrium point of these systems, their saddle point. In particular, we showed that a saddle point with a time-scale separation between its attractive and repulsive directions can delay the decision-making process. In this paper, we study the effects of white Gaussian noise on this mechanism of delayed decision-making induced by the saddle point. Results show that the mean decision-time strongly depends on the balance between the initial distance to the separatrix and the noise strength. © IFAC.