Bayesian learning of noisy Markov decision processes

This work addresses the problem of estimating the optimal value function in a Markov Decision Process from observed state-action pairs. We adopt a Bayesian approach to inference, which allows both the model to be estimated and predictions about actions to be made in a unified framework, providing a principled approach to mimicry of a controller on the basis of observed data. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from theposterior distribution over the optimal value function. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Progress In Modeling Of Fluid Flows In Crystal Growth Processes

Modeling of fluid flows in crystal growth processes has become an important research area in theoretical and applied mechanics. Most crystal growth processes involve fluid flows, such as flows in the melt, solution or vapor. Theoretical modeling has played an important role in developing technologies used for growing semiconductor crystals for high performance electronic and optoelectronic devices. The application of devices requires large diameter crystals with a high degree of crystallographic perfection, low defect density and uniform dopant distribution. In this article, the flow models developed in modeling of the crystal growth processes such as Czochralski, ammonothermal and physical vapor transport methods are reviewed. In the Czochralski growth modeling, the flow models for thermocapillary flow, turbulent flow and MHD flow have been developed. In the ammonothermal growth modeling, the buoyancy and porous media flow models have been developed based on a single-domain and continuum approach for the composite fluid-porous layer systems. In the physical vapor transport growth modeling, the Stefan flow model has been proposed based on the flow-kinetics theory for the vapor growth. In addition, perspectives for future studies on crystal growth modeling are proposed. (c) 2008 National Natural Science Foundation of China and Chinese Academy of Sciences. Published by Elsevier Limited and Science in China Press. All rights reserved.

Comparison Of The Quasi-Static Method And The Dynamic Method For Simulating Fracture Processes In Concrete

Concrete is heterogeneous and usually described as a three-phase material, where matrix, aggregate and interface are distinguished. To take this heterogeneity into consideration, the Generalized Beam (GB) lattice model is adopted. The GB lattice model is much more computationally efficient than the beam lattice model. Numerical procedures of both quasi-static method and dynamic method are developed to simulate fracture processes in uniaxial tensile tests conducted on a concrete panel. Cases of different loading rates are compared with the quasi-static case. It is found that the inertia effect due to load increasing becomes less important and can be ignored with the loading rate decreasing, but the inertia effect due to unstable crack propagation remains considerable no matter how low the loading rate is. Therefore, an unrealistic result will be obtained if a fracture process including unstable cracking is simulated by the quasi-static procedure.

Observation and modeling for terrestrial processes in alpine meadow

The water-heat transfer process between land and atmosphere in Haibei alpine meadow area has been systematically observed. A multi-layer coupling model for land-atmosphere interaction was presented with special attention paid to the moisture transfer in leaf stomata under unsaturated condition. A profound investigation on the physical process of turbulent transfer inside the vegetation has been performed with a revised formula of water absorption for root system. The present model facilitates the study of vertically distributed physical variables in detail. Numerical simulation was conducted according to the transfer process of Kinesia humility meadow in the area of Haibei Alpine Meadow Ecosystem Station, CAS. The calculated results agree well with observation.