Seeing the Wood for the Trees: A Critical Evaluation of Methods to Estimate the Parameters of Stochastic Differential Equations

Maximum-likelihood estimates of the parameters of stochastic differential equations are consistent and asymptotically efficient, but unfortunately difficult to obtain if a closed-form expression for the transitional probability density function of the process is not available. As a result, a large number of competing estimation procedures have been proposed. This article provides a critical evaluation of the various estimation techniques. Special attention is given to the ease of implementation and comparative performance of the procedures when estimating the parameters of the Cox–Ingersoll–Ross and Ornstein–Uhlenbeck equations respectively.

Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

Radiation effects on natural convection laminar flow from a horizontal circular cylinder

The effect of radiation on natural convection flow from an isothermal circular cylinder has been investigated numerically in this study. The governing boundary layer equations of motion are transformed into a non-dimensional form and the resulting nonlinear systems of partial differential equations are reduced to convenient boundary layer equations, which are then solved numerically by two distinct efficient methods namely: (i) implicit finite differencemethod or the Keller-Box Method (KBM) and (ii) Straight Forward Finite Difference Method (SFFD). Numerical results are presented by velocity and temperature distribution of the fluid as well as heat transfer characteristics, namely the shearing stress and the local heat transfer rate in terms of the local skin-friction coefficient and the local Nusselt number for a wide range of surface heating parameter and radiation-conduction parameter. Due to the effects of the radiation the skin-friction coefficients as well as the rate of heat transfer increased and consequently the momentum and thermal boundary layer thickness enhanced.

The effect of temperature dependent viscosity on MHD natural convection flow from an isothermal sphere

Laminar magnetohydrodynamic (MHD) natural convection flow from an isothermal sphere immersed in a fluid with viscosity proportional to linear function of temperature has been studied. The governing boundary layer equations are transformed into a non-dimensional form and the resulting nonlinear system of partial differential equations are reduced to convenient form which are solved numerically by two very efficient methods, namely, (i) Implicit finite difference method together with Keller box scheme and (ii) Direct numerical scheme. Numerical results are presented by velocity and temperature distribution, streamlines and isotherms of the fluid as well as heat transfer characteristics, namely the local skin-friction coefficients and the local heat transfer rate for a wide range of magnetohydrodynamic paramagnet and viscosity-variation parameter.

Geometry vs appearance for discriminating between posed and spontaneous emotions

Spontaneous facial expressions differ from posed ones in appearance, timing and accompanying head movements. Still images cannot provide timing or head movement information directly. However, indirectly the distances between key points on a face extracted from a still image using active shape models can capture some movement and pose changes. This information is superposed on information about non-rigid facial movement that is also part of the expression. Does geometric information improve the discrimination between spontaneous and posed facial expressions arising from discrete emotions? We investigate the performance of a machine vision system for discrimination between posed and spontaneous versions of six basic emotions that uses SIFT appearance based features and FAP geometric features. Experimental results on the NVIE database demonstrate that fusion of geometric information leads only to marginal improvement over appearance features. Using fusion features, surprise is the easiest emotion (83.4% accuracy) to be distinguished, while disgust is the most difficult (76.1%). Our results find different important facial regions between discriminating posed versus spontaneous version of one emotion and classifying the same emotion versus other emotions. The distribution of the selected SIFT features shows that mouth is more important for sadness, while nose is more important for surprise, however, both the nose and mouth are important for disgust, fear, and happiness. Eyebrows, eyes, nose and mouth are important for anger.