Competitive training in hidden Markov models

The use of hidden Markov models is placed in a connectionist framework, and an alternative approach to improving their ability to discriminate between classes is described. Using a network style of training, a measure of discrimination based on the a posteriori probability of state occupation is proposed, and the theory for its optimization using error back-propagation and gradient ascent is presented. The method is shown to be numerically well behaved, and results are presented which demonstrate that when using a simple threshold test on the probability of state occupation, the proposed optimization scheme leads to improved recognition performance.

Recurrent input transformations for Hidden Markov models

This paper presents a new architecture which integrates recurrent input transformations (RIT) and continuous density HMMs. The basic HMM structure is extended to accommodate recurrent neural networks which transform the input observations before they enter the Gaussian output distributions associated with the states of the HMM. During training the parameters of both HMM and RIT are simultaneously optimized according to the Maximum Mutual Information (MMI) criterion. Results are presented for the E-set recognition task which demonstrate the ability of recurrent input transformations to exploit longer term correlations in the speech signal and to give improved discrimination.

APPROXIMATE GAUSS-MARKOV REALISATION OF MULTIVARIATE STOCHASTIC PROCESSES.

Given a spectral density matrix or, equivalently, a real autocovariance sequence, the author seeks to determine a finite-dimensional linear time-invariant system which, when driven by white noise, will produce an output whose spectral density is approximately PHI ( omega ), and an approximate spectral factor of PHI ( omega ). The author employs the Anderson-Faurre theory in his analysis.

Blind equalization and sequence estimation using Markov chain Monte Carlo methods

This paper discusses the problem of restoring a digital input signal that has been degraded by an unknown FIR filter in noise, using the Gibbs sampler. A method for drawing a random sample of a sequence of bits is presented; this is shown to have faster convergence than a scheme by Chen and Li, which draws bits independently. ©1998 IEEE.

On the relationship between hidden Markov Models and convex functional transforms for simulating scanning probe Microscopy

Models for simulating Scanning Probe Microscopy (SPM) may serve as a reference point for validating experimental data and practice. Generally, simulations use a microscopic model of the sample-probe interaction based on a first-principles approach, or a geometric model of macroscopic distortions due to the probe geometry. Examples of the latter include use of neural networks, the Legendre Transform, and dilation/erosion transforms from mathematical morphology. Dilation and the Legendre Transform fall within a general family of functional transforms, which distort a function by imposing a convex solution.In earlier work, the authors proposed a generalized approach to modeling SPM using a hidden Markov model, wherein both the sample-probe interaction and probe geometry may be taken into account. We present a discussion of the hidden Markov model and its relationship to these convex functional transforms for simulating and restoring SPM images.©2009 SPIE.

A markov chain Monte Carlo method for estimating population mixing using Y-chromosome markers: Mixing of the Han people in China

We present a new approach for estimating mixing between populations based on non-recombining markers, specifically Y-chromosome microsatellites. A Markov chain Monte Carlo (MCMC) Bayesian statistical approach is used to calculate the posterior probability

Partially Observable Markov Decision Processes with continuous observations for dialogue management

This work shows how a dialogue model can be represented as a Partially Observable Markov Decision Process (POMDP) with observations composed of a discrete and continuous component. The continuous component enables the model to directly incorporate a confidence score for automated planning. Using a testbed simulated dialogue management problem, we show how recent optimization techniques are able to find a policy for this continuous POMDP which outperforms a traditional MDP approach. Further, we present a method for automatically improving handcrafted dialogue managers by incorporating POMDP belief state monitoring, including confidence score information. Experiments on the testbed system show significant improvements for several example handcrafted dialogue managers across a range of operating conditions.

State estimation schemes for independent component coupled hidden Markov models

Conventional Hidden Markov models generally consist of a Markov chain observed through a linear map corrupted by additive noise. This general class of model has enjoyed a huge and diverse range of applications, for example, speech processing, biomedical signal processing and more recently quantitative finance. However, a lesser known extension of this general class of model is the so-called Factorial Hidden Markov Model (FHMM). FHMMs also have diverse applications, notably in machine learning, artificial intelligence and speech recognition [13, 17]. FHMMs extend the usual class of HMMs, by supposing the partially observed state process is a finite collection of distinct Markov chains, either statistically independent or dependent. There is also considerable current activity in applying collections of partially observed Markov chains to complex action recognition problems, see, for example, [6]. In this article we consider the Maximum Likelihood (ML) parameter estimation problem for FHMMs. Much of the extant literature concerning this problem presents parameter estimation schemes based on full data log-likelihood EM algorithms. This approach can be slow to converge and often imposes heavy demands on computer memory. The latter point is particularly relevant for the class of FHMMs where state space dimensions are relatively large. The contribution in this article is to develop new recursive formulae for a filter-based EM algorithm that can be implemented online. Our new formulae are equivalent ML estimators, however, these formulae are purely recursive and so, significantly reduce numerical complexity and memory requirements. A computer simulation is included to demonstrate the performance of our results. © Taylor & Francis Group, LLC.

Bayesian Learning of Noisy Markov Decision Processes

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Markov过程在森林资源结构动态预测中的应用──以福建省南平地区的树种结构为例

根据马尔可夫决策过程理论和森林资源连续清查的固定样地调查资料，对南平地区的树种结构进行了预测与调整，结果表明，按现状发展，针阔比例将日趋严重，并且毛竹、经济林的占有率呈下降趋势，最终达到以杉木２８．０５％、马尾松１６．６３％、阔叶树１９．０１％、毛竹５．４３％、经济林２．２６％、其它类２８．７１％的树种结构．经调整后稳定状态的树种结构基本趋于合理，即各树种的占有率分别为杉木１８．７２％、马尾松１３．２４％、阔叶树２６．９８％、毛竹１０．８４％、经济林５．４５％、其它类２４．７７％．