Markov chains, R-trivial monoids and representation theory

We develop a general theory of Markov chains realizable as random walks on R-trivial monoids. It provides explicit and simple formulas for the eigenvalues of the transition matrix, for multiplicities of the eigenvalues via Mobius inversion along a lattice, a condition for diagonalizability of the transition matrix and some techniques for bounding the mixing time. In addition, we discuss several examples, such as Toom-Tsetlin models, an exchange walk for finite Coxeter groups, as well as examples previously studied by the authors, such as nonabelian sandpile models and the promotion Markov chain on posets. Many of these examples can be viewed as random walks on quotients of free tree monoids, a new class of monoids whose combinatorics we develop.

Ergodic control of partially observed Markov chains

The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear filter. Dynamic programming equations for the latter are derived, leading to existence and characterization of optimal stationary policies.

On the Milito-Cruz adaptive control scheme for Markov chains

Milito and Cruz have introduced a novel adaptive control scheme for finite Markov chains when a finite parametrized family of possible transition matrices is available. The scheme involves the minimization of a composite functional of the observed history of the process incorporating both control and estimation aspects. We prove the a.s. optimality of a similar scheme when the state space is countable and the parameter space a compact subset ofR.

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

Markov chain splitting methods in structural reliability integral estimation

Monte Carlo simulation methods involving splitting of Markov chains have been used in evaluation of multi-fold integrals in different application areas. We examine in this paper the performance of these methods in the context of evaluation of reliability integrals from the point of view of characterizing the sampling fluctuations. The methods discussed include the Au-Beck subset simulation, Holmes-Diaconis-Ross method, and generalized splitting algorithm. A few improvisations based on first order reliability method are suggested to select algorithmic parameters of the latter two methods. The bias and sampling variance of the alternative estimators are discussed. Also, an approximation to the sampling distribution of some of these estimators is obtained. Illustrative examples involving component and series system reliability analyses are presented with a view to bring out the relative merits of alternative methods. (C) 2015 Elsevier Ltd. All rights reserved.

Asymptotic behaviour of point processes with Markov-dependent intervals

This splitting techniques for MARKOV chains developed by NUMMELIN (1978a) and ATHREYA and NEY (1978b) are used to derive an imbedded renewal process in WOLD's point process with MARKOV-correlated intervals. This leads to a simple proof of renewal theorems for such processes. In particular, a key renewal theorem is proved, from which analogues to both BLACKWELL's and BREIMAN's forms of the renewal theorem can be deduced.

Markov Chain Model for Cracking Behavior of Reinforced Concrete Beams

In this paper, nonhomogeneous Markov chains are proposed for modeling the cracking behavior of reinforced concrete beams subjected to monotonically increasing loads. The model facilitates prediction of the maximum crackwidth at a given load given the crackwidth at a lower load level, and thus leads to a better understanding of the cracking phenomenon. To illustrate the methodology developed, the results of three reinforced concrete beams tested in the laboratory are analyzed and presented.

Markov chain modeling of evolution of strains in reinforced concrete flexural beams

From the analysis of experimentally observed variations in surface strains with loading in reinforced concrete beams, it is noted that there is a need to consider the evolution of strains (with loading) as a stochastic process. Use of Markov Chains for modeling stochastic evolution of strains with loading in reinforced concrete flexural beams is studied in this paper. A simple, yet practically useful, bi-level homogeneous Gaussian Markov Chain (BLHGMC) model is proposed for determining the state of strain in reinforced concrete beams. The BLHGMC model will be useful for predicting behavior/response of reinforced concrete beams leading to more rational design.

Hybrid aggregated-vector algorithm for efficient parallelization of fast multipole method

An efficient parallelization algorithm for the Fast Multipole Method which aims to alleviate the parallelization bottleneck arising from lower job-count closer to root levels is presented. An electrostatic problem of 12 million non-uniformly distributed mesh elements is solved with 80-85% parallel efficiency in matrix setup and matrix-vector product using 60GB and 16 threads on shared memory architecture.

Structure of 2-(2,6-dimethoxyphenyl)-2,5- dimethylbicyclo[3.2.1]octane-6,8-dione

C18H2204, orthorhombic, P212~21, a = 7.343 (4), b = 11.251 (4), c = 19.357 (4)A, Z = 4, Dr, ' = 1.20, D e = 1.254 g cm -3, F(000) = 648, p(MoKa) = 0.94 cm -~. X-ray intensity data were collected on a Nonius CAD-4 diffractometer and the structure was solved by direct methods. Full-matrix least-squares refinement gave R = 0.052 (R w = 0.045) for 1053 observed reflections. The stereochemical configuration at C(2) has been shown to be 2-exo-methyl-2-endo-(2,6-dimethoxyphenyl), i.e. (3) in contrast to the structure (2) assigned earlier based on its ~H NMR data.

Structure of 2-(2,6-Dimethoxyphenyl)-2,5-dimethylbicyclo[3.2. l]octane-6,8-dione

C18H2204, orthorhombic, P212~21, a = 7.343 (4), b = 11.251 (4), c = 19.357 (4)A, Z = 4, Dr, ' = 1.20, D e = 1.254 g cm -3, F(000) = 648, p(Mo Ka) = 0.94 cm -~. X-ray intensity data were collected on a Nonius CAD-4 diffractometer and the structure was solved by direct methods. Full-matrix least-squares refinement gave R = 0.052 (R w = 0.045) for 1053 observed reflections. The stereochemical configuration at C(2) has been shown to be 2-exo-methyl-2-endo- (2,6-dimethoxyphenyl), i.e. (3) in contrast to the structure (2) assigned earlier based on its ~H NMR data.