Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.

Risk-Sensitive Ergodic Control of Continuous Time Markov Processes With Denumerable State Space

In this article, we study risk-sensitive control problem with controlled continuous time Markov chain state dynamics. Using multiplicative dynamic programming principle along with the atomic structure of the state dynamics, we prove the existence and a characterization of optimal risk-sensitive control under geometric ergodicity of the state dynamics along with a smallness condition on the running cost.

Magnetic trapping of Yb in the metastable P-3(2) state

We report magnetic trapping of Yb in the excited P-3(2) state. This state, with a lifetime of 15 s, could play an important role in studies ranging from optical clocks and quantum computation to the search for a permanent electric dipole moment. Yb atoms are first cooled and trapped in the ground state in a 399-nm magneto-optic trap. The cold atoms are then pumped into the excited state by driving the S-1(0) -> P-3(1) -> S-3(1) transition. Atoms in the P-3(2) state are magnetically trapped in a spherical quadrupole field with an axial gradient of 110 G/cm. We trap up to 10(6) atoms with a lifetime of 1.5 s.

Critical age-dependent branching Markov processes and their scaling limits

This paper studies:(i)the long-time behaviour of the empirical distribution of age and normalized position of an age-dependent critical branching Markov process conditioned on non-extinction;and (ii) the super-process limit of a sequence of age-dependent critical branching Brownian motions.

Precise measurement of hyperfine structure in the 2P(1/2) state of Li-7 using saturated-absorption spectroscopy

We report a precise measurement of the hyperfine interval in the 2P(1/2) state of Li-7. The transition from the ground state (D-1 line) is accessed using a diode laser and the technique of saturated-absorption spectroscopy in hot Li vapor. The interval is measured by locking an acousto-optic modulator to the frequency difference between the two hyperfine peaks. The measured interval of 92.040(6) MHz is consistent with an earlier measurement reported by us using an atomic-beam spectrometer Das and Natarajan, J. Phys. B 41, 035001 (2008)]. The interval yields the magnetic dipole constant in the P-1/2 state as A = 46.047(3), which is discrepant from theoretical calculations by > 80 kHz.

Supercritical age-dependent branching Markov processes and their scaling limits

This paper studies the long-time behavior of the empirical distribution of age and normalized position of an age-dependent supercritical branching Markov process. The motion of each individual during its life is a random function of its age. It is shown that the empirical distribution of the age and the normalized position of all individuals alive at time t converges as t -> infinity to a deterministic product measure.

Residence time distribution for a class of Gaussian Markov processes

We study the distribution of residence time or equivalently that of "mean magnetization" for a family of Gaussian Markov processes indexed by a positive parameter alpha. The persistence exponent for these processes is simply given by theta=alpha but the residence time distribution is nontrivial. The shape of this distribution undergoes a qualitative change as theta increases, indicating a sharp change in the ergodic properties of the process. We develop two alternate methods to calculate exactly but recursively the moments of the distribution for arbitrary alpha. For some special values of alpha, we obtain closed form expressions of the distribution function. [S1063-651X(99)03306-1].

Observation of the nuclear magnetic octupole moment of Yb-173 from precise measurements of the hyperfine structure in the P-3(2) state

We measure hyperfine structure in the metastable P-3(2) state of Yb-173 and extract the nuclear magnetic octupole moment. We populate the state using dipole-allowed transitions through the P-3(1) and S-3(1) states. We measure frequencies of hyperfine transitions of the P-3(2) -> S-3(1) line at 770 nm using a Rb-stabilized ring cavity resonator with a precision of 200 kHz. Second-order corrections due to perturbations from the nearby P-3(1) and P-1(1) states are below 30 kHz. We obtain the hyperfine coefficients as A = -742.11(2) MHz and B = 1339.2(2) MHz, which represent a two orders-of-magnitude improvement in precision, and C = 0.54(2) MHz. From atomic structure calculations, we obtain the nuclear moments quadrupole Q = 2.46(12) b and octupole Omega = -34.4(21) b x mu(N). DOI: 10.1103/PhysRevA.87.012512

Stereospecific photodimerization of coumarins in crystalline inclusion complexes. Molecular and crystal structure of 1:2 complex of (S,S)-(-)-1,6-bis(o-chlorophenyl)-1,6-diphenyl-hexa-2,4-diyne-1,6-diol and coumarin

Proximity of molecules is a crucial factor in many solid- state photochemical processes.'S2 The biomolecular photodimerization reactions in the solid state depend on the relative geometry of reactant molecules in the crystal lattice with center-to-center distance of nearest neighbor double bonds of the order of ca. 4 A. This fact emanates from the incisive studies of Schmidt and Cohen.2 One of the two approaches to achieve this distance requirement is the so-called "Crystal-Engineering" of structures, which essentially involves the introduction of certain functional groups that display in-plane interstacking interactions (Cl...Cl, C-He-0, etc.) in the crystal The chloro group is by far the most successful in promoting the /3- packing m ~ d e ,th~o,u~gh recent studies have shown its limitations? Another approach involves the use of constrained media in which the reactants could hopefully be aligned.

Risk Minimizing Option Pricing for a Class of Exotic Options in a Markov-Modulated Market

We address risk minimizing option pricing in a regime switching market where the floating interest rate depends on a finite state Markov process. The growth rate and the volatility of the stock also depend on the Markov process. Using the minimal martingale measure, we show that the locally risk minimizing prices for certain exotic options satisfy a system of Black-Scholes partial differential equations with appropriate boundary conditions. We find the corresponding hedging strategies and the residual risk. We develop suitable numerical methods to compute option prices.

Ergodic control of partially observed Markov chains

The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear filter. Dynamic programming equations for the latter are derived, leading to existence and characterization of optimal stationary policies.

Reliable estimation via simulation

Let a and s denote the inter arrival times and service times in a GI/GI/1 queue. Let a (n), s (n) be the r.v.s, with distributions as the estimated distributions of a and s from iid samples of a and s of sizes n. Let w be a r.v. with the stationary distribution lr of the waiting times of the queue with input (a, s). We consider the problem of estimating E [w~], tx > 0 and 7r via simulations when (a (n), s (n)) are used as input. Conditions for the accuracy of the asymptotic estimate, continuity of the asymptotic variance and uniformity in the rate of convergence to the estimate are obtained. We also obtain rates of convergence for sample moments, the empirical process and the quantile process for the regenerative processes. Robust estimates are also obtained when an outlier contaminated sample of a and s is provided. In the process we obtain consistency, continuity and asymptotic normality of M-estimators for stationary sequences. Some robustness results for Markov processes are included.