Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

Impact of physical processes on the chlorophyll distribution in the Bay of Bengal

Satellite-derived chlorophyll a concentration (chl a) maps show three regions with high chl a in the Bay of Bengal. First among these is close to the coast, particularly off river mouths, with high values coinciding with the season of peak discharge; second is in the southwestern bay during the northeast monsoon, which is forced by local Ekman pumping; and the third is to the east of Sri Lanka in response to the summer monsoon winds. Chlorophyll-rich water from the mouths of rivers flows either along the coast or in an offshore direction, up to several hundred kilometers, depending on the prevailing ocean current pattern. The Irrawady River plume flows toward offshore and then turns northwestward during October–December, but it flows along the coast into the Andaman Sea for the rest of the year. From the Ganga-Brahmaputra river mouth, chl a–rich water flows directly southward into the open bay during spring but along the Indian coast during summer and winter. Along the Indian coast, the flow of chl a–rich water is determined by the East India Coastal Current (EICC). Whenever the EICC meanders off the Indian coast, it leads to an offshore outbreak of chl a–rich water from the coastal region into open ocean. The EICC as well as open ocean circulation in the bay is made up of several eddies, and these eddies show relatively higher chl a. Eddies near the coast, however, can often have higher chl a because of advection from the coastal region rather than generation within the eddy itself. The bay experiences several cyclones in a year, most of them occurring during October–November. These cyclones cause a drop in the sea surface temperature, a dip in the sea level, and a local increase in chl a. The impact of a cyclone is weaker in the northern part of the bay because of stronger stratification compared to the southern parts.

Metal-Insulator Transitions in Anion-Excess LaMnO3+? Controlled by the Mn4+ Content

LaMnO3+? samples with Mn4+ content up to 50% have been prepared by different methods. The structure of LaMnO3+? changes from orthorhombic to cubic (via rhombohedral) with increase in the Mn4+ content. LaMnO3+? samples containing greater than 20% Mn4+ are ferromagnetic and show resistivity maxima at a temperature Tt which is close to the ferromagnetic Curie temperature. The resistivity maximum is due to the occurrence of a metal-insulator transition. In samples heated to the same temperature, the value of Tt increases with % Mn4+. For a given sample, Tt increases with the temperature of heat treatment due to the increase in particle size. The onset of ferromagnetism in LaMnO3+? accompanied by an insulator-metal transition is similar to that found in La1-xCaxMnO3 and La1-xSrxCoO3.

Numerical and experimental study of the thermal behaviour of coining and upsetting processes

A numerical simulation technique has been employed to study the thermal behavior of hot-forging type forming processes. Experiments on the coining and upsetting of an aluminum billet were conducted to validate the numerical predictions. Typical forming conditions for both the coining and upsetting processes were then studied in detail. an electrical analogy scheme was used to determine the thermal contact resistance. This scheme can conviniently provide the interface characteristics for typical processing conditions, which normally involve high pressures and temperatures. A single forging cycle was first considered, and then a batch of twenty-five forgings was studied. Each forging cycle includes the billet mounting, ascent, loading, dwelling, unloading, descent, and billet removal stages. The temperature distribution in the first forging to be formed is found to be significantly different from that at the end of the batch. In industry, forging is essentially a batch operation. The influence of forming speed and reduction on thermal characteristics was investigated also. The variations that can occur in the process design by considering differences in temperature characteristics are discussed also.

On infinitesimal σ-fields generated by random processes

It is proved that the infinitesimal look-ahead and look-back σ-fields of a random process disagree at atmost countably many time instants.

The Exchange Processes in Zinc-Complexes of o-Vanillin Salicyloylhydrazone as studied by 2-Dimensional NMR

Zinc forms two types of complexes with o-vanillin salicyloylhydrazone. The H-1 and C-13 nmr studies suggest that it coordinates with azomethine nitrogen, the carbonyl oxygen and with one or both of the phenolic oxygens. The H-1-H-1 and H-1 decoupled C-13-C-13 two-dimensional nuclear Overhauser and exchange spectra show that there is an exchange between the two types of complexes.

The Zintl ion As-7](2-): an example of an electron-deficient As-x radical anion

K(2,2,2-crypt)](2)As-7]center dot THF, 1 (2,2,2-crypt = 4,7,13,16,21,24-hexaoxa-1,10-diazabicyclo8.8.8]hexacosane) is the first well characterized seven-atom radical anion of group 15. UV-Vis spectroscopy confirms the presence and electronic structure of As-7](2-). Cyclic voltammetry in DMF solution shows the As-7(3) /As-7(2) redox couple as a one-electron reversible process. Theoretical investigations explore the bonding and properties of compound 1.

Supercritical age-dependent branching Markov processes and their scaling limits

This paper studies the long-time behavior of the empirical distribution of age and normalized position of an age-dependent supercritical branching Markov process. The motion of each individual during its life is a random function of its age. It is shown that the empirical distribution of the age and the normalized position of all individuals alive at time t converges as t -> infinity to a deterministic product measure.

An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes

We develop in this article the first actor-critic reinforcement learning algorithm with function approximation for a problem of control under multiple inequality constraints. We consider the infinite horizon discounted cost framework in which both the objective and the constraint functions are suitable expected policy-dependent discounted sums of certain sample path functions. We apply the Lagrange multiplier method to handle the inequality constraints. Our algorithm makes use of multi-timescale stochastic approximation and incorporates a temporal difference (TD) critic and an actor that makes a gradient search in the space of policy parameters using efficient simultaneous perturbation stochastic approximation (SPSA) gradient estimates. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal policy. (C) 2010 Elsevier B.V. All rights reserved.

T. E. Harris and branching processes

T. E. Harris was a pioneer par excellence in many fields of probability theory. In this paper, we give a brief survey of the many fundamental contributions of Harris to the theory of branching processes, starting with his doctoral work at Princeton in the late forties and culminating in his fundamental book ``The Theory of Branching Processes,'' published in 1963.

Dimple and nose coalescences in phase-separation processes

Coalescence processes are investigated during phase separation in a density-matched liquid mixture (partially deuterated cyclohexane and methanol) under near-critical conditions. As a result of the interplay between capillary and lubrication forces, ''nose'' coalescence appears to be always associated with the slow growth of isolated droplets (exponent almost-equal-to 1/3), whereas ''dimple'' coalescence corresponds to the fast growth of interconnected droplets (exponent almost-equal-to 1). At each stage of growth, the distribution of droplets trapped during dimple coalescence is reminiscent of all of the previous coalescence events.

ChemInform Abstract: Synthesis of Anion-Deficient Layered Perovskites, ACa2Nb3-xMxO10-x (A: Rb, Cs; M: Al, Fe), Exhibiting Ion-Exchange and Intercalation. Evidence for the Formation of Layered Brownmillerites, ACa2Nb2AlO9 (A: Cs, H)

Anion-deficient layered perovskite oxides of the formula, ACa2Nb3-xMxO10-x (A = Rb, Cs; M = Al, Fe) for 0 < x less-than-or-equal-to 1.0, possessing tetragonal structures similar to the parent ACa2Nb3O10, have been synthesized. The interlayer A cations in these materials are readily exchanged with protons in aqueous HNO3 to give the protonated derivatives, HCa2Nb3-xMxO10-x; the latter are solid Bronsted acids intercalating a number of organic amines including aniline (pK(a) = 4.63). The distribution of acid sites in the interlayer region of HCa2Nb2MO9 inferred from n-alkylamine intercalation suggests that oxygen vacancies and Nb/M atoms are disordered in the ACa2Nb2MO9 samples prepared at 1100-1200-degrees-C. Annealing a disordered sample of CsCa2Nb2AlO9 for a long time at lower temperatures tends to order the Nb/Al atoms and oxygen vacancies to produce octahedral (NbO6/2)-tetrahedral (AlO4/2)-octahedral (NbO6/2) layer sequence reminiscent of the brownmillerite structure.

The exchange processes in o-vanillin salicyloylhydrazone as studied by two-dimensional NMR

Multidimensional NMR studies of o-vanillin salicyloylhydrazone at various temperatures have been undertaken in deuterated dimethyl sulfoxide and its cryoprotective mixture in H2O and D2O, acetone and acetonitrile. The molecule is found to exist in two conformers in dimethyl sulfoxide and the cryoprotective mixture. The exchange between the two conformers has been detected from the two-dimensional experiments - information which is not easily obtainable from the normal one-dimensional spectra. Results in the different solvents are interpreted in terms of solvent-solute interactions.

AM(1-x)Al(x)O(3-x) (A=Na or K; M=Nb or Ta): New anion-deficient Perovskite oxides exhibiting oxide ion conduction

Anion-deficient perovskite oxides of the formula AM(1-x)Al(x)O(3-x) (A = Na or K; M = Nb or Ta) have been prepared for 0 < x less than or equal to 0.5. Diffraction experiments reveal that while the potassium compounds adopt orthorhombic/cubic perovskite structures similar to the parent KNbO3/KTaO3, the sodium compound, NaNb0.5Al0.5O2.5, possesses a brownmillerite/LaSr-CuAlO5-like superstructure. Al-27 NMR spectra show an exclusive tetrahedral oxygen coordination for AI(III) in Na-Nb0.5Al0.5O2.5 (I) and both tetrahedral and octahedral coordination for Al(III) in KNb0.5Al0.5O2.5 (II). The results suggest a long-range and short-range ordering of oxide ion vacancies in I and II respectively. Electrical conductivity measurements show a significant oxide ion conduction for KNb1-xAlxO3-x, with the conductivity increasing with x up to x = 0.5. The differences in the Arrhenius plots of the ionic conductivity of I and II have been rationalized in terms of the long-range and short-range ordering of oxide ion vacancies in the anion-deficient perovskite oxides.