Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

Impact of physical processes on the chlorophyll distribution in the Bay of Bengal

Satellite-derived chlorophyll a concentration (chl a) maps show three regions with high chl a in the Bay of Bengal. First among these is close to the coast, particularly off river mouths, with high values coinciding with the season of peak discharge; second is in the southwestern bay during the northeast monsoon, which is forced by local Ekman pumping; and the third is to the east of Sri Lanka in response to the summer monsoon winds. Chlorophyll-rich water from the mouths of rivers flows either along the coast or in an offshore direction, up to several hundred kilometers, depending on the prevailing ocean current pattern. The Irrawady River plume flows toward offshore and then turns northwestward during October–December, but it flows along the coast into the Andaman Sea for the rest of the year. From the Ganga-Brahmaputra river mouth, chl a–rich water flows directly southward into the open bay during spring but along the Indian coast during summer and winter. Along the Indian coast, the flow of chl a–rich water is determined by the East India Coastal Current (EICC). Whenever the EICC meanders off the Indian coast, it leads to an offshore outbreak of chl a–rich water from the coastal region into open ocean. The EICC as well as open ocean circulation in the bay is made up of several eddies, and these eddies show relatively higher chl a. Eddies near the coast, however, can often have higher chl a because of advection from the coastal region rather than generation within the eddy itself. The bay experiences several cyclones in a year, most of them occurring during October–November. These cyclones cause a drop in the sea surface temperature, a dip in the sea level, and a local increase in chl a. The impact of a cyclone is weaker in the northern part of the bay because of stronger stratification compared to the southern parts.

Numerical and experimental study of the thermal behaviour of coining and upsetting processes

A numerical simulation technique has been employed to study the thermal behavior of hot-forging type forming processes. Experiments on the coining and upsetting of an aluminum billet were conducted to validate the numerical predictions. Typical forming conditions for both the coining and upsetting processes were then studied in detail. an electrical analogy scheme was used to determine the thermal contact resistance. This scheme can conviniently provide the interface characteristics for typical processing conditions, which normally involve high pressures and temperatures. A single forging cycle was first considered, and then a batch of twenty-five forgings was studied. Each forging cycle includes the billet mounting, ascent, loading, dwelling, unloading, descent, and billet removal stages. The temperature distribution in the first forging to be formed is found to be significantly different from that at the end of the batch. In industry, forging is essentially a batch operation. The influence of forming speed and reduction on thermal characteristics was investigated also. The variations that can occur in the process design by considering differences in temperature characteristics are discussed also.

On infinitesimal σ-fields generated by random processes

It is proved that the infinitesimal look-ahead and look-back σ-fields of a random process disagree at atmost countably many time instants.

The Exchange Processes in Zinc-Complexes of o-Vanillin Salicyloylhydrazone as studied by 2-Dimensional NMR

Zinc forms two types of complexes with o-vanillin salicyloylhydrazone. The H-1 and C-13 nmr studies suggest that it coordinates with azomethine nitrogen, the carbonyl oxygen and with one or both of the phenolic oxygens. The H-1-H-1 and H-1 decoupled C-13-C-13 two-dimensional nuclear Overhauser and exchange spectra show that there is an exchange between the two types of complexes.

Supercritical age-dependent branching Markov processes and their scaling limits

This paper studies the long-time behavior of the empirical distribution of age and normalized position of an age-dependent supercritical branching Markov process. The motion of each individual during its life is a random function of its age. It is shown that the empirical distribution of the age and the normalized position of all individuals alive at time t converges as t -> infinity to a deterministic product measure.

An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes

We develop in this article the first actor-critic reinforcement learning algorithm with function approximation for a problem of control under multiple inequality constraints. We consider the infinite horizon discounted cost framework in which both the objective and the constraint functions are suitable expected policy-dependent discounted sums of certain sample path functions. We apply the Lagrange multiplier method to handle the inequality constraints. Our algorithm makes use of multi-timescale stochastic approximation and incorporates a temporal difference (TD) critic and an actor that makes a gradient search in the space of policy parameters using efficient simultaneous perturbation stochastic approximation (SPSA) gradient estimates. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal policy. (C) 2010 Elsevier B.V. All rights reserved.

T. E. Harris and branching processes

T. E. Harris was a pioneer par excellence in many fields of probability theory. In this paper, we give a brief survey of the many fundamental contributions of Harris to the theory of branching processes, starting with his doctoral work at Princeton in the late forties and culminating in his fundamental book ``The Theory of Branching Processes,'' published in 1963.

Dimple and nose coalescences in phase-separation processes

Coalescence processes are investigated during phase separation in a density-matched liquid mixture (partially deuterated cyclohexane and methanol) under near-critical conditions. As a result of the interplay between capillary and lubrication forces, ''nose'' coalescence appears to be always associated with the slow growth of isolated droplets (exponent almost-equal-to 1/3), whereas ''dimple'' coalescence corresponds to the fast growth of interconnected droplets (exponent almost-equal-to 1). At each stage of growth, the distribution of droplets trapped during dimple coalescence is reminiscent of all of the previous coalescence events.

The exchange processes in o-vanillin salicyloylhydrazone as studied by two-dimensional NMR

Multidimensional NMR studies of o-vanillin salicyloylhydrazone at various temperatures have been undertaken in deuterated dimethyl sulfoxide and its cryoprotective mixture in H2O and D2O, acetone and acetonitrile. The molecule is found to exist in two conformers in dimethyl sulfoxide and the cryoprotective mixture. The exchange between the two conformers has been detected from the two-dimensional experiments - information which is not easily obtainable from the normal one-dimensional spectra. Results in the different solvents are interpreted in terms of solvent-solute interactions.

Theory of electron transfer processes via chemisorbed intermediates: Part II. Current-potential characteristics

Current-potential characteristics are obtained numerically for a lone-adsorbate-mediated anodic charge transfer at the electrode-solution interface. An increase in the overpotential leads to the appearance of maxima in the anodic current-potential plots instead of the extended activationless region (i.e. a saturation current at large positive overpotentials) predicted by the direct heterogeneous outer-sphere anodic charge transfer process. A detailed analysis of the dependence of current-potential profiles and other kinetic parameters on various system parameters is also presented.

Thermodynamics of flow and vaporization processes in long-chain liquids

The flow and vaporization behaviors of long-chain esters of varying molecular weights (300-900) ana branching (linear, Y-shaped, and +-shaped molecules) have been studied. The flow behavior is found to depend on the structure as well as the molecular weight. Below a molecular weight of 600, the molecules flow wholly but above this, segmental motion occurs, and the flow becomes independent of the molecular weight which is explained from the blob model. The blob concept demonstrates that the hole of a size of about 11 angstrom is needed for the flow to occur and it is much less than the size of the molecule. The blob size is observed to slightly decrease along the series linear and Y- and +-branched esters. The heat of vaporization is found to be independent of the molecular structure since the molecules acquire a coiled spherical shape during vaporization and hence depends only on the molecular weight. A significant structural effect is observed for the esters on their glass transition temperature (T(g)). The T(g) vs molecular weight plot displays contrasting trend for linear and +-branched esters, with Y esters showing an intermediate behavior. It is explained from their molecular packing and entanglement as visualized by the blob model.

Low electric field, easily reversible electrical set and reset processes in a Ge15Te83Si2 glass for phase change memory applications

We report here an easily reversible set-reset process in a new Ge15Te83Si2 glass that could be a promising candidate for phase change random access memory applications. The I-V characteristics of the studied sample show a comparatively low threshold electric field (E-th) of 7.3 kV/cm. Distinct differences in the type of switching behavior are achieved by means of controlling the on state current. It enables the observation of a threshold type for less than 0.7 mA beyond memory type (set) switching. The set and reset processes have been achieved with a similar magnitude of 1 mA, and with a triangular current pulse for the set process and a short duration rectangular pulse of 10 msec width for the reset operation. Further, a self-resetting effect is seen in this material upon excitation with a saw-tooth/square pulse, and their response of leading and trailing edges are discussed. About 6.5 x 10(4) set-reset cycles have been undertaken without any damage to the device. (C) 2011 American Institute of Physics. doi: 10.1063/1.3574659]

Some Basic Aspects of Reaction Engineering of Precipitation Processes

Analysis of precipitation reactions is extremely important in the technology of production of fine particles from the liquid phase. The control of composition and particle size in precipitation processes requires careful analysis of the several reactions that comprise the precipitation system. Since precipitation systems involve several, rapid ionic dissociation reactions among other slower ones, the faster reactions may be assumed to be nearly at equilibrium. However, the elimination of species, and the consequent reduction of the system of equations, is an aspect of analysis fraught with the possibility of subtle errors related to the violation of conservation principles. This paper shows how such errors may be avoided systematically by relying on the methods of linear algebra. Applications are demonstrated by analyzing the reactions leading to the precipitation of calcium carbonate in a stirred tank reactor as well as in a single emulsion drop. Sample calculations show that supersaturation dynamics can assume forms that can lead to subsequent dissolution of particles that have once been precipitated.