A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Critical age-dependent branching Markov processes and their scaling limits

This paper studies:(i)the long-time behaviour of the empirical distribution of age and normalized position of an age-dependent critical branching Markov process conditioned on non-extinction;and (ii) the super-process limit of a sequence of age-dependent critical branching Brownian motions.

Mean square displacements of hydrogen interstitial and its neighbours in the intermetallic compound Fe0.5Ti0.5

A Green's function technique is used in the scattering matrix formalism to compute the mean square displacement of hydrogen and deuterium interstitials in the intermetallic compound Fe0.5Ti0.5 for low hydrogen/deuterium concentration. The mean square amplitudes of the metal atoms surrounding the interstitial are found to be smaller than those for the host crystal. This anomalous effect is due to the stiffening of the lattice by the dissolved hydrogen or deuterium at low concentration. This type of effect is experimentally observed in the case of NbHx at low hydrogen concentration.

Characterization of unitary processes with independent and stationary increments

This is a continuation of the earlier work (Publ. Res. Inst. Math. Sci. 45 (2009) 745-785) to characterize unitary stationary independent increment Gaussian processes. The earlier assumption of uniform continuity is replaced by weak continuity and with technical assumptions on the domain of the generator, unitary equivalence of the process to the solution of an appropriate Hudson-Parthasarathy equation is proved.

Reactivity Of Pph3 Toward Ruiiruiiicl(O2car)4 - Syntheses, Molecular-Structures, And Spectroscopic And Electrochemical Properties Of Ruiiruiii(Oh2)Cl(Mecn)(O2car)4(Pph3)2 And Ruii2(Oh2)(Mecn)2(O2car)4(Pph3)2

By the reaction of Ru2Cl(O2CAr)4 (1) and PPh3 in MeCN-H2O the diruthenium(II,III) and diruthenium(II) compounds of the type Ru2(OH2)Cl(MeCN)(O2CAr)4(PPh3)2 (2) and Ru2(OH2)(MeCN)2(O2CAr)4(PPh3)2 (3) were prepared and characterized by analytical, spectral, and electrochemical data (Ar is an aryl group, C6H4-p-X; X = H, OMe, Me, Cl, NO2). The molecular structure of Ru2(OH2)Cl(MeCN)(O2CC6H4-p-OMe)4(PPh3)2 was determined by X-ray crystallography. Crystal data are as follows: triclinic, P1BAR, a = 13.538 (5) angstrom, b = 15.650 (4) angstrom, c = 18.287 (7) angstrom, alpha = 101.39 (3)-degrees, beta = 105.99 (4)-degrees, gamma = 97.94 (3)-degrees, V = 3574 angstrom 3, Z = 2. The molecule is asymmetric, and the two ruthenium centers are clearly distinguishable. The Ru(III)-Ru(II), Ru(III)-(mu-OH2), and Ru(II)-(mu-OH2) distances and the Ru-(mu-OH2)-Ru angle in [{Ru(III)Cl(eta-1-O2CC6H4-p-OMe)(PPh3)}(mu-OH2)(mu-O2CC6H4-p-OMe)2{Ru(II)(MeCN)(eta-1-O2CC6H4-p-OMe)(PPh3)}] are 3.604 (1), 2.127 (8), and 2.141 (10) angstrom and 115.2 (5)-degrees, respectively. The compounds are paramagnetic and exhibit axial EPR spectra in the polycrystalline form. An intervalence transfer (IT) transition is observed in the range 900-960 nm in chloroform in these class II type trapped mixed-valence species 2. Compound 2 displays metal-centered one-electron reduction and oxidation processes near -0.4 and +0.6 V (vs SCE), respectively in CH2Cl2-TBAP. Compound 2 is unstable in solution phase and disproportionates to (mu-aquo)diruthenium(II) and (mu-oxo)diruthenium(III) complexes. The mechanistic aspects of the core conversion are discussed. The molecular structure of a diruthenium(II) compound, Ru2(OH2)(MeCN)2(O2CC6H4-p-NO2)4(PPh3)2.1.5CH2Cl2, was obtained by X-ray crystallography. The compound crystallizes in the space group P2(1)/c with a = 23.472 (6) angstrom, b = 14.303 (3) angstrom, c = 23.256 (7) angstrom, beta = 101.69 (2)-degrees, V = 7645 angstrom 3, and Z = 4. The Ru(II)-Ru(II) and two Ru(II)-(mu-OH2) distances and the Ru(II)-(mu-OH2)-Ru(II) angle in [{(PPh3)-(MeCN)(eta-1-O2CC6H4-p-NO2)Ru}2(mu-OH2)(mu-O2CC6H4-p-NO2)2] are 3.712 (1), 2.173 (9), and 2.162 (9) angstrom and 117.8 (4)-degrees, respectively. In both diruthenium(II,III) and diruthenium(II) compounds, each metal center has three facial ligands of varying pi-acidity and the aquo bridges are strongly hydrogen bonded with the eta-1-carboxylato facial ligands. The diruthenium(II) compounds are diamagnetic and exhibit characteristic H-1 NMR spectra in CDCl3. These compounds display two metal-centered one-electron oxidations near +0.3 and +1.0 V (vs SCE) in CH2Cl2-TBAP. The overall reaction between 1 and PPh3 in MeCN-H2O through the intermediacy of 2 is of the disproportionation type. The significant role of facial as well as bridging ligands in stabilizing the core structures is observed from electrochemical studies.

Experimental analysis of thermo-physical processes in acoustically levitated heated droplets

An experimental setup using radiative heating has been used to understand the thermo-physical phenomena and chemical transformations inside acoustically levitated cerium nitrate precursor droplets. In this transformation process, through infrared thermography and high speed imaging, events such as vaporization, precipitation and chemical reaction have been recorded at high temporal resolution, leading to nanoceria formation with a porous morphology. The cerium nitrate droplet undergoes phase and shape changes throughout the vaporization process. Four distinct stages were delineated during the entire vaporization process namely pure evaporation, evaporation with precipitate formation, chemical reaction with phase change and formation of final porous precipitate. The composition was examined using scanning and transmission electron microscopy that revealed nanostructures and confirmed highly porous morphology with trapped gas pockets. Transmission electron microscopy (TEM) and high speed imaging of the final precipitate revealed the presence of trapped gases in the form of bubbles. TEM also showed the presence of nanoceria crystalline structures at 70 degrees C. The current study also looked into the effect of different heating powers on the process. At higher power, each phase is sustained for smaller duration and higher maximum temperature. In addition, the porosity of the final precipitate increased with power. A non-dimensional time scale is proposed to correlate the effect of laser intensity and vaporization rate of the solvent (water). The effect of acoustic levitation was also studied. Due to acoustic streaming, the solute selectively gets transported to the bottom portion of the droplet due to strong circulation, providing it rigidity and allows it become bowl shaped. (C) 2010 Elsevier Ltd. All rights reserved.

Indian Ocean Dipole: Processes and impacts

Equatorial Indian Ocean is warmer in the east, has a deeper thermocline and mixed layer, and supports a more convective atmosphere than in the west. During certain years, the eastern Indian Ocean becomes unusually cold, anomalous winds blow from east to west along the equator and southeastward off the coast of Sumatra, thermocline and mixed layer lift up and the atmospheric convection gets suppressed. At the same time, western Indian Ocean becomes warmer and enhances atmospheric convection. This coupled ocean-atmospheric phenomenon in which convection, winds, sea surface temperature (SST) and thermocline take part actively is known as the Indian Ocean Dipole (IOD). Propagation of baroclinic Kelvin and Rossby waves excited by anomalous winds, play an important role in the development of SST anomalies associated with the IOD. Since mean thermocline in the Indian Ocean is deep compared to the Pacific, it was believed for a long time that the Indian Ocean is passive and merely responds to the atmospheric forcing. Discovery of the IOD and studies that followed demonstrate that the Indian Ocean can sustain its own intrinsic coupled ocean-atmosphere processes. About 50% percent of the IOD events in the past 100 years have co-occurred with El Nino Southern Oscillation (ENSO) and the other half independently. Coupled models have been able to reproduce IOD events and process experiments by such models – switching ENSO on and off – support the hypothesis based on observations that IOD events develop either in the presence or absence of ENSO. There is a general consensus among different coupled models as well as analysis of data that IOD events co-occurring during the ENSO are forced by a zonal shift in the descending branch of Walker cell over to the eastern Indian Ocean. Processes that initiate the IOD in the absence of ENSO are not clear, although several studies suggest that anomalies of Hadley circulation are the most probable forcing function. Impact of the IOD is felt in the vicinity of Indian Ocean as well as in remote regions. During IOD events, biological productivity of the eastern Indian Ocean increases and this in turn leads to death of corals over a large area.Moreover, the IOD affects rainfall over the maritime continent, Indian subcontinent, Australia and eastern Africa. The maritime continent and Australia suffer from deficit rainfall whereas India and east Africa receive excess. Despite the successful hindcast of the 2006 IOD by a coupled model, forecasting IOD events and their implications to rainfall variability remains a major challenge as understanding reasons behind an increase in frequency of IOD events in recent decades.

Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

Impact of physical processes on the chlorophyll distribution in the Bay of Bengal

Satellite-derived chlorophyll a concentration (chl a) maps show three regions with high chl a in the Bay of Bengal. First among these is close to the coast, particularly off river mouths, with high values coinciding with the season of peak discharge; second is in the southwestern bay during the northeast monsoon, which is forced by local Ekman pumping; and the third is to the east of Sri Lanka in response to the summer monsoon winds. Chlorophyll-rich water from the mouths of rivers flows either along the coast or in an offshore direction, up to several hundred kilometers, depending on the prevailing ocean current pattern. The Irrawady River plume flows toward offshore and then turns northwestward during October–December, but it flows along the coast into the Andaman Sea for the rest of the year. From the Ganga-Brahmaputra river mouth, chl a–rich water flows directly southward into the open bay during spring but along the Indian coast during summer and winter. Along the Indian coast, the flow of chl a–rich water is determined by the East India Coastal Current (EICC). Whenever the EICC meanders off the Indian coast, it leads to an offshore outbreak of chl a–rich water from the coastal region into open ocean. The EICC as well as open ocean circulation in the bay is made up of several eddies, and these eddies show relatively higher chl a. Eddies near the coast, however, can often have higher chl a because of advection from the coastal region rather than generation within the eddy itself. The bay experiences several cyclones in a year, most of them occurring during October–November. These cyclones cause a drop in the sea surface temperature, a dip in the sea level, and a local increase in chl a. The impact of a cyclone is weaker in the northern part of the bay because of stronger stratification compared to the southern parts.

Numerical and experimental study of the thermal behaviour of coining and upsetting processes

A numerical simulation technique has been employed to study the thermal behavior of hot-forging type forming processes. Experiments on the coining and upsetting of an aluminum billet were conducted to validate the numerical predictions. Typical forming conditions for both the coining and upsetting processes were then studied in detail. an electrical analogy scheme was used to determine the thermal contact resistance. This scheme can conviniently provide the interface characteristics for typical processing conditions, which normally involve high pressures and temperatures. A single forging cycle was first considered, and then a batch of twenty-five forgings was studied. Each forging cycle includes the billet mounting, ascent, loading, dwelling, unloading, descent, and billet removal stages. The temperature distribution in the first forging to be formed is found to be significantly different from that at the end of the batch. In industry, forging is essentially a batch operation. The influence of forming speed and reduction on thermal characteristics was investigated also. The variations that can occur in the process design by considering differences in temperature characteristics are discussed also.

On extremal solutions to stochastic control problems, II

The set of attainable laws of the joint state-control process of a controlled diffusion is analyzed from a convex analytic viewpoint. Various equivalence relations depending on one-dimensional marginals thereof are defined on this set and the corresponding equivalence classes are studied.

On infinitesimal σ-fields generated by random processes

It is proved that the infinitesimal look-ahead and look-back σ-fields of a random process disagree at atmost countably many time instants.