The actor-critic algorithm as multi-time-scale stochastic approximation

The actor-critic algorithm of Barto and others for simulation-based optimization of Markov decision processes is cast as a two time Scale stochastic approximation. Convergence analysis, approximation issues and an example are studied.

Polar and nonpolar solvation dynamics, ion diffusion, and vibrational relaxation: Role of biphasic solvent response in chemical dynamics

ChemInform is a weekly Abstracting Service, delivering concise information at a glance that was extracted from about 100 leading journals. To access a ChemInform Abstract of an article which was published elsewhere, please select a “Full Text” option. The original article is trackable via the “References” option.

Diffusion and Growth of the mu Phase (Ni6Nb7) in the Ni-Nb System

Incremental diffusion couple experiments are conducted to determine the average interdiffusion coefficient and the intrinsic diffusion coefficients of the species in the Ni6Nb7 (mu phase) in the Ni-Nb system. Further, the tracer diffusion coefficients are calculated from the knowledge of thermodynamic parameters. The diffusion rate of Ni is found to be higher than that of Nb, which indicates higher defect concentration in the Ni sublattice.

Providing quality of service via opportunistic splitting using stochastic approximation

We consider the problem of wireless channel allocation to multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. Each metric sequence can be Markov. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Opportunistic splitting for scheduling via stochastic approximation

We consider the problem of scheduling a wireless channel among multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Growth mechanism of phases by interdiffusion and atomic mechanism of diffusion in the molybdenum-silicon system

Tracer diffusion coefficients are calculated in different phases in the Mo-Si system from diffusion couple experiments using the data available on thermodynamic parameters. Following, possible atomic diffusion mechanism of the species is discussed based on the crystal structure. Unusual diffusion behaviour is found in the Mo(5)Si(3) and Mo(3)Si phases, which indicate the nature of defects present on different sublattices. Further the growth mechanism of the phases is discussed and morphological evolution during interdiffusion is explained. (C) 2011 Elsevier Ltd. All rights reserved.

Hydrolytic degradation and diffusion studies on a polyphosphate ester

A polyphosphate ester was synthesized by interfacial polycondensation of bisphenol-A and phenylphosphorodichloridate. Accelerated hydrolytic degradation studies were conducted under alkaline conditions. The effect of concentration of alkali and temperature were monitored. The rate of degradation reached a maximum value at 6 molar sodium hydroxide solution and then reduced. The activation energy for hydrolytic degradation was found to be 45 kcal/mol. Diffusion of alkali into the polymer pellet was studied at various concentrations of alkali and at various temperatures. The rate of diffusion also attained a maximum at 6M NaOH and the activation energy for diffusion process was found to be 12 kcal/mol. (C) 2002 John Wiley Sons, Inc.

Pseudodynamic systems approach based on a quadratic approximation of update equations for diffuse optical tomography

We explore a pseudodynamic form of the quadratic parameter update equation for diffuse optical tomographic reconstruction from noisy data. A few explicit and implicit strategies for obtaining the parameter updates via a semianalytical integration of the pseudodynamic equations are proposed. Despite the ill-posedness of the inverse problem associated with diffuse optical tomography, adoption of the quadratic update scheme combined with the pseudotime integration appears not only to yield higher convergence, but also a muted sensitivity to the regularization parameters, which include the pseudotime step size for integration. These observations are validated through reconstructions with both numerically generated and experimentally acquired data. (C) 2011 Optical Society of America

Role of diffusion creep in a superplastic zirconia-alumina composite

The phenomenon of superplasticity has been demonstrated in several zirconia-alumina composites. However, the rate controlling mechanism has not yet been unambiguously identified, due to the limited data available on these materials in comparison with 3 mol% yttria stabilized tetragonal zirconia (3YTZ). The limited data on a zirconia-20 wt% alumina (3Y20A) composite suggest that the mechanical characteristics are similar to those of 3YTZ. The present experimental study on 3Y20A reveals the occurrence of diffusion creep. The experimental results are examined critically in terms of dislocation activity and diffusion creep, and their relevance to superplastic deformation.

Unsteady free convection flow over an infinite vertical porous plate due to the combined effects of thermal and mass diffusion, magnetic field and Hall currents

The unsteady free convection flow over an infinite vertical porous plate, which moves with time-dependent velocity in an ambient fluid, has been studied. The effects of the magnetic field and Hall current are included in the analysis. The buoyancy forces arise due to both the thermal and mass diffusion. The partial differential equations governing the flow have been solved numerically using both the implicit finite difference scheme and the difference-differential method. For the steady case, analytical solutions have also been obtained. The effect of time variation on the skin friction, heat transfer and mass transfer is very significant. Suction increases the skin friction coefficient in the primary flow, and also the Nusselt and Sherwood numbers, but the skin friction coefficient in the secondary flow is reduced. The effect of injection is opposite to that of suction. The buoyancy force, injection and the Hall parameter induce an overshoot in the velocity profiles in the primary flow which changes the velocity gradient from a negative to a positive value, but the magnetic field and suction reduce this velocity overshoot.

Reactive Diffusion Between Vanadium and Silicon

Interdiffusion study is conducted in the V-Si system to determine integrated diffusion coefficients of the phases. Activation energy values are calculated from the experiments conducted at different temperatures. The average values are found to be 208, 240 and 141 kJ/mol, respectively, for the V(3)Si, V(5)Si(3) and VSi(2) phases. The low activation energy for the VSi(2) phase indicates very high concentration of defects or the significant contribution from the grain boundary diffusion. The error in calculation of diffusion parameters from a very thin phase layer in a multiphase diffusion couple is discussed. Further the data available in the literature in this system is compared and the problems in the indirect methodology followed previously to calculate the diffusion parameters are discussed.

Dynamics of pulled desorption with effects of excluded-volume interaction: The p-Laplacian diffusion equation and its exact solution

We analyze the dynamics of desorption of a polymer molecule which is pulled at one of its ends with force f, trying to desorb it. We assume a monomer to desorb when the pulling force on it exceeds a critical value f(c). We formulate an equation for the average position of the n-th monomer, which takes into account excluded-volume interaction through the blob-picture of a polymer under external constraints. The approach leads to a diffusion equation with a p-Laplacian for the propagation of the stretching along the chain. This has to be solved subject to a moving boundary condition. Interestingly, within this approach, the problem can be solved exactly in the trumpet, stem-flower and stem regimes. In the trumpet regime, we get tau = tau(0)n(d)(2), where n(d) is the number of monomers that have desorbed at the time tau. tau(0) is known only numerically, but for f close to f(c), it is found to be tau(0) similar to f(c)/(f(2/3) - f(c)(2/3)) If one used simple Rouse dynamics, this result would change to tau similar to f(c)n(d)(2)/(f - f(c)). In the other regimes too, one can find exact solution, and interestingly, in all regimes tau similar to n(d)(2). Copyright (C) EPLA, 2011

Theory of Gas Hydrates: Effect of the Approximation of Rigid Water Lattice

One of the assumptions of the van der Waals and Platteeuw theory for gas hydrates is that the host water lattice is rigid and not distorted by the presence of guest molecules. In this work, we study the effect of this approximation on the triple-point lines of the gas hydrates. We calculate the triple-point lines of methane and ethane hydrates via Monte Carlo molecular simulations and compare the simulation results with the predictions of van der Waals and Platteeuw theory. Our study shows that even if the exact intermolecular potential between the guest molecules and water is known, the dissociation temperatures predicted by the theory are significantly higher. This has serious implications to the modeling of gas hydrate thermodynamics, and in spite of the several impressive efforts made toward obtaining an accurate description of intermolecular interactions in gas hydrates, the theory will suffer from the problem of robustness if the issue of movement of water molecules is not adequately addressed.