An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Stabilization of stochastic approximation by step size adaptation

A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same limiting behavior, while avoiding the difficulties associated with projection schemes. The proof technique requires only that the limiting o.d.e. descend a certain Lyapunov function outside an arbitrarily large bounded set. (C) 2012 Elsevier B.V. All rights reserved.

Diffusion ordered spectroscopy for resolution of double bonded cis, trans-isomers

NMR spectroscopic separation of double bonded cis- and trans-isomers, that have different molecular shapes but identical mass have been carried out using Diffusion Ordered Spectroscopy (DOSY). The mixtures of fumaric acid and maleic acid, that have similar hydrodynamic radii, have resolved been on the basis of their diffusion coefficients arising due to their different tendencies to associate with micelles or reverse micelles. Sodium dodecyl sulfate (SDS) and Dioctyl sulfosuccinate sodium salt (AOT) have been used as the media to mimic the chromatographic conditions, modify the average mobility and to achieve differential diffusion rates. The best separation of the components has been achieved by Dioctyl sulfosuccinate sodium salt (AOT) in D2O solution. (C) 2012 Elsevier B.V. All rights reserved.

Robust solver based on modified particle swarm optimization for improved solution of diffusion transport through containment facilities

Accurate estimation of mass transport parameters is necessary for overall design and evaluation processes of the waste disposal facilities. The mass transport parameters, such as effective diffusion coefficient, retardation factor and diffusion accessible porosity, are estimated from observed diffusion data by inverse analysis. Recently, particle swarm optimization (PSO) algorithm has been used to develop inverse model for estimating these parameters that alleviated existing limitations in the inverse analysis. However, PSO solver yields different solutions in successive runs because of the stochastic nature of the algorithm and also because of the presence of multiple optimum solutions. Thus the estimated mean solution from independent runs is significantly different from the best solution. In this paper, two variants of the PSO algorithms are proposed to improve the performance of the inverse analysis. The proposed algorithms use perturbation equation for the gbest particle to gain information around gbest region on the search space and catfish particles in alternative iterations to improve exploration capabilities. Performance comparison of developed solvers on synthetic test data for two different diffusion problems reveals that one of the proposed solvers, CPPSO, significantly improves overall performance with improved best, worst and mean fitness values. The developed solver is further used to estimate transport parameters from 12 sets of experimentally observed diffusion data obtained from three diffusion problems and compared with published values from the literature. The proposed solver is quick, simple and robust on different diffusion problems. (C) 2012 Elsevier Ltd. All rights reserved.

Confinement and viscoelastic effects on chain closure dynamics

Chemical reactions inside cells are typically subject to the effects both of the cell's confining surfaces and of the viscoelastic behavior of its contents. In this paper, we show how the outcome of one particular reaction of relevance to cellular biochemistry - the diffusion-limited cyclization of long chain polymers - is influenced by such confinement and crowding effects. More specifically, starting from the Rouse model of polymer dynamics, and invoking the Wilemski-Fixman approximation, we determine the scaling relationship between the mean closure time t(c) of a flexible chain (no excluded volume or hydrodynamic interactions) and the length N of its contour under the following separate conditions: (a) confinement of the chain to a sphere of radius d and (b) modulation of its dynamics by colored Gaussian noise. Among other results, we find that in case (a) when d is much smaller than the size of the chain, t(c) similar to Nd-2, and that in case (b), t(c) similar to N-2/(2 (2H)), H being a number between 1/2 and 1 that characterizes the decay of the noise correlations. H is not known a priori, but values of about 0.7 have been used in the successful characterization of protein conformational dynamics. At this value of H (selected for purposes of illustration), t(c) similar to N-3.4, the high scaling exponent reflecting the slow relaxation of the chain in a viscoelastic medium. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4729041]

Diffusion mechanism in XSi2 and X5Si3 (X= Nb, Mo, V) phases

In view of the importance of the suicides in the high temperature applications, the diffusion behaviour is compared in different systems for two types of silicides, XSi2 and X5Si3 (X=Nb, Mo, V). Atomic mechanism of diffusion and defects present in the structure are discussed. In both the phases, Si has faster diffusion rate than the metal species. This is expected from the nearest neighbour (NN) bonds present in the XSi2 phase but rather unusual in the X5Si3 phase. Relative mobilities of the species calculated indicate the presence of high concentration of Si antisites. Moreover, the concentration of the defects is different in different systems to find different diffusion rates.

Diffusion mechanism in the mu phase in Nb-X (X = Ni, Co, Fe) systems

Nb is one of the common refractory elements added in Ni, Co and Fe based superalloys. This lead to the formation of brittle topological close packed (tcp) mu phase, which is deleterious to the structure. It mainly grows by interdiffusion and in the present article, the interdiffusion process in different Nb-X (X=Ni, Co, Fe) systems is discussed. The activation energy for interdiffusion is lower in the Co-Nb system (173 kJ/mol) than Fe-Nb system (233 kJ/mol), which is again lower than the value found in the Ni-Nb system (319.7 kJ/mol). The mole fraction of Nb in this phase is less than Fe or Co at stoichiometric compositions in the Nb-Fe (that is Fe7Nb6) and Nb-Co (that is Co7Nb6) systems. On the other hand, the mole fraction of Nb is higher than Ni in the same phase (Ni6Nb2) in Ni-Nb system. However, in all the phases, Nb has lower diffusion rate. Possible diffusion mechanism in this phase is discussed with respect to the crystal structure.

A versatile resolving agent for diffusion edited separation of enantiomers, complex mixtures and constitutional isomers

The study is the first report of the utilization of a crown ether as a new and versatile resolving agent for the diffusion edited separation of enantiomers, complex mixtures and constitutional isomers. As a consequence of different binding affinities of enantiomers of a chiral molecule and individual components of the complex mixtures with the crown ether, the molecules diffuse at different rates. The enhanced separation achieved due to matrix assisted diffusion permitted their separation in the diffusion dimension. The generality and wide utility of the new resolving agent and the methodology are demonstrated on diverse examples, such as an organic chiral molecule, constitutional isomers and complex mixture of molecules possessing different functional groups that possess nearly identical molecular weights.

Use of Fick's law and Maxwell–Stefan equations in computation of multicomponent diffusion

This article does not have an abstract.

Diffusion mechanism in the gold-copper system

Interdiffusion study is conducted in the Au-Cu system, which has complete solid solution in the higher temperature range and ordered phases in the lower temperature range. First experiments are conducted at higher temperatures, where atoms can diffuse randomly. Higher values of interdiffusion coefficients are found in the range of 40-50 at.% Cu. This trend is explained with the help of thermodynamic factor and possible concentration of vacancies. Following an experiment is conducted at 623 K (350 degrees C), where the ordered phases are grown. The interdiffusion coefficients at this temperature are compared after extrapolating the data calculated at higher temperatures.

Diffusion and growth mechanism of phases in the Pd-Sn system

Growth mechanism of phases and atomic mechanism of diffusion are discussed in the Pd-Sn system. The Kirkendall marker plane location indicates that the PdSn4 phase grows because of diffusion of Sn. Atomic arrangement in the crystal indicates that Sn can diffuse through its own sublattice but Pd cannot diffuse unless antisites are present. The negligible diffusion of Pd indicates the absence of Pd antisites. The activation energy value indicates that the contribution from grain boundary diffusion cannot be neglected although experiments were conducted in the homologous temperature range of 0.7-0.79.

Directional Diffusion Regulator (DDR) for some numerical solvers of hyperbolic conservation laws

A computational tool called ``Directional Diffusion Regulator (DDR)'' is proposed to bring forth real multidimensional physics into the upwind discretization in some numerical schemes of hyperbolic conservation laws. The direction based regulator when used with dimension splitting solvers, is set to moderate the excess multidimensional diffusion and hence cause genuine multidimensional upwinding like effect. The basic idea of this regulator driven method is to retain a full upwind scheme across local discontinuities, with the upwind bias decreasing smoothly to a minimum in the farthest direction. The discontinuous solutions are quantified as gradients and the regulator parameter across a typical finite volume interface or a finite difference interpolation point is formulated based on fractional local maximum gradient in any of the weak solution flow variables (say density, pressure, temperature, Mach number or even wave velocity etc.). DDR is applied to both the non-convective as well as whole unsplit dissipative flux terms of some numerical schemes, mainly of Local Lax-Friedrichs, to solve some benchmark problems describing inviscid compressible flow, shallow water dynamics and magneto-hydrodynamics. The first order solutions consistently improved depending on the extent of grid non-alignment to discontinuities, with the major influence due to regulation of non-convective diffusion. The application is also experimented on schemes such as Roe, Jameson-Schmidt-Turkel and some second order accurate methods. The consistent improvement in accuracy either at moderate or marked levels, for a variety of problems and with increasing grid size, reasonably indicate a scope for DDR as a regular tool to impart genuine multidimensional upwinding effect in a simpler framework. (C) 2012 Elsevier Inc. All rights reserved.

Cyclodextrin and its complexation for resolution of isomers using diffusion ordered spectroscopy

Diffusion ordered spectroscopy (DOSY) generally fails to separate the peaks pertaining to isomeric species possessing identical molecular weights and similar hydrodynamic radii. The present study demonstrates the resolution of isomers using alpha/beta-cyclodextrin as a co-solute by Matrix Assisted Diffusion Ordered Spectroscopy. The resolution of isomers has been achieved by measuring the significant differences in the diffusion rates between the positional isomers of aminobenzoic acids, benzenedicarboxylic acids and between the cis, trans isomers, fumaric acid and maleic acid. (C) 2012 Elsevier B.V. All rights reserved.

Optical properties change with the addition and diffusion of Bi to As2S3 in the Bi/As2S3 bilayer thin film

The role of Bi layer (thickness similar to 7 nm) on As2S3 film was extensively studied for different optical applications in which Bi (top layer) as active and diffusing layer and As2S3 as barrier (matrix) layer. Bilayer thin films of Bi/As2S3 were prepared from Bi and As2S3 by thermal evaporation technique under high vacuum. The decrease of optical band gap with the addition of Bi to As2S3 has been explained on the basis of density of states and the increase in disorder in the system. It was found that the efficient changes of optical parameters (transmission, optical band gap, refraction) could be realized due to the photo induced diffusion activated by the focused 532 nm laser irradiation and formation of different bonds. The diffusion of Bi into As2S3 matrix increases the optical band gap producing photo bleaching effect. The changes were characterised by different experimental techniques. (C) 2012 Elsevier B.V. All rights reserved.

A constant factor approximation algorithm for boxicity of circular arc graphs

Boxicity of a graph G(V, E) is the minimum integer k such that G can be represented as the intersection graph of k-dimensional axis parallel boxes in Rk. Equivalently, it is the minimum number of interval graphs on the vertex set V such that the intersection of their edge sets is E. It is known that boxicity cannot be approximated even for graph classes like bipartite, co-bipartite and split graphs below O(n0.5-ε)-factor, for any ε > 0 in polynomial time unless NP = ZPP. Till date, there is no well known graph class of unbounded boxicity for which even an nε-factor approximation algorithm for computing boxicity is known, for any ε < 1. In this paper, we study the boxicity problem on Circular Arc graphs - intersection graphs of arcs of a circle. We give a (2+ 1/k)-factor polynomial time approximation algorithm for computing the boxicity of any circular arc graph along with a corresponding box representation, where k ≥ 1 is its boxicity. For Normal Circular Arc(NCA) graphs, with an NCA model given, this can be improved to an additive 2-factor approximation algorithm. The time complexity of the algorithms to approximately compute the boxicity is O(mn+n2) in both these cases and in O(mn+kn2) which is at most O(n3) time we also get their corresponding box representations, where n is the number of vertices of the graph and m is its number of edges. The additive 2-factor algorithm directly works for any Proper Circular Arc graph, since computing an NCA model for it can be done in polynomial time.