Incremental natural-gradient actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic rein- forcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their com- patibility with function approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further re- duce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal differ- ence learning in the actor and by incorporating natural gradients, and they extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

Application of composition gradient solid electrolytes in sensors and thermodynamic measurements

New compos~tiong radient solid electrolytes are developed which have application in high temperature solid state galvanic sensors and provide a new tool for thermodynamic measurements. The electrolyte consists oi a solid solution between two ionic conductors with a common mobile ion and spatial variation in composition of otber coxup nents. Incorporation of the composite electrolyte in sensors permits the use oi dissimilar gas electrodes. It is demonsuated, both experimentall y and theoretically, that the composition gradient of the relativeiy immobile species does not give rise to a diffusion potential.The emi of a cell is determined by the activity of the mobile species at the two eiectrodes. The thermodynamic properties of solid solutions can be measured using the gradient solid electrolyte. The experimental stuay is based on model systems A?(COj)x(S04)l-x (A=Na,K),where S \.aria across the electrolyte. The functionally gradient solid electrolytes used for activity measurements consist of pure carbonate at one ena and the solid solution under stuav at the other. The identical vaiues of activity, obtained h m t hree different modes of operation of the ceil. indicate unit transport number for the ddi metal ion in the graciient electrolyte. Tlle activities in the solid solutions exhibit moderate positive deviations from Raoult 's law.

Temperature gradient induced phase transitions and morphological changes in diamond thin film

The diamond films were deposited onto a wurtzite gallium nitride (GaN) thin film substrate using hot-filament chemical vapor deposition (HFCVD). During the film deposition a lateral temperature gradient was imposed across the substrate by inclining the substrate. As grown films predominantly showed the hexagonal phase, when no inclination was applied to the substrate. Tilting the substrate with respect to the heating filament by 6 degrees imposed a lateral temperature gradient across the substrate, which induced the formation of a cubic diamond phase. Diamond grains were predominantly oriented in the (100) direction. However, a further increase in the substrate tilt angle to 12 degrees, resulted in grains oriented in the (111) direction. The growth rate and hence the morphology of diamond grains varied along the inclined substrate. The present study focuses on the measurements of dominant phase formation and crystal orientation with varying substrate inclination using orientation-imaging microscopy (OIM). This technique enables direct examination of individual diamond grains and their crystallographic orientation. (C) 2012 Elsevier B.V. All rights reserved.

Quantum corrections to screening at strong coupling

We compute a certain class of corrections to (specific) screening lengths in strongly coupled non-abelian plasmas using the AdS/CFT correspondence. In this holographic framework, these corrections arise from various higher curvature interactions modifying the leading Einstein gravity action. The changes in the screening lengths are perturbative in inverse powers of the `t Hooft coupling or of the number of colors, as can be made precise in the context where the dual gauge theory is superconformal. We also compare the results of these holographic calculations to lattice results for the analogous screening lengths in QCD. In particular, we apply these results within the program of making quantitative comparisons between the strongly coupled quark-gluon plasma and holographic descriptions of conformal field theory. (c) 2012 Elsevier B.V. All rights reserved.

Effect of compressibility on the velocity gradient tensor at free shear flow boundaries

Analyses of the invariants of the velocity gradient ten- sor were performed on flow fields obtained by DNS of compressible plane mixing layers at convective Mach num- bers Mc=0:15 and 1.1. Joint pdfs of the 2nd and 3rd invariants were examined at turbulent/nonturbulent (T/NT) boundaries—defined as surfaces where the local vorticity first exceeds a threshold fraction of the maximum of the mean vorticity. By increasing the threshold from very small lev-els, the boundary points were moved closer into the turbulent region, and the effects on the pdfs of the invariants were ob-served. Generally, T/NT boundaries are in sheet-like regions at both Mach numbers. At the higher Mach number a distinct lobe appears in the joint pdf isolines which has not been ob-served/reported before. A connection to the delayed entrain-ment and reduced growth rate of the higher Mach number flow is proposed.

Bilateral smoothing of gradient vector field and application to image segmentation

Medical image segmentation finds application in computer-aided diagnosis, computer-guided surgery, measuring tissue volumes, locating tumors, and pathologies. One approach to segmentation is to use active contours or snakes. Active contours start from an initialization (often manually specified) and are guided by image-dependent forces to the object boundary. Snakes may also be guided by gradient vector fields associated with an image. The first main result in this direction is that of Xu and Prince, who proposed the notion of gradient vector flow (GVF), which is computed iteratively. We propose a new formalism to compute the vector flow based on the notion of bilateral filtering of the gradient field associated with the edge map - we refer to it as the bilateral vector flow (BVF). The range kernel definition that we employ is different from the one employed in the standard Gaussian bilateral filter. The advantage of the BVF formalism is that smooth gradient vector flow fields with enhanced edge information can be computed noniteratively. The quality of image segmentation turned out to be on par with that obtained using the GVF and in some cases better than the GVF.

Characterization of groundwater chemistry under the influence of lithologic and anthropogenic factors along a climatic gradient in Upper Cauvery basin, South India

Hydrogeological and climatic effect on chemical behavior of groundwater along a climatic gradient is studied along a river basin. `Semi-arid' (500-800 mm of mean annual rainfall), `sub-humid' (800-1,200 mm/year) and `humid' (1,200-1,500 mm/year) are the climatic zones chosen along the granito-gneissic plains of Kabini basin in South India for the present analysis. Data on groundwater chemistry is initially checked for its quality using NICB ratio (<+/- 5 %), EC versus TZ+ (similar to 0.85 correlation), EC versus TDS and EC versus TH analysis. Groundwater in the three climatic zones is `hard' to `very hard' in terms of Ca-Mg hardness. Polluted wells are identified (> 40 % of pollution) and eliminated for the characterization. Piper's diagram with mean concentrations indicates the evolution of CaNaHCO3 (semi-arid) from CaHCO3 (humid zone) along the climatic gradient. Carbonates dominate other anions and strong acids exceeded weak acids in the region. Mule Hole SEW, an experimental watershed in sub-humid zone, is characterized initially using hydrogeochemistry and is observed to be a replica of entire sub-humid zone (with 25 wells). Extension of the studies for the entire basin (120 wells) showed a chemical gradient along the climatic gradient with sub-humid zone bridging semi-arid and humid zones. Ca/Na molar ratio varies by more than 100 times from semi-arid to humid zones. Semi-arid zone is more silicaceous than sub-humid while humid zone is more carbonaceous (Ca/Cl similar to 14). Along the climatic gradient, groundwater is undersaturated (humid), saturated (sub-humid) and slightly supersaturated (semi-arid) with calcite and dolomite. Concentration-depth profiles are in support of the geological stratification i.e., not approximate to 18 m of saprolite and similar to 25 m of fracture rock with parent gneiss beneath. All the wells are classified into four groups based on groundwater fluctuations and further into `deep' and `shallow' based on the depth to groundwater. Higher the fluctuations, larger is its impact on groundwater chemistry. Actual seasonal patterns are identified using `recharge-discharge' concept based on rainfall intensity instead of traditional monsoon-non-monsoon concept. Non-pumped wells have low Na/Cl and Ca/Cl ratios in recharge period than in discharge period (Dilution). Few other wells, which are subjected to pumping, still exhibit dilution chemistry though water level fluctuations are high due to annual recharge. Other wells which do not receive sufficient rainfall and are constantly pumped showed high concentrations in recharge period rather than in discharge period (Anti-dilution). In summary, recharge-discharge concept demarcates the pumped wells from natural deep wells thus, characterizing the basin.

Efficient gradient-free simplex method for estimation of optical properties in image-guided diffuse optical tomography

Typical image-guided diffuse optical tomographic image reconstruction procedures involve reduction of the number of optical parameters to be reconstructed equal to the number of distinct regions identified in the structural information provided by the traditional imaging modality. This makes the image reconstruction problem less ill-posed compared to traditional underdetermined cases. Still, the methods that are deployed in this case are same as those used for traditional diffuse optical image reconstruction, which involves a regularization term as well as computation of the Jacobian. A gradient-free Nelder-Mead simplex method is proposed here to perform the image reconstruction procedure and is shown to provide solutions that closely match ones obtained using established methods, even in highly noisy data. The proposed method also has the distinct advantage of being more efficient owing to being regularization free, involving only repeated forward calculations. (C) 2013 Society of Photo-Optical Instrumentation Engineers (SPIE)

Differentially private stochastic gradient descent algorithm for multiparty classification

We consider the problem of developing privacy-preserving machine learning algorithms in a dis-tributed multiparty setting. Here different parties own different parts of a data set, and the goal is to learn a classifier from the entire data set with-out any party revealing any information about the individual data points it owns. Pathak et al [7]recently proposed a solution to this problem in which each party learns a local classifier from its own data, and a third party then aggregates these classifiers in a privacy-preserving manner using a cryptographic scheme. The generaliza-tion performance of their algorithm is sensitive to the number of parties and the relative frac-tions of data owned by the different parties. In this paper, we describe a new differentially pri-vate algorithm for the multiparty setting that uses a stochastic gradient descent based procedure to directly optimize the overall multiparty ob-jective rather than combining classifiers learned from optimizing local objectives. The algorithm achieves a slightly weaker form of differential privacy than that of [7], but provides improved generalization guarantees that do not depend on the number of parties or the relative sizes of the individual data sets. Experimental results corrob-orate our theoretical findings.

3D flat holography: entropy and logarithmic corrections

We compute the leading corrections to the Bekenstein-Hawking entropy of the Flat Space Cosmological (FSC) solutions in 3D flat spacetimes, which are the flat analogues of the BTZ black holes in AdS(3). The analysis is done by a computation of density of states in the dual 2D Galilean Conformal Field Theory and the answer obtained by this matches with the limiting value of the expected result for the BTZ inner horizon entropy as well as what is expected for a generic thermodynamic system. Along the way, we also develop other aspects of holography of 3D flat spacetimes.

SPATIO-TEMPORAL DYNAMICS ALONG THE TERRAIN GRADIENT OF DIVERSE LANDSCAPE

Land use (LU) land cover (LC) information at a temporal scale illustrates the physical coverage of the Earth's terrestrial surface according to its use and provides the intricate information for effective planning and management activities. LULC changes are stated as local and location specific, collectively they act as drivers of global environmental changes. Understanding and predicting the impact of LULC change processes requires long term historical restorations and projecting into the future of land cover changes at regional to global scales. The present study aims at quantifying spatio temporal landscape dynamics along the gradient of varying terrains presented in the landscape by multi-data approach (MDA). MDA incorporates multi temporal satellite imagery with demographic data and other additional relevant data sets. The gradient covers three different types of topographic features, planes; hilly terrain and coastal region to account the significant role of elevation in land cover change. The seasonality is another aspect to be considered in the vegetation dominated landscapes; variations are accounted using multi seasonal data. Spatial patterns of the various patches are identified and analysed using landscape metrics to understand the forest fragmentation. The prediction of likely changes in 2020 through scenario analysis has been done to account for the changes, considering the present growth rates and due to the proposed developmental projects. This work summarizes recent estimates on changes in cropland, agricultural intensification, deforestation, pasture expansion, and urbanization as the causal factors for LULC change.

The generalized Onsager model for a binary gas mixture

The Onsager model for the secondary flow field in a high-speed rotating cylinder is extended to incorporate the difference in mass of the two species in a binary gas mixture. The base flow is an isothermal solid-body rotation in which there is a balance between the radial pressure gradient and the centrifugal force density for each species. Explicit expressions for the radial variation of the pressure, mass/mole fractions, and from these the radial variation of the viscosity, thermal conductivity and diffusion coefficient, are derived, and these are used in the computation of the secondary flow. For the secondary flow, the mass, momentum and energy equations in axisymmetric coordinates are expanded in an asymptotic series in a parameter epsilon = (Delta m/m(av)), where Delta m is the difference in the molecular masses of the two species, and the average molecular mass m(av) is defined as m(av) = (rho(w1)m(1) + rho(w2)m(2))/rho(w), where rho(w1) and rho(w2) are the mass densities of the two species at the wall, and rho(w) = rho(w1) + rho(w2). The equation for the master potential and the boundary conditions are derived correct to O(epsilon(2)). The leading-order equation for the master potential contains a self-adjoint sixth-order operator in the radial direction, which is different from the generalized Onsager model (Pradhan & Kumaran, J. Fluid Mech., vol. 686, 2011, pp. 109-159), since the species mass difference is included in the computation of the density, viscosity and thermal conductivity in the base state. This is solved, subject to boundary conditions, to obtain the leading approximation for the secondary flow, followed by a solution of the diffusion equation for the leading correction to the species mole fractions. The O(epsilon) and O(epsilon(2)) equations contain inhomogeneous terms that depend on the lower-order solutions, and these are solved in a hierarchical manner to obtain the O(epsilon) and O(epsilon(2)) corrections to the master potential. A similar hierarchical procedure is used for the Carrier-Maslen model for the end-cap secondary flow. The results of the Onsager hierarchy, up to O(epsilon(2)), are compared with the results of direct simulation Monte Carlo simulations for a binary hard-sphere gas mixture for secondary flow due to a wall temperature gradient, inflow/outflow of gas along the axis, as well as mass and momentum sources in the flow. There is excellent agreement between the solutions for the secondary flow correct to O(epsilon(2)) and the simulations, to within 15 %, even at a Reynolds number as low as 100, and length/diameter ratio as low as 2, for a low stratification parameter A of 0.707, and when the secondary flow velocity is as high as 0.2 times the maximum base flow velocity, and the ratio 2 Delta m/(m(1) + m(2)) is as high as 0.5. Here, the Reynolds number Re = rho(w)Omega R-2/mu, the stratification parameter A = root m Omega R-2(2)/(2k(B)T), R and Omega are the cylinder radius and angular velocity, m is the molecular mass, rho(w) is the wall density, mu is the viscosity and T is the temperature. The leading-order solutions do capture the qualitative trends, but are not in quantitative agreement.

Logarithmic corrections to extremal black hole entropy in N=2, 4 and 8 supergravity

We compute the logarithmic correction to black hole entropy about exponentially suppressed saddle points of the Quantum Entropy Function corresponding to Z(N) orbifolds of the near horizon geometry of the extremal black hole under study. By carefully accounting for zero mode contributions we show that the logarithmic contributions for quarter-BPS black holes in N = 4 supergravity and one-eighth BPS black holes in N = 8 supergravity perfectly match with the prediction from the microstate counting. We also find that the logarithmic contribution for half-BPS black holes in N = 2 supergravity depends non-trivially on the Z(N) orbifold. Our analysis draws heavily on the results we had previously obtained for heat kernel coefficients on Z(N) orbifolds of spheres and hyperboloids in arXiv:1311.6286 and we also propose a generalization of the Plancherel formula to Z(N) orbifolds of hyperboloids to an expression involving the Harish-Chandra character of sl (2, R), a result which is of possible mathematical interest.