Incremental natural-gradient actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic rein- forcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their com- patibility with function approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further re- duce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal differ- ence learning in the actor and by incorporating natural gradients, and they extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

Potentiometric determination of the Gibbs energies of formation of lead aluminates

The Gibbs energies of formation of three compounds in the PbO-Al2O3 system—2PbO · Al2O3, PbO · Al2O3, andPbO· 6Al2O3—have been determined from potentiometric measurements on reversible solid-state galvanic cells [dformula Pt, Ir | Pb, alpha-Al[sub 2]O[sub 3], PbO [center-dot] 6Al[sub 2]O[sub 3] | ZrO[sub 2]-CaO | NiO, Ni | Pt] [dformula Pt | NiO, Ni | ZrO[sub 2]-CaO | Pb, PbO [center-dot] 6Al[sub 2]O[sub 3], PbO [center-dot] Al[sub 2]O[sub 3] | Ir, Pt] and [dformula Pt | NiO, Ni | ZrO[sub 2]-CaO | Pb, PbO [center-dot] Al[sub 2]O[sub 3], 2PbO [center-dot] Al[sub 2]O[sub 3] | Ir, Pt] in the temperature range 850–1375 K. The results are discussed in the light of reported phase diagrams for the PbO-Al2O3system. The partial pressures of different lead oxide species, PbnOn, n = 1–6, in the gas phase in equilibrium withthe aluminates are calculated by combining the results of this study with the mass-spectrometric data of Drowart et al.(1) for polymerization equilibria in the gas phase. The concentration of oxygen in lead in equilibrium with the aluminatesare also derived from the results and the literature data on the Gibbs energy of solution of oxygen in liquid lead.

Analysis of a linear one-dimensional active noise control system by means of block diagrams and transfer functions

In arriving at the ideal filter transfer function for an active noise control system in a duct, the effect of the auxiliary sources (generally loudspeakers) on the waves generated by the primary source has invariably been neglected in the existing literature, implying a rigid wall or infinite impedance. The present paper presents a fairly general analysis of a linear one-dimensional noise control system by means of block diagrams and transfer functions. It takes into account the passive as well as active role of a terminal primary source, wall-mounted auxiliary source, open duct radiation impedance, and the effects of mean flow and damping. It is proved that the pressure generated by a source against a load impedance can be looked upon as a sum of two pressure waves, one generated by the source against an anechoic termination and the other by reflecting the rearward wave (incident on the source) off the passive source impedance. Application of this concept is illustrated for both the types of sources. A concise closed-form expression for the ideal filter transfer function is thus derived and discussed. Finally, the dynamics of an adaptive noise control system is discussed briefly, relating its standing-wave variables and transfer functions with those of the progressive-wave model presented here.

Minor mergers and their impact on the kinematics of old and young stellar populations in disk galaxies

By means of N-body simulations we investigate the impact of minor mergers on the angular momentum and dynamical properties of the merger remnant. Our simulations cover a range of initial orbital characteristics and gas-to-stellar mass fractions (from 0 to 20%), and include star formation and supernova feedback. We confirm and extend previous results by showing that the specific angular momentum of the stellar component always decreases independently of the orbital parameters or morphology of the satellite, and that the decrease in the rotation velocity of the primary galaxy is accompanied by a change in the anisotropy of the orbits. However, the decrease affects only the old stellar population, and not the new population formed from gas during the merging process. This means that the merging process induces an increasing difference in the rotational support of the old and young stellar components, with the old one lagging with respect to the new. Even if our models are not intended specifically to reproduce the Milky Way and its accretion history, we find that, under certain conditions, the modeled rotational lag found is compatible with that observed in the Milky Way disk, thus indicating that minor mergers can be a viable way to produce it. The lag can increase with the vertical distance from the disk midplane, but only if the satellite is accreted along a direct orbit, and in all cases the main contribution to the lag comes from stars originally in the primary disk rather than from stars in the satellite galaxy. We also discuss the possibility of creating counter-rotating stars in the remnant disk, their fraction as a function of the vertical distance from the galaxy midplane, and the cumulative effect of multiple mergers on their creation.

Intra-molecular hydrogen bonding with organic fluorine in the solution state: Deriving evidence by a two dimensional NMR experiment

The direct evidence for the existence of intra-molecular C-F center dot center dot center dot H-N hydrogen bond in organofluorine molecules, in the liquid state, is derived using NMR spectroscopy by the detection of long range interactions among fluorine, nitrogen and hydrogen atoms. The present study reports the determination of the relative signs and magnitudes of through space and through bond couplings to draw unambiguous evidence on the existence of weak molecular interactions involving organic fluorine. It is a simple, easy to implement, N-15 natural abundant two dimensional heteronuclear N-15-H-1 double quantum-single quantum correlation experiment. The existence of intra-molecular hydrogen bond is conclusively established in the investigated molecules. (C) 2011 Elsevier B.V. All rights reserved.

Climatic fluctuations during the LIA and post-LIA in the Kumaun Lesser Himalaya, India: Evidence from a 400 y old stalagmite record

This paper presents the first stable isotope (delta O-18 and delta C-13) data of a similar to 400 years (1590-2006 AD) long annual to decadal-resolution speleothem record collected from the Indian Lesser Himalaya. The data show a variation from -2.7 to -5.9 parts per thousand in delta O-18 and -5.3 to -8.8 parts per thousand in delta C-13. The isotopic analyses indicate that the climate during this period can be divided into two stages: a wet phase during the Little Ice Age (LIA) (1590-1850 AD) and comparatively dry phase during the post-LIA after 1850 AD. However, the record also documents the minor dry events during the LIA and a wet episode after the LIA. Within the age uncertainty, the dry spells during the LIA are linked with the historical drought events in the Indian subcontinent and similar latitudes. The isotopic record is consistent with a number of previous studies in the areas influenced by the Westerlies but appears to be conflicting to the regions, dominated by the Indian Summer Monsoon (ISM). This may be due to the possible changes in the strength of Westerlies in the study area and added by negative anomaly of North Atlantic Oscillation (NAO) during the LIA. (C) 2012 Elsevier Ltd and INQUA. All rights reserved.

Insights to urban dynamics through landscape spatial pattern analysis

Urbanisation is a dynamic complex phenomenon involving large scale changes in the land uses at local levels. Analyses of changes in land uses in urban environments provide a historical perspective of land use and give an opportunity to assess the spatial patterns, correlation, trends, rate and impacts of the change, which would help in better regional planning and good governance of the region. Main objective of this research is to quantify the urban dynamics using temporal remote sensing data with the help of well-established landscape metrics. Bangalore being one of the rapidly urbanising landscapes in India has been chosen for this investigation. Complex process of urban sprawl was modelled using spatio temporal analysis. Land use analyses show 584% growth in built-up area during the last four decades with the decline of vegetation by 66% and water bodies by 74%. Analyses of the temporal data reveals an increase in urban built up area of 342.83% (during 1973-1992), 129.56% (during 1992-1999), 106.7% (1999-2002), 114.51% (2002-2006) and 126.19% from 2006 to 2010. The Study area was divided into four zones and each zone is further divided into 17 concentric circles of 1 km incrementing radius to understand the patterns and extent of the urbanisation at local levels. The urban density gradient illustrates radial pattern of urbanisation for the period 1973-2010. Bangalore grew radially from 1973 to 2010 indicating that the urbanisation is intensifying from the central core and has reached the periphery of the Greater Bangalore. Shannon's entropy, alpha and beta population densities were computed to understand the level of urbanisation at local levels. Shannon's entropy values of recent time confirms dispersed haphazard urban growth in the city, particularly in the outskirts of the city. This also illustrates the extent of influence of drivers of urbanisation in various directions. Landscape metrics provided in depth knowledge about the sprawl. Principal component analysis helped in prioritizing the metrics for detailed analyses. The results clearly indicates that whole landscape is aggregating to a large patch in 2010 as compared to earlier years which was dominated by several small patches. The large scale conversion of small patches to large single patch can be seen from 2006 to 2010. In the year 2010 patches are maximally aggregated indicating that the city is becoming more compact and more urbanised in recent years. Bangalore was the most sought after destination for its climatic condition and the availability of various facilities (land availability, economy, political factors) compared to other cities. The growth into a single urban patch can be attributed to rapid urbanisation coupled with the industrialisation. Monitoring of growth through landscape metrics helps to maintain and manage the natural resources. (C) 2012 Elsevier B.V. All rights reserved.

Characterization and iodide adsorption behaviour of HDPY+ modified bentonite

Owing to its favourable physical, chemical and rheological properties, densely compacted bentonite or bentonite-sand mix is considered as a suitable buffer material in deep geological repositories to store high level nuclear waste. Iodine-129 is one of the significant nuclides in the high level waste owing to its long half life and poor sorption onto most geologic media. Bentonite by virtue of negatively charged surface has negligible affinity to retain iodide ions. As organo-bentonites are known to retain iodide ions, the present study characterizes hexadecylpyridinium chloride (HDPyCl.H2O) treated bentonite from Barmer India (referred as HDPy+B) for physico-chemical properties, engineering properties and the iodide adsorption behavior of the organo clay. Batch experiments revealed that HDPy+ ions are largely retained (94 % retention) via cation exchange; the ion-exchange process neutralizes the negative surface charge and bridges clay particles leading to reduction in Atterberg limits, clay content and sediment volume. The organo clay retains iodide by Coulombic attraction (at primary sites) and anion exchange (at secondary sites). The free-energy change (Delta G (o) = -25.5 kJ/mol) value indicated that iodide retention by organo clay is favored physical adsorption process. Iodide adsorption capacity of organo clay decreased significantly (85-100 %) on dilution with 50-80 % bentonite. On the other hand, dilution of bentonite with 50 % organo clay caused 58 % reduction in swell potential and 21 % reduction in swell pressure.

Lacrimal Proline Rich 4 (LPRR4) Protein in the Tear Fluid Is a Potential Biomarker of Dry Eye Syndrome

Dry eye syndrome (DES) is a complex, multifactorial, immune-associated disorder of the tear and ocular surface. DES with a high prevalence world over needs identification of potential biomarkers so as to understand not only the disease mechanism but also to identify drug targets. In this study we looked for differentially expressed proteins in tear samples of DES to arrive at characteristic biomarkers. As part of a prospective case-control study, tear specimen were collected using Schirmer strips from 129 dry eye cases and 73 age matched controls. 2D electrophoresis (2DE) and Differential gel electrophoresis (DIGE) was done to identify differentially expressed proteins. One of the differentially expressed protein in DES is lacrimal proline rich 4 protein (LPRR4). LPRR4 protein expression was quantified by enzyme immune sorbent assay (ELISA). LPRR4 was down regulated significantly in all types of dry eye cases, correlating with the disease severity as measured by clinical investigations. Further characterization of the protein is required to assess its therapeutic potential in DES.

DRIFTS studies on CO and NO adsorption and NO plus CO reaction over Pd2+-substituted CeO2 and Ce0.75Sn0.25O2 catalysts

Sequential adsorption of CO and NO as well as equimolar NO + CO reaction with variation of temperature over Pd2+ ion-substituted CeO2 and Ce0.75Sn0.25O2 supports has been studied by DRIFTS technique. The results are compared with 2 at.% Pd/Al2O3 containing Pd-0. Both linear and bridging Pd-0-CO bands are observed over 2 at.% Pd/Al2O3. But, band positions are shifted to higher frequencies in Ce0.98Pd0.02O2-delta and Ce0.73Sn0.25Pd0.02O2-delta catalysts that could be associated with Pd delta+-CO species. In contrast, a Pd2+-CO band at 2160 cm(-1) is observed upon CO adsorption over Ce0.98Pd0.02O2-delta and Ce0.73Sn0.25Pd0.02O2-delta catalysts pre-adsorbed with NO and a Pd+-CO band at 2120 cm(-1) is slowly developed on Ce(0.73)Srl(0.25)Pd(0.02)O(2-delta) over time. An intense linear Pd-0-NO band at 1750 cm(-1) found upon NO exposure to CO pre-adsorbed 2 at.% Pd/Al2O3 indicates molecular adsorption of NO. On the other hand, a weak Pd2+-NO band at 1850 cm(-1) is noticed after NO exposure to Ce0.98Pd0.02O2-delta catalyst pre-adsorbed with CO indicating dissociative adsorption of NO which is crucial for NO reduction. Pd-0-NO band is initially formed over CO pre-adsorbed Ce0.73Sn0.25Pd0.02O2-delta which is red-shifted over time along with formation of Pd2+-NO band. Several intense bands related to nitrates and nitrites are observed after exposure of NO to fresh as well as CO pre-adsorbed Ce0.98Pd0.02O2-delta and Ce0.73Sn0.25Pd0.02O2-delta catalysts. Ramping the temperature in a DRIFTS cell upon NO and CO adsorption shows the formation of N2O and NCO surface species, and N2O-formation temperature is comparable with the reaction done in a reactor.

Chromium isotope variations (delta Cr-53/52) in mantle-derived sources and their weathering products: Implications for environmental studies and the evolution of delta Cr-53/52 in the Earth's mantle over geologic time

Here we report chromium isotope compositions, expressed as delta Cr-53/ 52 in per mil (&) relative to NIST 979, measured in selected Cr-rich minerals and rocks formed by the primary magmatic as well as the secondary metamorphic and weathering processes. The main objectives of this study were: (i) to further constrain the isotope composition of the Earth's mantle Cr inventory and its possible variation during geological history, based on the analysis of globally distributed and stratigraphically constrained mantle-derived chromites; and (ii) to investigate the magnitude and systematics of Cr isotope fractionation during oxidative weathering and secondary alteration (i. e., hydration, serpentinization) of the magmatic Cr sources. Specifically, we analyzed delta Cr-53/ 52 in a set of globally distributed mantle-derived chromites (FeMgCr2O4, n = 30) collected from various locations in Europe, Asia, Africa and South America, and our results confirm that a chromite-hosted Earth's mantle Cr inventory is uniform at - 0.079 +/- 0.129& (2SD), which we named here as a ` canonical' mantle d 53/ 52 Cr signature. Furthermore our dataset of stratigraphically constrained chromites, whose crystallization ages cover most of the Earth's geological history, indicate that the bulk Cr isotope composition of the chromite-hosted mantle inventory has remained uniform, within about +/- 0.100&, since at least the Early Archean times (similar to 3500 million years ago, Ma). To investigate the systematics of Cr isotope fractionation associated with alteration processes we analyzed a number of secondary Cr-rich minerals and variably altered ultramafic rocks (i. e., serpentinized harzburgites, lherzolites) that revealed large positive delta Cr-53/ 52 anomalies that are systematically shifted to higher values with an increasing degree of alteration and serpentinization. The degree of aqueous alteration and serpentinization was quantified by the abundances of fluid-mobile (Rb, K) elements, and by the Loss On Ignition (LOI) parameter, which determines the amount of structurally bound water (OH/ H2O) present in secondary hydrated minerals like serpentine. Overall, we observed that altered ultramafic rocks that yielded the highest LOI values, and the lowest amounts of fluid mobile elements, also yielded the heaviest delta Cr-53/ 52 signatures. Therefore, we conclude that secondary alteration (i.e., hydration, serpentinization) of ultramafic rocks in near-surface oxidative environments tend to shift the bulk Cr isotope composition of the weathered products to isotopically heavier values, pointing to a dynamic redox cycling of Cr in the Earth's crustal and near-surface environments. Hence, if validated by future