Incremental natural-gradient actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic rein- forcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their com- patibility with function approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further re- duce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal differ- ence learning in the actor and by incorporating natural gradients, and they extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

Multi-point sensing system for plantar pressure measurement

In this paper, the development of a novel multipoint pressure sensor system suitable for the measurement of human foot pressure distribution has been presented. It essentially consists of a matrix of cantilever sensing elements supported by beams. Foil type strain gauges have been employed for the conversion of foot pressure in to proportional electrical response. Information on the signal conditioning circuitry used is given. Also, the results obtained on the performance of the system are included.

MEMS based pressure sensor with triple modular redundancy

In this paper, the design and development of micro electro mechanical systems (MEMS) based pressure sensor with triple modular redundancy (TMR) for space applications has been presented. In order to minimize the mass of the system and also to avoid the uncertainty in the pressure measurement of the three independent hardware, an integrated approach with TMR is adopted. Sequential steps of TMR logic followed and the test results obtained are included.

Effect of oxygen pressure on the grain and domain structure of polycrystalline 0.85PbMg(1/3)Nb(2/3)O(3)-0.15PbTiO(3) thin films studied by scanning probe microscopy

0.85PbMg(1/3)Nb(2/3)O(3)-0.15PbTiO(3) ferroelectric-relaxor thin films have been deposited on La(0.5)nSr(0.5)CoO(3)/(1 1 1) Pt/TiO(2)/SiO(2)/Si by pulsed laser ablation at various oxygen partial pressures in the range 0.05 to 0.4 Torr. All the films have a rhombohedral perovskite structure. The grain morphology and orientation are drastically affected by the oxygen pressure, studied by x-ray diffraction and scanning electron microscopy. The domain structure investigations by dynamic contact electrostatic force microscopy have revealed that the distribution of polar nanoregions and their dynamics is influenced by the grain morphology, orientation and more importantly, oxygen vacancies. The correlation length extracted from autocorrelation function images has shown that the polarization disorder decreases with oxygen pressure up to 0.3 Torr. The presence of polarized domains and their electric field induced switching is discussed in terms of internal bias field and domain wall pinning. Film deposited at 0.4 Torr presents a curious case with unique triangular grain morphology and large polarization disorder.

Vapor pressure and dissociation energy of (In2O)

The vapor pressure of pure liquid indium, and the sum of pressures of (In) and (In2O) species over the condensed phase mixture {In} + , contained in a silica vessel, have been measured by Knudsen effusion and Langmuir free vaporization methods in the temperatue range 600 to 950°C. Mass spectrometric studies reported in the literature show that (In) and (In2O) are the important species in the vapor phase over the {In} + ; mixture. The vapor pressure of (In2O) corresponding to the reaction, deduced from the present measurements is given by the equation, The “apparent evaporation coefficient” for the condensed phase mixture is approximately 0.8. The energy for the dissociation (In2O) molecule into atoms calculated from the above equation is D°0 = 180.0 (± 1.0) kcal mol−1.

Vapor pressure and thermodynamic properties of MgIn2O4

The vapor pressure of pure indium, and the sum of the pressures of (In) and (In2O) species over the condensed phase mixture {In} + 〈MgIn2O4〉 + 〈MgO〉, have been measured by the Knudsen effusion technique in the temperature range 1095–1350 K. The materials under study were contained in a zirconia crucible, which had a Knudsen orifice along the vertical wall. The major vapor species over the condensed phase mixture were identified as (In) and (In2O) using a mass-spectrometer. The vapor pressure of (In2O) corresponding to the reaction,View the MathML source was deduced from the experimental results;View the MathML source The standard free energy of formation of the inverse spinel 〈MgIn2O4〉 from its component oxides, is given by,View the MathML source View the MathML source The entropy of transformation of 〈In2O3〉 from the C rare-earth structure to the corundum structure is evaluated from the measured entropy of formation of (MgIn2O4) and a semi-empirical correlation for the entropy of formation of spinel phases from component oxides with rock-salt and corundum structures.

Vapor pressure of liquid metals and alloys

The Gibbs-Bogoliubov formalism in conjunction with the pseudopotential theory is applied to the calculation of the vapour pressure of eight liquid metals from Groups I to IV of the periodic table and of alloys (Na-K). The calculated vapour pressure of the elements and their temperature dependencies, the partial pressures, activities and boiling points of the alloys are all found to be in reasonable agreement with measured data.

Application of composition gradient solid electrolytes in sensors and thermodynamic measurements

New compos~tiong radient solid electrolytes are developed which have application in high temperature solid state galvanic sensors and provide a new tool for thermodynamic measurements. The electrolyte consists oi a solid solution between two ionic conductors with a common mobile ion and spatial variation in composition of otber coxup nents. Incorporation of the composite electrolyte in sensors permits the use oi dissimilar gas electrodes. It is demonsuated, both experimentall y and theoretically, that the composition gradient of the relativeiy immobile species does not give rise to a diffusion potential.The emi of a cell is determined by the activity of the mobile species at the two eiectrodes. The thermodynamic properties of solid solutions can be measured using the gradient solid electrolyte. The experimental stuay is based on model systems A?(COj)x(S04)l-x (A=Na,K),where S \.aria across the electrolyte. The functionally gradient solid electrolytes used for activity measurements consist of pure carbonate at one ena and the solid solution under stuav at the other. The identical vaiues of activity, obtained h m t hree different modes of operation of the ceil. indicate unit transport number for the ddi metal ion in the graciient electrolyte. Tlle activities in the solid solutions exhibit moderate positive deviations from Raoult 's law.