Incremental natural-gradient actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic rein- forcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their com- patibility with function approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further re- duce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal differ- ence learning in the actor and by incorporating natural gradients, and they extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

REDEFINE: Architecture of a SoC Fabric for Runtime Composition of Computation Structures

In this paper we propose the architecture of a SoC fabric onto which applications described in a HLL are synthesized. The fabric is a homogeneous layout of computation, storage and communication resources on silicon. Through a process of composition of resources (as opposed to decomposition of applications), application specific computational structures are defined on the fabric at runtime to realize different modules of the applications in hardware. Applications synthesized on this fabric offers performance comparable to ASICs while retaining the programmability of processing cores. We outline the application synthesis methodology through examples, and compare our results with software implementations on traditional platforms with unbounded resources.

Redundancy resolution using a tractrix and its application to real-time simulations of hyper-redundant manipulators, snakes and tying of knots

To realistically simulate the motion of flexible objects such as ropes, strings, snakes, or human hair,one strategy is to discretise the object into a large number of small rigid links connected by rotary or spherical joints. The discretised system is highly redundant and the rotations at the joints (or the motion of the other links) for a desired Cartesian motion of the end of a link cannot be solved uniquely. In this paper, we propose a novel strategy to resolve the redundancy in such hyper-redundant systems.We make use of the classical tractrix curve and its attractive features. For a desired Cartesian motion of the `head'of a link, the `tail' of the link is moved according to a tractrix,and recursively all links of the discretised objects are moved along different tractrix curves. We show that the use of a tractrix curve leads to a more `natural' motion of the entire object since the motion is distributed uniformly along the entire object with the displacements tending to diminish from the `head' to the `tail'. We also show that the computation of the motion of the links can be done in real time since it involves evaluation of simple algebraic, trigonometric and hyperbolic functions. The strategy is illustrated by simulations of a snake, tying of knots with a rope and a solution of the inverse kinematics of a planar hyper-redundant manipulator.

Effective thermal conductivity of a heat generating rod bundle dissipating heat by natural convection and radiation

A numerical study of conjugate natural convection and surface radiation in a horizontal hexagonal sheath housing 19 solid heat generating rods with cladding and argon as the fill gas, is performed. The natural convection in the sheath is driven by the volumetric heat generation in the solid rods. The problem is solved using the FLUENT CFD code. A correlation is obtained to predict the maximum temperature in the rod bundle for different pitch-to-diameter ratios and heat generating rates. The effective thermal conductivity is related to the heat generation rate, maximum temperature and the sheath temperature. Results are presented for the dimensionless maximum temperature, Rayleigh number and the contribution of radiation with changing emissivity, total wattage and the pitch-to-diameter ratio. In the simulation of a larger system that contains a rod bundle, the effective thermal conductivity facilitates simplified modelling of the rod bundle by treating it as a solid of effective thermal conductivity. The parametric studies revealed that the contribution of radiation can be 38-65% of the total heat generation, for the parameter ranges chosen. Data for critical Rayleigh number above which natural convection comes into effect is also presented. (C) 2011 Elsevier B.V. All rights reserved.

The Future of Computation

The purpose of life is to obtain knowledge, use it to live with as much satisfaction as possible, and pass it on with improvements and modifications to the next generation.'' This may sound philosophical, and the interpretation of words may be subjective, yet it is fairly clear that this is what all living organisms--from bacteria to human beings--do in their life time. Indeed, this can be adopted as the information theoretic definition of life. Over billions of years, biological evolution has experimented with a wide range of physical systems for acquiring, processing and communicating information. We are now in a position to make the principles behind these systems mathematically precise, and then extend them as far as laws of physics permit. Therein lies the future of computation, of ourselves, and of life.

Computation of thermodynamic properties of liquid ferrous alloys from structural measurements

Relation between X-ray scattering intensities, mean square thermal fluctuations and thermodynamic properties. High temperature X-ray diffraction study of liquid Fe-Ni and Fe-Si alloys using reflection and transmission geometries. Calculation of the structure factor as a function of wave vector. Extrapolation to zero wave vector. Calculation of the concentration-concentration correlation function defined by A. B. Bhatia and D. E. Thorton. Computation of thermodynamic quantities of mixing A G, LlH and LlS for the binary alloys. Comparison with direct thermodynamic measurements reported in the literature.

Computation of thermodynamic properties of quaternary and higher order systems from binary data using shortest distance composition paths

Equations for the computation of integral and partial thermodynamic properties of mixing in quarternary systems are derived using data on constituent binary systems and shortest distance composition paths to the binaries. The composition path from a quarternary composition to the i-j binary is characterized by a constant value of (Xi − Xj). The merits of this composition path over others with constant values for View the MathML source or Xi are discussed. Finally the equations are generalized for higher order systems. They are exact for regular solutions, but may be used in a semiempirical mode for non-regular solutions.

Spinel deoxidation equilibria in the Fe Mn Al O system - experiments and computation

The concentration and chemical potential of oxygen in liquid Fe--Mn alloys equilibrated with the spinel solution, (Fe, Mn)Al sub 2+2x O sub 4+3x , and alpha -Al sub 2 O sub 3 have been determined at 1873K as a function of manganese concentration. The composition of the spinel phase has been determined using electron probe microanalysis. The results are compared with data reported in the literature. The deoxidation equilibrium has been computed using data on free energy of solution of oxygen in liquid iron, free energies of formation of hercynite and galaxite, and interaction parameters reported in the literature. The activity--composition relationship in spinel solution was derived from a cation distribution model. The model is in excellent agreement with the experimental data on oxygen concentration and potential and the composition of the spinel phase. 23 ref.--AA

Ultrasound modulated optical tomography: Young's modulus of the insonified region from measurement of natural frequency of vibration

We demonstrate a method to recover the Young's modulus (E) of a tissue-mimicking phantom from measurements of ultrasound modulated optical tomography (UMOT). The object is insonified by a dualbeam, confocal ultrasound transducer (US) oscillating at frequencies f(0) and f(0) + Delta f and the variation of modulation depth (M) in the autocorrelation of light traversed through the focal region of the US transducer against Delta f is measured. From the dominant peaks observed in the above variation, the natural frequencies of the insonified region associated with the vibration along the US transducer axis are deduced. A consequence of the above resonance is that the speckle fluctuation at the resonance frequency has a higher signal-to-noise to ratio (SNR). From these natural frequencies and the associated eigenspectrum of the oscillating object, Young's modulus (E) of the material in the focal region is recovered. The working of this method is confirmed by recovering E in the case of three tissue-mimicking phantoms of different elastic modulus values. (C) 2011 Optical Society of America

Parallel Computation of 2D Morse-Smale Complexes

The Morse-Smale complex is a useful topological data structure for the analysis and visualization of scalar data. This paper describes an algorithm that processes all mesh elements of the domain in parallel to compute the Morse-Smale complex of large two-dimensional data sets at interactive speeds. We employ a reformulation of the Morse-Smale complex using Forman's Discrete Morse Theory and achieve scalability by computing the discrete gradient using local accesses only. We also introduce a novel approach to merge gradient paths that ensures accurate geometry of the computed complex. We demonstrate that our algorithm performs well on both multicore environments and on massively parallel architectures such as the GPU.

Filling Characteristics for an Activated Carbon Based Adsorbed Natural Gas Storage System

The storage capacity of an activated carbon bed is studied using a 2D transport model with constant inlet flow conditions. The predicted filling times and variation in bed pressure and temperature are in good agreement with experimental observations obtained using a 1.82 L prototype ANG storage cylinder. Storage efficiencies based on the maximum achievable V/V (volume of gas/volume of container) and filling times are used to quantify the performance of the charging process. For the high permeability beds used in the experiments, storage efficiencies are controlled by the rate of heat removal. Filling times, defined as the time at which the bed pressure reaches 3.5 MPa, range from 120 to 3.4 min for inlet flow rates of 1.0 L min(-1) and 30.0 L min(-1), respectively. The corresponding storage efficiencies, eta(s), vary from 90% to 76%, respectively. Simulations with L/D ratios ranging from 0.35 to 7.8 indicate that the storage efficiencies can be improved with an increase in the LID ratios and/or with water cooled convection. Thus for an inlet flow rate of 30.0 L min(-1), an eta(s) value of 90% can be obtained with water cooling for an L/D ratio of 7.8 and a filling time of a few minutes. In the absence of water cooling the eta(s) value reduces to 83% at the same L/D ratio. Our study suggests that with an appropriate choice of cylinder dimensions, solutions based on convective cooling during adsorptive storage are possible with some compromise in the storage capacity.