Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

The partition of unity finite element method for elastic wave propagation in Reissner-Mindlin plates

This paper reports a numerical method for modelling the elastic wave propagation in plates. The method is based on the partition of unity approach, in which the approximate spectral properties of the infinite dimensional system are embedded within the space of a conventional finite element method through a consistent technique of waveform enrichment. The technique is general, such that it can be applied to the Lagrangian family of finite elements with specific waveform enrichment schemes, depending on the dominant modes of wave propagation in the physical system. A four-noded element for the Reissner-indlin plate is derived in this paper, which is free of shear locking. Such a locking-free property is achieved by removing the transverse displacement degrees of freedom from the element nodal variables and by recovering the same through a line integral and a weak constraint in the frequency domain. As a result, the frequency-dependent stiffness matrix and the mass matrix are obtained, which capture the higher frequency response with even coarse meshes, accurately. The steps involved in the numerical implementation of such element are discussed in details. Numerical studies on the performance of the proposed element are reported by considering a number of cases, which show very good accuracy and low computational cost. Copyright (C)006 John Wiley & Sons, Ltd.

Ductile deformation of single inclusions in simple shear with a finite-strain hyperelastoviscoplastic rheology

We examine the 2D plane-­strain deformation of initially round, matrix-­bonded, deformable single inclusions in isothermal simple shear using a recently introduced hyperelastoviscoplastic rheology. The broad parameter space spanned by the wide range of effective viscosities, yield stresses, relaxation times, and strain rates encountered in the ductile lithosphere is explored systematically for weak and strong inclusions, the effective viscosity of which varies with respect to the matrix. Most inclusion studies to date focused on elastic or purely viscous rheologies. Comparing our results with linear-­viscous inclusions in a linear-­viscous matrix, we observe significantly different shape evolution of weak and strong inclusions over most of the relevant parameter space. The evolution of inclusion inclination relative to the shear plane is more strongly affected by elastic and plastic contributions to rheology in the case of strong inclusions. In addition, we found that strong inclusions deform in the transient viscoelastic stress regime at high Weissenberg numbers (≥0.01) up to bulk shear strains larger than 3. Studies using the shapes of deformed objects for finite-­strain analysis or viscosity-­ratio estimation should establish carefully which rheology and loading conditions reflect material and deformation properties. We suggest that relatively strong, deformable clasts in shear zones retain stored energy up to fairly high shear strains. Hence, purely viscous models of clast deformation may overlook an important contribution to the energy budget, which may drive dissipation processes within and around natural inclusions.

The dynamics of solvation of an electron in the image potential state by a layer of polar adsorbates

Recently, ultrafast two-photon photoemission has been used to study electron solvation at a two-dimensional metal/polar adsorbate interfaces [A. Miller , Science 297, 1163 (2002)]. The electron is bound to the surface by the image interaction. Earlier we have suggested a theoretical description of the states of the electron interacting with a two-dimensional layer of the polar adsorbate [K. L. Sebastian , J. Chem. Phys. 119, 10350 (2003)]. In this paper we have analyzed the dynamics of electron solvation, assuming a trial wave function for the electron and the solvent polarization and then using the Dirac-Frenkel variational method to determine it. The electron is initially photoexcited to a delocalized state, which has a finite but large size, and causes the polar molecules to reorient. This reorientation acts back on the electron and causes its wave function to shrink, which will cause further reorientation of the polar molecules, and the process continues until the electron gets self-trapped. For reasonable values for the parameters, we are able to obtain fair agreement with the experimental observations. (c) 2005 American Institute of Physics.

Thermal stresses in a finite hollow cylinder due to an axisymmetric temperature field at the end surface

A three-dimensional exact solution for determining the thermal stresses in a finite hollow cylinder subject to a steady state axisymmetric temperature field over one of its end surfaces has been given. Numerical results for a hollow cylinder, having lenght to outer diameter ratio equal to one and inner to outer diameter ratio equal to 0.75, subjected to a symmetric temperature variation over the end surfaces of the cylinder have been given.

Thermal stresses in a finite solid cylinder due to an axisymmetric temperature field at the end surface.

A three-dimensional rigorous solution for determining thermal stresses in a finite solid cylinder due to a steady state axisymmetric temperature field over one of its end surfaces is given. Numerical results for a solid cylinder having a length to diameter ratio equal to one and subjected to a symmetric temperature variation over half the radius of the cylinder at the end surfaces are included. These results have been compared with the results of the approximate solution given by W. Nowacki.

Influence of finite size effect on magnetic and magnetotransport properties of La0.5Sr0.5CoO3 thin films

We present a comprehensive study of the thickness dependent structural, magnetic and magnetotransport properties of oriented La0.5Sr0.5CoO3 thin films grown on LaAlO3 by Pulsed Laser Deposition. We observe that these films undergo a reduction in Curie temperature (T-c) with a decrease in film thickness, and it is found to be primarily caused by the finite size effect since the finite scaling law [T-c(infinity) T-c(t)/T-c(infinity) = (c/t)lambda holds good over the studied thickness range. We rule out the contribution from the strain induced suppression of Curie temperature with decreasing film thickness since all the films exhibit a constant out of plane tensile strain (0.5%) irrespective of their varying thickness. However, we observe that the coercivity of the films is an order of magnitude higher than that of the bulk due to the tensile strain. In addition, we also observe an increase in the magneto resistance peak and a decrease in coercivity and electrical resistivity with an increase in film thickness. (C) 2010 Elsevier Ltd. All rights reserved.

Generation of parabolic trajectories for current error space phasor similar to an SVPWM controller for control of switching frequency variation in current hysteresis PWM controlled IM drives

Switching frequency variation over a fundamental period is a major problem associated with hysteresis controller based VSI fed IM drives. This paper describes a novel concept of generating parabolic trajectories for current error space phasor for controlling the switching frequency variation in the hysteresis controller based two-level inverter fed IM drives. A generalized algorithm is developed to determine unique set of parabolic trajectories for different speeds of operation for any given IM load. Proposed hysteresis controller provides the switching frequency spectrum of inverter output voltage, similar to that of the constant switching frequency VC-SVPWM based IM drive. The scheme is extensively simulated and experimentally verified on a 3.7 kW IM drive for steady state and transient performance.

A finite element analysis of quasistatic crack growth in a pressure sensitive constrained ductile layer

Polymeric adhesive layers are employed for bonding two components in a wide variety of technological applications, It has been observed that, unlike in metals, the yield behavior of polymers is affected by the state of hydrostatic stress. In this work, the effect of pressure sensitivity of yielding and layer thickness on quasistatic interfacial crack growth in a ductile adhesive layer is investigated. To this end, finite deformation, finite element analyses of a cracked sandwiched layer are carried out under plane strain, small-scale yielding conditions for a wide range of mode mixities. The Drucker-Prager constitutive equations are employed to represent the behavior of the layer. Crack propagation is simulated through a cohesive zone model, in which the interface is assumed to follow a prescribed traction-separation law. The results show that for a given mode mixity, the steady state Fracture toughness [K](ss) is enhanced as the degree of pressure sensitivity increases. Further, for a given level of pressure sensitivity, [K](ss) increases steeply as mode Il loading is approached. (C) 2000 Elsevier Science Ltd. All rights reserved.

Spatial survival probability for one-dimensional fluctuating interfaces in the steady state

We report numerical and analytic results for the spatial survival probability for fluctuating one-dimensional interfaces with Edwards-Wilkinson or Kardar-Parisi-Zhang dynamics in the steady state. Our numerical results are obtained from analysis of steady-state profiles generated by integrating a spatially discretized form of the Edwards-Wilkinson equation to long times. We show that the survival probability exhibits scaling behavior in its dependence on the system size and the "sampling interval" used in the measurement for both "steady-state" and "finite" initial conditions. Analytic results for the scaling functions are obtained from a path-integral treatment of a formulation of the problem in terms of one-dimensional Brownian motion. A "deterministic approximation" is used to obtain closed-form expressions for survival probabilities from the formally exact analytic treatment. The resulting approximate analytic results provide a fairly good description of the numerical data.

A rectangular laminated anisotropic shallow thin shell finite element

This report contains the details of the development of the stiffness matrix for a rectangular laminated anisotropic shallow thin shell finite element. The derivation is done under linear thin shell assumptions. Expressing the assumed displacement state over the middle surface of the shell as products of one-dimensional first-order Hermite interpolation polynomials, it is possible to insure that the displacement state for the assembled set of such elements, to be geometrically admissible. Monotonic convergence of the total potential energy is therefore possible as the modelling is successively refined. The element is systematically evaluated for its performance considering various examples for which analytical or other solutions are available

On the charging state of atmospheric aerosols and ion-induced nucleation

Floating in the air that surrounds us is a number of small particles, invisible to the human eye. The mixture of air and particles, liquid or solid, is called an aerosol. Aerosols have significant effects on air quality, visibility and health, and on the Earth's climate. Their effect on the Earth's climate is the least understood of climatically relevant effects. They can scatter the incoming radiation from the Sun, or they can act as seeds onto which cloud droplets are formed. Aerosol particles are created directly, by human activity or natural reasons such as breaking ocean waves or sandstorms. They can also be created indirectly as vapors or very small particles are emitted into the atmosphere and they combine to form small particles that later grow to reach climatically or health relevant sizes. The mechanisms through which those particles are formed is still under scientific discussion, even though this knowledge is crucial to make air quality or climate predictions, or to understand how aerosols will influence and will be influenced by the climate's feedback loops. One of the proposed mechanisms responsible for new particle formation is ion-induced nucleation. This mechanism is based on the idea that newly formed particles were ultimately formed around an electric charge. The amount of available charges in the atmosphere varies depending on radon concentrations in the soil and in the air, as well as incoming ionizing radiation from outer space. In this thesis, ion-induced nucleation is investigated through long-term measurements in two different environments: in the background site of Hyytiälä and in the urban site that is Helsinki. The main conclusion of this thesis is that ion-induced nucleation generally plays a minor role in new particle formation. The fraction of particles formed varies from day to day and from place to place. The relative importance of ion-induced nucleation, i.e. the fraction of particles formed through ion-induced nucleation, is bigger in cleaner areas where the absolute number of particles formed is smaller. Moreover, ion-induced nucleation contributes to a bigger fraction of particles on warmer days, when the sulfuric acid and water vapor saturation ratios are lower. This analysis will help to understand the feedbacks associated with climate change.

Exact static polarizabilities of correlated finite model systems

A finite-field method for calculating exact polarizabilities of correlated conjugated model systems within the valence bond (VB) framework is presented. The correlations reduce the polarizabilities from their noninteracting values and extend the range of linearity to higher external fields. The large nonlinear polarizabilities observed in strongly correlated conjugated organic molecules cannot be directly attributed to electron correlations. The method described can be employed to calculate static polarizabilities for any desired state of a correlated system.

An accurate numerical integration scheme for finite rotations using rotation vector parametrization

A numerical integration procedure for rotational motion using a rotation vector parametrization is explored from an engineering perspective by using rudimentary vector analysis. The incremental rotation vector, angular velocity and acceleration correspond to different tangent spaces of the rotation manifold at different times and have a non-vectorial character. We rewrite the equation of motion in terms of vectors lying in the same tangent space, facilitating vector space operations consistent with the underlying geometric structure. While any integration algorithm (that works within a vector space setting) may be used, we presently employ a family of explicit Runge-Kutta algorithms to solve this equation. While this work is primarily motivated out of a need for highly accurate numerical solutions of dissipative rotational systems of engineering interest, we also compare the numerical performance of the present scheme with some of the invariant preserving schemes, namely ALGO-C1, STW, LIEMIDEA] and SUBCYC-M. Numerical results show better local accuracy via the present approach vis-a-vis the preserving algorithms. It is also noted that the preserving algorithms do not simultaneously preserve all constants of motion. We incorporate adaptive time-stepping within the present scheme and this in turn enables still higher accuracy and a `near preservation' of constants of motion over significantly longer intervals. (C) 2010 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.

Spin-1 Kitaev model in one dimension

We study a one-dimensional version of the Kitaev model on a ring of size N, in which there is a spin S > 1/2 on each site and the Hamiltonian is J Sigma(nSnSn+1y)-S-x. The cases where S is integer and half-odd integer are qualitatively different. We show that there is a Z(2)-valued conserved quantity W-n for each bond (n, n + 1) of the system. For integer S, the Hilbert space can be decomposed into 2N sectors, of unequal sizes. The number of states in most of the sectors grows as d(N), where d depends on the sector. The largest sector contains the ground state, and for this sector, for S=1, d=(root 5+1)/2. We carry out exact diagonalization for small systems. The extrapolation of our results to large N indicates that the energy gap remains finite in this limit. In the ground-state sector, the system can be mapped to a spin-1/2 model. We develop variational wave functions to study the lowest energy states in the ground state and other sectors. The first excited state of the system is the lowest energy state of a different sector and we estimate its excitation energy. We consider a more general Hamiltonian, adding a term lambda Sigma W-n(n), and show that this has gapless excitations in the range lambda(c)(1)<=lambda <=lambda(c)(2). We use the variational wave functions to study how the ground-state energy and the defect density vary near the two critical points lambda(c)(1) and lambda(c)(2).