Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

The partition of unity finite element method for elastic wave propagation in Reissner-Mindlin plates

This paper reports a numerical method for modelling the elastic wave propagation in plates. The method is based on the partition of unity approach, in which the approximate spectral properties of the infinite dimensional system are embedded within the space of a conventional finite element method through a consistent technique of waveform enrichment. The technique is general, such that it can be applied to the Lagrangian family of finite elements with specific waveform enrichment schemes, depending on the dominant modes of wave propagation in the physical system. A four-noded element for the Reissner-indlin plate is derived in this paper, which is free of shear locking. Such a locking-free property is achieved by removing the transverse displacement degrees of freedom from the element nodal variables and by recovering the same through a line integral and a weak constraint in the frequency domain. As a result, the frequency-dependent stiffness matrix and the mass matrix are obtained, which capture the higher frequency response with even coarse meshes, accurately. The steps involved in the numerical implementation of such element are discussed in details. Numerical studies on the performance of the proposed element are reported by considering a number of cases, which show very good accuracy and low computational cost. Copyright (C)006 John Wiley & Sons, Ltd.

Thermal stresses in a finite hollow cylinder due to an axisymmetric temperature field at the end surface

A three-dimensional exact solution for determining the thermal stresses in a finite hollow cylinder subject to a steady state axisymmetric temperature field over one of its end surfaces has been given. Numerical results for a hollow cylinder, having lenght to outer diameter ratio equal to one and inner to outer diameter ratio equal to 0.75, subjected to a symmetric temperature variation over the end surfaces of the cylinder have been given.

Thermal stresses in a finite solid cylinder due to an axisymmetric temperature field at the end surface.

A three-dimensional rigorous solution for determining thermal stresses in a finite solid cylinder due to a steady state axisymmetric temperature field over one of its end surfaces is given. Numerical results for a solid cylinder having a length to diameter ratio equal to one and subjected to a symmetric temperature variation over half the radius of the cylinder at the end surfaces are included. These results have been compared with the results of the approximate solution given by W. Nowacki.

The Line Spectral Frequency Model of a Finite-Length Sequence

The line spectral frequency (LSF) of a causal finite length sequence is a frequency at which the spectrum of the sequence annihilates or the magnitude spectrum has a spectral null. A causal finite-length sequencewith (L + 1) samples having exactly L-LSFs, is referred as an Annihilating (AH) sequence. Using some spectral properties of finite-length sequences, and some model parameters, we develop spectral decomposition structures, which are used to translate any finite-length sequence to an equivalent set of AH-sequences defined by LSFs and some complex constants. This alternate representation format of any finite-length sequence is referred as its LSF-Model. For a finite-length sequence, one can obtain multiple LSF-Models by varying the model parameters. The LSF-Model, in time domain can be used to synthesize any arbitrary causal finite-length sequence in terms of its characteristic AH-sequences. In the frequency domain, the LSF-Model can be used to obtain the spectral samples of the sequence as a linear combination of spectra of its characteristic AH-sequences. We also summarize the utility of the LSF-Model in practical discrete signal processing systems.

Comparison of elasticity and shell-theory solutions for finite circular cylindrical shells

A finite circular cylindrical shell subjected to a band of uniform pressure on its outer rim was investigated, using three-dimensional elasticity theory and the classical shell theories of Timoshenko (or Donnell) and Flügge. Detailed comparison of the resulting stresses and displacements was carried out for shells with ratios of inner to outer shell radii equal to 0.80, 0.85, 0.90 and 0.93 and for ratios of outer shell diameter to length of the shell equal to 0.5, 1 and 2. The ratio of band width to length of the shell was 0.2 and Poisson's ratio used was equal to 0.3. An Elliot 803 digital computer was used for numerical computations.

Influence of finite size effect on magnetic and magnetotransport properties of La0.5Sr0.5CoO3 thin films

We present a comprehensive study of the thickness dependent structural, magnetic and magnetotransport properties of oriented La0.5Sr0.5CoO3 thin films grown on LaAlO3 by Pulsed Laser Deposition. We observe that these films undergo a reduction in Curie temperature (T-c) with a decrease in film thickness, and it is found to be primarily caused by the finite size effect since the finite scaling law [T-c(infinity) T-c(t)/T-c(infinity) = (c/t)lambda holds good over the studied thickness range. We rule out the contribution from the strain induced suppression of Curie temperature with decreasing film thickness since all the films exhibit a constant out of plane tensile strain (0.5%) irrespective of their varying thickness. However, we observe that the coercivity of the films is an order of magnitude higher than that of the bulk due to the tensile strain. In addition, we also observe an increase in the magneto resistance peak and a decrease in coercivity and electrical resistivity with an increase in film thickness. (C) 2010 Elsevier Ltd. All rights reserved.

A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Topology design of three-dimensional structures using hybrid finite elements

Conventional three-dimensional isoparametric elements are susceptible to problems of locking when used to model plate/shell geometries or when the meshes are distorted etc. Hybrid elements that are based on a two-field variational formulation are immune to most of these problems, and hence can be used to efficiently model both "chunky" three-dimensional and plate/shell type structures. Thus, only one type of element can be used to model "all" types of structures, and also allows us to use a standard dual algorithm for carrying out the topology optimization of the structure. We also address the issue of manufacturability of the designs.

Parallelization of an unstructured data based cell centre finite volume code, HIFUN-3D

This work describes the parallelization of High Resolution flow solver on unstructured meshes, HIFUN-3D, an unstructured data based finite volume solver for 3-D Euler equations. For mesh partitioning, we use METIS, a software based on multilevel graph partitioning. The unstructured graph used for partitioning is associated with weights both on its vertices and edges. The data residing on every processor is split into four layers. Such a novel procedure of handling data helps in maintaining the effectiveness of the serial code. The communication of data across the processors is achieved by explicit message passing using the standard blocking mode feature of Message Passing Interface (MPI). The parallel code is tested on PACE++128 available in CFD Center

A finite element analysis of quasistatic crack growth in a pressure sensitive constrained ductile layer

Polymeric adhesive layers are employed for bonding two components in a wide variety of technological applications, It has been observed that, unlike in metals, the yield behavior of polymers is affected by the state of hydrostatic stress. In this work, the effect of pressure sensitivity of yielding and layer thickness on quasistatic interfacial crack growth in a ductile adhesive layer is investigated. To this end, finite deformation, finite element analyses of a cracked sandwiched layer are carried out under plane strain, small-scale yielding conditions for a wide range of mode mixities. The Drucker-Prager constitutive equations are employed to represent the behavior of the layer. Crack propagation is simulated through a cohesive zone model, in which the interface is assumed to follow a prescribed traction-separation law. The results show that for a given mode mixity, the steady state Fracture toughness [K](ss) is enhanced as the degree of pressure sensitivity increases. Further, for a given level of pressure sensitivity, [K](ss) increases steeply as mode Il loading is approached. (C) 2000 Elsevier Science Ltd. All rights reserved.

Bearing capacity of interfering multiple strip footings by using lower bound finite elements limit analysis

The ultimate bearing capacity of a number of multiple strip footings, identically spaced and equally loaded to failure at the same time,is computed by using the lower bound limit analysis in combination with finite elements. The efficiency factor due to the component of soil unit weight, is computed with respect to changes in the clear spacing (xi(gamma)) between the footings. It is noted that the failure load for a footing in the group becomes always greater than that of a single isolated footing. The values of xi(gamma) for the smooth footings are found to be always lower than the rough footings. The values ofxi(gamma) are found to increase continuously with a decrease in the spacing between footings. As compared to the available theoretical and experimental results reported in literature, the present analysis provides generally a little lower values of xi(gamma). (C) 2010 Elsevier Ltd. All rights reserved.

Prediction of breakout noise from an elliptical duct of finite length

This paper describes a predictive model for breakout noise from an elliptical duct or shell of finite length. The transmission mechanism is essentially that of ``mode coupling'', whereby higher structural modes in the duct walls get excited because of non-circularity of the wall. Effect of geometry has been taken care of by evaluating Fourier coefficients of the radius of curvature. The noise radiated from the duct walls is represented by that from a finite vibrating length of a semi infinite cylinder in a free field. Emphasis is on understanding the physics of the problem as well as analytical modeling. The analytical model is validated with 3-D FEM. Effects of the ovality, curvature, and axial terminations of the duct have been demonstrated. (C) 2010 Institute of Noise Control Engineering.