Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

A study of symbolic processing and computational aspects in helicopter dynamics

Even research models of helicopter dynamics often lead to a large number of equations of motion with periodic coefficients; and Floquet theory is a widely used mathematical tool for dynamic analysis. Presently, three approaches are used in generating the equations of motion. These are (1) general-purpose symbolic processors such as REDUCE and MACSYMA, (2) a special-purpose symbolic processor, DEHIM (Dynamic Equations for Helicopter Interpretive Models), and (3) completely numerical approaches. In this paper, comparative aspects of the first two purely algebraic approaches are studied by applying REDUCE and DEHIM to the same set of problems. These problems range from a linear model with one degree of freedom to a mildly non-linear multi-bladed rotor model with several degrees of freedom. Further, computational issues in applying Floquet theory are also studied, which refer to (1) the equilibrium solution for periodic forced response together with the transition matrix for perturbations about that response and (2) a small number of eigenvalues and eigenvectors of the unsymmetric transition matrix. The study showed the following: (1) compared to REDUCE, DEHIM is far more portable and economical, but it is also less user-friendly, particularly during learning phases; (2) the problems of finding the periodic response and eigenvalues are well conditioned.

Automatic identification of modal damping from Floquet analysis

A practical method is proposed to identify the mode associated with the frequency part of the eigenvalue of the Floquet transition matrix (FTM). From the FTM eigenvector, which contains the states and their derivatives, the ratio of the derivative and the state corresponding to the largest component is computed. The method exploits the fact that the imaginary part of this (complex) ratio closely approximates the frequency of the mode. It also lends itself well to automation and has been tested over a large number of FTMs of order as high as 250.

Parallel computing concepts and methods for floquet analysis of helicopter trim and stability

Floquet analysis is widely used for small-order systems (say, order M < 100) to find trim results of control inputs and periodic responses, and stability results of damping levels and frequencies, Presently, however, it is practical neither for design applications nor for comprehensive analysis models that lead to large systems (M > 100); the run time on a sequential computer is simply prohibitive, Accordingly, a massively parallel Floquet analysis is developed with emphasis on large systems, and it is implemented on two SIMD or single-instruction, multiple-data computers with 4096 and 8192 processors, The focus of this development is a parallel shooting method with damped Newton iteration to generate trim results; the Floquet transition matrix (FTM) comes out as a byproduct, The eigenvalues and eigenvectors of the FTM are computed by a parallel QR method, and thereby stability results are generated, For illustration, flap and flap-lag stability of isolated rotors are treated by the parallel analysis and by a corresponding sequential analysis with the conventional shooting and QR methods; linear quasisteady airfoil aerodynamics and a finite-state three-dimensional wake model are used, Computational reliability is quantified by the condition numbers of the Jacobian matrices in Newton iteration, the condition numbers of the eigenvalues and the residual errors of the eigenpairs, and reliability figures are comparable in both the parallel and sequential analyses, Compared to the sequential analysis, the parallel analysis reduces the run time of large systems dramatically, and the reduction increases with increasing system order; this finding offers considerable promise for design and comprehensive-analysis applications.

Markov chains, R-trivial monoids and representation theory

We develop a general theory of Markov chains realizable as random walks on R-trivial monoids. It provides explicit and simple formulas for the eigenvalues of the transition matrix, for multiplicities of the eigenvalues via Mobius inversion along a lattice, a condition for diagonalizability of the transition matrix and some techniques for bounding the mixing time. In addition, we discuss several examples, such as Toom-Tsetlin models, an exchange walk for finite Coxeter groups, as well as examples previously studied by the authors, such as nonabelian sandpile models and the promotion Markov chain on posets. Many of these examples can be viewed as random walks on quotients of free tree monoids, a new class of monoids whose combinatorics we develop.

A finite variant of the Toom Model

We present results for a finite variant of the one-dimensional Toom model with closed boundaries. We show that the steady state distribution is not of product form, but is nonetheless simple. In particular, we give explicit formulas for the densities and some nearest neighbour correlation functions. We also give exact results for eigenvalues and multiplicities of the transition matrix using the theory of R-trivial monoids in joint work with A. Schilling, B. Steinberg and N. M. Thiery.

Size effect on the superconducting transition of embedded lead particles in an Al-Cu-V amorphous matrix

We have synthesized specimens of nanometric lead dispersion in a glassy Al-Cu-V matrix by rapid solidification of the corresponding melt. The microstructure has been designed to avoid superconducting percolation due to coupling of the neighboring particles by the proximity effect. Using these specimens, we have determined quantitatively the effect of size of the ultrafine lead particles on the superconducting transition. (C) 1999 American Institute of Physics. [S0003-6951(99)02037-9].

Low temperature electrical transport properties of carbon matrix containing iron nanoparticles

We present a comparative study of the low temperature electrical transport properties of the carbon matrix containing iron nanoparticles and the films. The conductivity of the nanoparticles located just below the metal-insulator transition exhibits metallic behavior with a logarithmic temperature dependence over a large temperature interval. The zero-field conductivity and the negative magnetoresistance, showing a characteristic upturn at liquid helium temperature, are consistently explained by incorporating the Kondo relation and the two dimensional electron-electron interaction. The films, in contrast, exhibit a crossover of the conductivity from power-law dependence at high temperatures to an activated hopping law dependence in the low temperature region. The transition is attributed to changes in the energy dependence of the density of states near the Fermi level. The observed magnetoresistance is discussed in terms of quantum interference effect on a three-dimensional variable range hopping mechanism.

Study of linear and nonlinear optical properties of dendrimers using density matrix renormalization group method

We have used the density matrix renormalization group (DMRG) method to study the linear and nonlinear optical responses of first generation nitrogen based dendrimers with donor acceptor groups. We have employed Pariser–Parr–Pople Hamiltonian to model the interacting pi electrons in these systems. Within the DMRG method we have used an innovative scheme to target excited states with large transition dipole to the ground state. This method reproduces exact optical gaps and polarization in systems where exact diagonalization of the Hamiltonian is possible. We have used a correction vector method which tacitly takes into account the contribution of all excited states, to obtain the ground state polarizibility, first hyperpolarizibility, and two photon absorption cross sections. We find that the lowest optical excitations as well as the lowest excited triplet states are localized. It is interesting to note that the first hyperpolarizibility saturates more rapidly with system size compared to linear polarizibility unlike that of linear polyenes.

Hanle-Zeeman redistribution matrix. I. Classical theory expressions in the laboratory frame

Polarized scattering in spectral lines is governed by a 4; 4 matrix that describes how the Stokes vector is scattered and redistributed in frequency and direction. Here we develop the theory for this redistribution matrix in the presence of magnetic fields of arbitrary strength and direction. This general magnetic field case is called the Hanle- Zeeman regime, since it covers both of the partially overlapping weak- and strong- field regimes in which the Hanle and Zeeman effects dominate the scattering polarization. In this general regime, the angle-frequency correlations that describe the so-called partial frequency redistribution (PRD) are intimately coupled to the polarization properties. We develop the theory for the PRD redistribution matrix in this general case and explore its detailed mathematical properties and symmetries for the case of a J = 0 -> 1 -> 0 scattering transition, which can be treated in terms of time-dependent classical oscillator theory. It is shown how the redistribution matrix can be expressed as a linear superposition of coherent and noncoherent parts, each of which contain the magnetic redistribution functions that resemble the well- known Hummer- type functions. We also show how the classical theory can be extended to treat atomic and molecular scattering transitions for any combinations of quantum numbers.

Modified density matrix renormalization group algorithm for the zigzag spin-1/2 chain with frustrated antiferromagnetic exchange: Comparison with field theory at large J(2)/J(1)

A modified density matrix renormalization group (DMRG) algorithm is applied to the zigzag spin-1/2 chain with frustrated antiferromagnetic exchange J(1) and J(2) between first and second neighbors. The modified algorithm yields accurate results up to J(2)/J(1) approximate to 4 for the magnetic gap Delta to the lowest triplet state, the amplitude B of the bond order wave phase, the wavelength lambda of the spiral phase, and the spin correlation length xi. The J(2)/J(1) dependences of Delta, B, lambda, and xi provide multiple comparisons to field theories of the zigzag chain. The twist angle of the spiral phase and the spin structure factor yield additional comparisons between DMRG and field theory. Attention is given to the numerical accuracy required to obtain exponentially small gaps or exponentially long correlations near a quantum phase transition.

External Bias Dependent Direct To Indirect Band Gap Transition in Graphene Nanoribbon

In this work, using self-consistent tight-binding calculations. for the first time, we show that a direct to indirect band gap transition is possible in an armchair graphene nanoribbon by the application of an external bias along the width of the ribbon, opening up the possibility of new device applications. With the help of the Dirac equation, we qualitatively explain this band gap transition using the asymmetry in the spatial distribution of the perturbation potential produced inside the nanoribbon by the external bias. This is followed by the verification of the band gap trends with a numerical technique using Magnus expansion of matrix exponentials. Finally, we show that the carrier effective masses possess tunable sharp characters in the vicinity of the band gap transition points.

Magnetic, electric, and dielectric properties of FeCo alloy nanoparticles dispersed in amorphous matrix

Nanocrystalline Fe53Co47 alloy was synthesized by a single-step transmetallation chemical method at room temperature. The Fe53Co47 alloy nanoparticles of 77 and 47 wt% were dispersed in silica matrix by the sol-gel process using tetraethyl orthosilcate. Structural studies reveal that the as-prepared alloy powders are in bcc phase and silica is in an amorphous state. The phase-transition temperature and Mossbauer spectra analysis of the Fe-Co alloy establishes the homogeneous alloy formation. A saturation magnetization of 218 emu/g was obtained for pure FeCo alloy at room temperature. Scanning electron microscopic analysis demonstrates the hollow-sphere morphology for FeCo alloy particles. Magnetic nanocomposite consisting of 47 wt% FeCo-silica shows enhanced thermal stability over the native FeCo alloy. Electrical and dielectric properties of 47 wt% FeCo-silica nanocomposites were investigated as a function of frequency and temperature. It was found that the dielectric constants and dielectric loss were stable throughout the measured temperature (310-373 K). Our results indicate that FeCo-silica nanocomposite is a promising candidate for high-frequency applications. (C) 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim

Large negative magnetoresistance and metal-insulator transition observed in MnO/C composite coatings grown on ceramic alumina by metalorganic chemical vapor deposition

MnO/C composite coatings were grown by the metalorganic chemical vapor deposition process on ceramic alumina in argon ambient. Characterization by various techniques confirms that these coatings are homogeneous composites comprising nanometer-sized MnO particles embedded in a matrix of nanometer-sized graphite. Components of the MnO/C composite coating crystalline disordered, but are electrically quite conductive. Resistance vs. temperature measurements show that coating resistance increases exponentially from a few hundred ohms at room temperature to a few megaohms at 30 K. Logarithmic plots of reduced activation energy vs. temperature show that the coating material undergoes a metal-insulator transition. The reduced activation energy exponent for the film under zero magnetic field was 2.1, which is unusually high, implying that conduction is suppressed at much faster rate than the Mott or the Efros-Shklovskii hopping mechanism. Magnetoconductance us. magnetic field plots obtained at various temperatures show a high magnetoconductance (similar to 28.8%) at 100 K, which is unusually large for a disordered system, wherein magnetoresistance is attributed typically to weak localization. A plausible explanation for the unusual behavior observed in the carbonaceous disordered composite material is proposed. (C) 2010 Elsevier Ltd. All rights reserved.

A study of microstructure and wear behaviour of TiB2/Al metal matrix composites

The present paper deals with the study of microstructure and wear characteristics of TiB2 reinforced aluminium metal matrix composites (MMCs). Matrix alloys with 5, 10 and 15% of TiB2 were made using stir casting technique. Effect of sliding velocity on the wear behaviour and tribo-chemistry of the worn surfaces of both matrix and composites sliding against a EN24 steel disc has been investigated under dry conditions. A pin-on-disc wear testing machine was used to find the wear rate, in which EN24 steel disc was used as the counter face, loads of 10-60N in steps of 10N and speeds of 100, 200, 300, 400 and 500 rpm were employed. The results showed that the wear rate was increased with an increase in load and sliding speed for both the materials. However, a lower wear rate was obtained for MMCs when compared to the matrix alloys. The wear transition from slight to severe was presented at the critical applied loads. The transition loads for the MMCs were much higher than that of the matrix alloy. The transition loads were increased with increase in TiB2 and the same was decreased with the increase of sliding speeds. The SEM and EDS analyses were undertaken to demonstrate the effect of TiB2 particles on the wear mechanism for each conditions.