Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

Size-dependent magnetic properties of iron carbide nanoparticles embedded in a carbon matrix

Here we report on the magnetic properties of iron carbide nanoparticles embedded in a carbon matrix. Granular distributions of nanoparticles in an inert matrix, of potential use in various applications, were prepared by pyrolysis of organic precursors using the thermally assisted chemical vapour deposition method. By varying the precursor concentration and preparation temperature, compositions with varying iron concentration and nanoparticle sizes were made. Powder x-ray diffraction, transmission electron microscopy and Mossbauer spectroscopy studies revealed the nanocrystalline iron carbide (Fe3C) presence in the partially graphitized matrix. The dependence of the magnetic properties on the particle size and temperature (10 K < T < 300 K) were studied using superconducting quantum interference device magnetometry. Based on the affect of surrounding carbon spins, the observed magnetic behaviour of the nanoparticle compositions, such as the temperature dependence of magnetization and coercivity, can be explained.

Modified density matrix renormalization group algorithm for the zigzag spin-1/2 chain with frustrated antiferromagnetic exchange: Comparison with field theory at large J(2)/J(1)

A modified density matrix renormalization group (DMRG) algorithm is applied to the zigzag spin-1/2 chain with frustrated antiferromagnetic exchange J(1) and J(2) between first and second neighbors. The modified algorithm yields accurate results up to J(2)/J(1) approximate to 4 for the magnetic gap Delta to the lowest triplet state, the amplitude B of the bond order wave phase, the wavelength lambda of the spiral phase, and the spin correlation length xi. The J(2)/J(1) dependences of Delta, B, lambda, and xi provide multiple comparisons to field theories of the zigzag chain. The twist angle of the spiral phase and the spin structure factor yield additional comparisons between DMRG and field theory. Attention is given to the numerical accuracy required to obtain exponentially small gaps or exponentially long correlations near a quantum phase transition.

Stochastic simulations of the origins and implications of long-tailed distributions in gene expression

Gene expression noise results in protein number distributions ranging from long-tailed to Gaussian. We show how long-tailed distributions arise from a stochastic model of the constituent chemical reactions and suggest that, in conjunction with cooperative switches, they lead to more sensitive selection of a subpopulation of cells with high protein number than is possible with Gaussian distributions. Single-cell-tracking experiments are presented to validate some of the assumptions of the stochastic simulations. We also examine the effect of DNA looping on the shape of protein distributions. We further show that when switches are incorporated in the regulation of a gene via a feedback loop, the distributions can become bimodal. This might explain the bimodal distribution of certain morphogens during early embryogenesis.

Optimal threshold policies for admission control in communication networks via discrete parameter stochastic approximation

The problem of admission control of packets in communication networks is studied in the continuous time queueing framework under different classes of service and delayed information feedback. We develop and use a variant of a simulation based two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm for finding an optimal feedback policy within the class of threshold type policies. Even though SPSA has originally been designed for continuous parameter optimization, its variant for the discrete parameter case is seen to work well. We give a proof of the hypothesis needed to show convergence of the algorithm on our setting along with a sketch of the convergence analysis. Extensive numerical experiments with the algorithm are illustrated for different parameter specifications. In particular, we study the effect of feedback delays on the system performance.

On the effect of matrix cracks in composite helicopter rotor blade

This paper investigates the feasibility of an on-line damage detection capability for helicopter main rotor blades made of composite material. Damage modeled in the composite is matrix cracking. A box-beam with stiffness properties similar to a hingeless rotor blade is designed using genetic algorithm for the typical [+/-theta(m)/90(n)](s) family of composites. The effect of matrix cracks is included in an analytical model of composite box-beam. An aeroelastic analysis of the helicopter rotor based on finite elements in space and time is used to study the effects of matrix cracking in the rotor blade in forward flight. For global fault detection, rotating frequencies, tip bending and torsion response, and blade root loads are studied. It is observed that the effect of matrix cracking on lag bending and elastic twist deflection at the blade tip and blade root yawing moment is significant and these parameters can be monitored for online health monitoring. For implementation of local fault detection technique, the effect on axial and shear strain, for matrix cracks in the whole blade as well as matrix cracks occurring locally is studied. It is observed that using strain measurement along the blade it is possible to locate the matrix cracks as well as to predict density of matrix cracks. (C) 2004 Elsevier Ltd. All rights reserved.

Hybrid moving block bootstrap for stochastic simulation of multi-site multi-season streamflows

The Hybrid approach introduced by the authors for at-site modeling of annual and periodic streamflows in earlier works is extended to simulate multi-site multi-season streamflows. It bears significance in integrated river basin planning studies. This hybrid model involves: (i) partial pre-whitening of standardized multi-season streamflows at each site using a parsimonious linear periodic model; (ii) contemporaneous resampling of the resulting residuals with an appropriate block size, using moving block bootstrap (non-parametric, NP) technique; and (iii) post-blackening the bootstrapped innovation series at each site, by adding the corresponding parametric model component for the site, to obtain generated streamflows at each of the sites. It gains significantly by effectively utilizing the merits of both parametric and NP models. It is able to reproduce various statistics, including the dependence relationships at both spatial and temporal levels without using any normalizing transformations and/or adjustment procedures. The potential of the hybrid model in reproducing a wide variety of statistics including the run characteristics, is demonstrated through an application for multi-site streamflow generation in the Upper Cauvery river basin, Southern India. (C) 2004 Elsevier B.V. All rights reserved.

Search for the Higgs boson produced in association with Z→ℓ+ℓ- using the matrix element method at CDF II

We present a search for associated production of the standard model (SM) Higgs boson and a $Z$ boson where the $Z$ boson decays to two leptons and the Higgs decays to a pair of $b$ quarks in $p\bar{p}$ collisions at the Fermilab Tevatron. We use event probabilities based on SM matrix elements to construct a likelihood function of the Higgs content of the data sample. In a CDF data sample corresponding to an integrated luminosity of 2.7 fb$^{-1}$ we see no evidence of a Higgs boson with a mass between 100 GeV$/c^2$ and 150 GeV$/c^2$. We set 95% confidence level (C.L.) upper limits on the cross-section for $ZH$ production as a function of the Higgs boson mass $m_H$; the limit is 8.2 times the SM prediction at $m_H = 115$ GeV$/c^2$.

Measurement of the WW+WZ production cross section using a matrix element technique in lepton+jets events

We present a measurement of the $WW+WZ$ production cross section observed in a final state consisting of an identified electron or muon, two jets, and missing transverse energy. The measurement is carried out in a data sample corresponding to up to 4.6~fb$^{-1}$ of integrated luminosity at $\sqrt{s} = 1.96$ TeV collected by the CDF II detector. Matrix element calculations are used to separate the diboson signal from the large backgrounds. The $WW+WZ$ cross section is measured to be $17.4\pm3.3$~pb, in agreement with standard model predictions. A fit to the dijet invariant mass spectrum yields a compatible cross section measurement.

Top Quark Mass Measurement in the lepton+jets Channel Using a Matrix Element Method and in situ Jet Energy Calibration

A precision measurement of the top quark mass m_t is obtained using a sample of ttbar events from ppbar collisions at the Fermilab Tevatron with the CDF II detector. Selected events require an electron or muon, large missing transverse energy, and exactly four high-energy jets, at least one of which is tagged as coming from a b quark. A likelihood is calculated using a matrix element method with quasi-Monte Carlo integration taking into account finite detector resolution and jet mass effects. The event likelihood is a function of m_t and a parameter DJES to calibrate the jet energy scale /in situ/. Using a total of 1087 events, a value of m_t = 173.0 +/- 1.2 GeV/c^2 is measured.

Top quark mass measurement in the lepton plus jets channel using a modified matrix element method

We report a measurement of the top quark mass, m_t, obtained from ppbar collisions at sqrt(s) = 1.96 TeV at the Fermilab Tevatron using the CDF II detector. We analyze a sample corresponding to an integrated luminosity of 1.9 fb^-1. We select events with an electron or muon, large missing transverse energy, and exactly four high-energy jets in the central region of the detector, at least one of which is tagged as coming from a b quark. We calculate a signal likelihood using a matrix element integration method, with effective propagators to take into account assumptions on event kinematics. Our event likelihood is a function of m_t and a parameter JES that determines /in situ/ the calibration of the jet energies. We use a neural network discriminant to distinguish signal from background events. We also apply a cut on the peak value of each event likelihood curve to reduce the contribution of background and badly reconstructed events. Using the 318 events that pass all selection criteria, we find m_t = 172.7 +/- 1.8 (stat. + JES) +/- 1.2 (syst.) GeV/c^2.

Top quark mass measurement in the tt̅ all hadronic channel using a matrix element technique in pp̅ collisions at √s=1.96 TeV

We present a measurement of the top quark mass in the all-hadronic channel (\tt $\to$ \bb$q_{1}\bar{q_{2}}q_{3}\bar{q_{4}}$) using 943 pb$^{-1}$ of \ppbar collisions at $\sqrt {s} = 1.96$ TeV collected at the CDF II detector at Fermilab (CDF). We apply the standard model production and decay matrix-element (ME) to $\ttbar$ candidate events. We calculate per-event probability densities according to the ME calculation and construct template models of signal and background. The scale of the jet energy is calibrated using additional templates formed with the invariant mass of pairs of jets. These templates form an overall likelihood function that depends on the top quark mass and on the jet energy scale (JES). We estimate both by maximizing this function. Given 72 observed events, we measure a top quark mass of 171.1 $\pm$ 3.7 (stat.+JES) $\pm$ 2.1 (syst.) GeV/$c^{2}$. The combined uncertainty on the top quark mass is 4.3 GeV/$c^{2}$.

An alternate form of the ramberg-osgood formula for matrix displacement analysis

A new stress-strain law, which is a three parameter representation of stress in terms of strain has been proposed for the matrix displacement analysis of structures made of non-hookean materials. This formula has been utilized to study three typical problems. These studies brought out the effectiveness and suitability of this law for matrix displacement analysis.