An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Distribution of transverse chain fluctuations in harmonically confined semiflexible polymers

Two different experimental studies of polymer dynamics based on single-molecule fluorescence imaging have recently found evidence of heterogeneities in the widths of the putative tubes that surround filaments of F-actin during their motion in concentrated solution. In one J. Glaser, D. Chakraborty, K. Kroy, I. Lauter, M. Degawa, N. Kirchesner, B. Hoffmann, R. Merkel, and M. Giesen, Phys. Rev. Lett. 105, 037801 (2010)], the observations were explained in terms of the statistics of a worm-like chain confined to a potential determined self-consistently by a binary collision approximation, and in the other B. Wang, J. Guan, S. M. Anthony, S. C. Bae, K. S. Schweizer, and S. Granick, Phys. Rev. Lett. 104, 118301 (2010)], they were explained in terms of the scaling properties of a random fluid of thin rods. In this paper, we show, using an exact path integral calculation, that the distribution of the length-averaged transverse fluctuations of a harmonically confined weakly bendable rod (one possible realization of a semiflexible chain in a tube), is in good qualitative agreement with the experimental data, although it is qualitatively different in analytic structure from the earlier theoretical predictions. We also show that similar path integral techniques can be used to obtain an exact expression for the time correlation function of fluctuations in the tube cross section. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4712306]

The force distribution function of an oscillator model of polymer stretching at constant velocity

Recent simulations of the stretching of tethered biopolymers at a constant speed v (Ponmurugan and Vemparala, 2011 Phys. Rev. E 84 060101(R)) have suggested that for any time t, the distribution of the fluctuating forces f responsible for chain deformation is governed by a relation of the form P(+ f)/ P(- f) = expgamma f], gamma being a coefficient that is solely a function of v and the temperature T. This result, which is reminiscent of the fluctuation theorems applicable to stochastic trajectories involving thermodynamic variables, is derived in this paper from an analytical calculation based on a generalization of Mazonka and Jarzynski's classic model of dragged particle dynamics Mazonka and Jarzynski, 1999 arXiv:cond-\textbackslashmat/9912121v1]. However, the analytical calculations suggest that the result holds only if t >> 1 and the force fluctuations are driven by white rather than colored noise; they further suggest that the coefficient gamma in the purported theorem varies not as v(0.15)T-(0.7), as indicated by the simulations, but as vT(-1).

An invariant subspace-based approach to the random eigenvalue problem of systems with clustered spectrum

The repeated or closely spaced eigenvalues and corresponding eigenvectors of a matrix are usually very sensitive to a perturbation of the matrix, which makes capturing the behavior of these eigenpairs very difficult. Similar difficulty is encountered in solving the random eigenvalue problem when a matrix with random elements has a set of clustered eigenvalues in its mean. In addition, the methods to solve the random eigenvalue problem often differ in characterizing the problem, which leads to different interpretations of the solution. Thus, the solutions obtained from different methods become mathematically incomparable. These two issues, the difficulty of solving and the non-unique characterization, are addressed here. A different approach is used where instead of tracking a few individual eigenpairs, the corresponding invariant subspace is tracked. The spectral stochastic finite element method is used for analysis, where the polynomial chaos expansion is used to represent the random eigenvalues and eigenvectors. However, the main concept of tracking the invariant subspace remains mostly independent of any such representation. The approach is successfully implemented in response prediction of a system with repeated natural frequencies. It is found that tracking only an invariant subspace could be sufficient to build a modal-based reduced-order model of the system. Copyright (C) 2012 John Wiley & Sons, Ltd.

Carbon dioxide detection by boron nitride nanotubes

The effect of gas molecule adsorption is investigated on the density of states of (9,0) zigzag boron nitride nanotube within a random tight-binding Hamiltonian model. The Green function approach and coherent potential approximation have been implemented. The results show that the adsorption of carbon dioxide gas molecules by boron atoms only leads to a donor type semiconductor while the adsorption by nitrogen atoms only leads to an acceptor. Since the gas molecules are adsorbed by both boron and nitrogen atoms, a reduction of the band gap is found. In all cases, increasing the gas concentration causes an increase in the height of the peaks in the band gap. This is due to an increasing charge carrier concentration induced by adsorbed gas molecules.

MS_RHII-RSD, a Dual-Function RNase HII-(p)ppGpp Synthetase from Mycobacterium smegmatis

In the noninfectious soil saprophyte Mycobacterium smegmatis, intracellular levels of the stress alarmones guanosine tetraphosphate and guanosine pentaphosphate, together termed (p)ppGpp, are regulated by the enzyme Rel(Msm). This enzyme consists of a single, bifunctional polypeptide chain that is capable of both synthesizing and hydrolyzing (p)ppGpp. The rel(Msm), knockout strain of M. smegmatis (Delta rel(Msm)) is expected to show a (p)ppGpp null (p)ppGpp(0)] phenotype. Contrary to this expectation, the strain is capable of synthesizing (p)ppGpp in vivo. In this study, we identify and functionally characterize the open reading frame (ORF), MSMEG_5849, that encodes a second functional (p)ppGpp synthetase in M. smegmatis. In addition to (p)ppGpp synthesis, the 567-amino-acid-long protein encoded by this gene is capable of hydrolyzing RNA(.)DNA hybrids and bears similarity to the conventional RNase HII enzymes. We have classified this protein as actRel(Msm) in accordance with the recent nomenclature proposed and have named it MS_RHII-RSD, indicating the two enzymatic activities present RHII, RNase HII domain, originally identified as (d) under bar omain of (u) under bar nknown (f) under bar unction 429 (DUF429), and RSD, RelA_SpoT nucleotidyl transferase domain, the SYNTH domain responsible for (p)ppGpp synthesis activity]. MS_RHII-RSD is expressed and is constitutively active in vivo and behaves like a monofunctional (p)ppGpp synthetase in vitro. The occurrence of the RNase HII and (p)ppGpp synthetase domains together on the same polypeptide chain is suggestive of an in vivo role for this novel protein as a link connecting the essential life processes of DNA replication, repair, and transcription to the highly conserved stress survival pathway, the stringent response.

PocketAnnotate: towards site-based function annotation

A computational pipeline PocketAnnotate for functional annotation of proteins at the level of binding sites has been proposed in this study. The pipeline integrates three in-house algorithms for site-based function annotation: PocketDepth, for prediction of binding sites in protein structures; PocketMatch, for rapid comparison of binding sites and PocketAlign, to obtain detailed alignment between pair of binding sites. A novel scheme has been developed to rapidly generate a database of non-redundant binding sites. For a given input protein structure, putative ligand-binding sites are identified, matched in real time against the database and the query substructure aligned with the promising hits, to obtain a set of possible ligands that the given protein could bind to. The input can be either whole protein structures or merely the substructures corresponding to possible binding sites. Structure-based function annotation at the level of binding sites thus achieved could prove very useful for cases where no obvious functional inference can be obtained based purely on sequence or fold-level analyses. An attempt has also been made to analyse proteins of no known function from Protein Data Bank. PocketAnnotate would be a valuable tool for the scientific community and contribute towards structure-based functional inference. The web server can be freely accessed at http://proline.biochem.iisc.ernet.in/pocketannotate/.

Search on a fractal lattice using a quantum random walk

The spatial search problem on regular lattice structures in integer number of dimensions d >= 2 has been studied extensively, using both coined and coinless quantum walks. The relativistic Dirac operator has been a crucial ingredient in these studies. Here, we investigate the spatial search problem on fractals of noninteger dimensions. Although the Dirac operator cannot be defined on a fractal, we construct the quantum walk on a fractal using the flip-flop operator that incorporates a Klein-Gordon mode. We find that the scaling behavior of the spatial search is determined by the spectral (and not the fractal) dimension. Our numerical results have been obtained on the well-known Sierpinski gaskets in two and three dimensions.

Thermal degradation kinetics of poly(trimethylol propane triacrylate)/poly(hexane diol diacrylate) interpenetrating polymer network

Interpenetrating polymer networks (IPNs) of trimethylol propane triacrylate (TMPTA) and 1,6-hexane diol diacrylate (HDDA) at different weight ratios were synthesized. Temperature modulated differential scanning calorimetry (TMDSC) was used to determine whether the formation resulted in a copolymer or interpenetrating polymer network (IPN). These polymers are used as binders for microstereolithography (MSL) based ceramic microfabrication. The kinetics of thermal degradation of these polymers are important to optimize the debinding process for fabricating 3D shaped ceramic objects by MSL based rapid prototyping technique. Therefore, thermal and thermo-oxidative degradation of these IPNs have been studied by dynamic and isothermal thermogravimetry (TGA). Non-isothermal model-free kinetic methods have been adopted (isoconversional differential and KAS) to calculate the apparent activation energy (E a) as a function of conversion (α) in N 2 and air. The degradation of these polymers in N 2 atmosphere occurs via two mechanisms. Chain end scission plays a dominant role at lower temperature while the kinetics is governed by random chain scission at higher temperature. Oxidative degradation shows multiple degradation steps having higher activation energy than in N 2. Isothermal degradation was also carried out to predict the reaction model which is found to be decelerating. It was shown that the degradation of PTMPTA follows a contracting sphere reaction model in N 2. However, as the HDDA content increases in the IPNs, the degradation reaction follows Avrami-Erofeev model and diffusion governed mechanisms. The intermediate IPN compositions show both type of mechanism. Based on the above study, debinding strategy for MSL based microfabricated ceramic structure has been proposed. © 2012 Elsevier B.V.

Completely random nanoporous Cu4O3-CuO-C composite thin films for potential application as multiple channel photonic band gap based filter in the telecommunication wavelengths

In 2003, Babin et al. theoretically predicted (J. Appl. Phys. 94:4244, 2003) that fabrication of organic-inorganic hybrid materials would probably be required to implement structures with multiple photonic band gaps. In tune with their prediction, we report synthesis of such an inorganic-organic nanocomposite, comprising Cu4O3-CuO-C thin films that experimentally exhibit the highest (of any known material) number (as many as eleven) of photonic band gaps in the near infrared. On contrary to the report by Wang et al. (Appl. Phys. Lett. 84:1629, 2004) that photonic crystals with multiple stop gaps require highly correlated structural arrangement such as multilayers of variable thicknesses, we demonstrate experimental realization of multiple stop gaps in completely randomized structures comprising inorganic oxide nanocrystals (Cu4O3 and CuO) randomly embedded in a randomly porous carbonaceous matrix. We report one step synthesis of such nanostructured films through the metalorganic chemical vapor deposition technique using a single source metalorganic precursor, Cu-4(deaH)(dea)(oAc)(5) a <...aEuro parts per thousand(CH3)(2)CO. The films displaying multiple (4/9/11) photonic band gaps with equal transmission losses in the infrared are promising materials to find applications as multiple channel photonic band gap based filter for WDM technology.

Random copolymers consisting of dithienylcyclopentadienone, thiophene and benzothiadiazole for bulk heterojunction solar cells

Novel random copolymers containing dithienylcyclopentadienone, thiophene and benzothiadiazole were synthesized and photovoltaic properties of these materials were evaluated. Thermal, structural, optical and electrochemical characterization of the synthesized copolymers was carried out. These thermally stable copolymers are solution processable unlike the homopolymer. The absorption spectra indicated that with the incorporation of alkyl chains in the thiophene moiety, the onset of absorption increases and hence band gap decreases (1.47 eV to 1.41 eV). Bulk heterojunction solar cells were fabricated with the blend of copolymer and phenyl-C61-butyric acid methyl ester (PCBM) as the active material and device parameters were extracted. The copolymer consists of alkyl thiophene exhibit higher open circuit voltage than the copolymer consisting of thiophene moiety. (c) 2012 Elsevier B.V. All rights reserved.

Relation between the work function and Young's modulus of RhSi and estimate of Schottky-barrier height at RhSi/Si interface: An ab-initio study

Density-functional calculations are performed to explore the relationship between the work function and Young's modulus of RhSi, and to estimate the p-Schottky-barrier height (SBH) at the Si/RhSi(010) interface. It is shown that the Young's modulus and the workfunction of RhSi satisfy the generic sextic relation, proposed recently for elemental metals. The calculated p-SBH at the Si/RhSi interface is found to differ only by 0.04 eV in opposite limits, viz., no-pinning and strong pinning. We find that the p-SBH is reduced as much as by 0.28 eV due to vacancies at the interface. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4761994]

Perturbative expansion of the QCD Adler function improved by renormalization-group summation and analytic continuation in the Borel plane

We examine the large-order behavior of a recently proposed renormalization-group-improved expansion of the Adler function in perturbative QCD, which sums in an analytically closed form the leading logarithms accessible from renormalization-group invariance. The expansion is first written as an effective series in powers of the one-loop coupling, and its leading singularities in the Borel plane are shown to be identical to those of the standard ``contour-improved'' expansion. Applying the technique of conformal mappings for the analytic continuation in the Borel plane, we define a class of improved expansions, which implement both the renormalization-group invariance and the knowledge about the large-order behavior of the series. Detailed numerical studies of specific models for the Adler function indicate that the new expansions have remarkable convergence properties up to high orders. Using these expansions for the determination of the strong coupling from the hadronic width of the tau lepton we obtain, with a conservative estimate of the uncertainty due to the nonperturbative corrections, alpha(s)(M-tau(2)) = 0.3189(-0.0151)(+0.0173), which translates to alpha(s)(M-Z(2)) = 0.1184(-0.0018)(+0.0021). DOI: 10.1103/PhysRevD.87.014008

Spectral moment sum rules for the retarded Green's function and self-energy of the inhomogeneous Bose-Hubbard model in equilibrium and nonequilibrium

We derive exact expressions for the zeroth and the first three spectral moment sum rules for the retarded Green's function and for the zeroth and the first spectral moment sum rules for the retarded self-energy of the inhomogeneous Bose-Hubbard model in nonequilibrium, when the local on-site repulsion and the chemical potential are time-dependent, and in the presence of an external time-dependent electromagnetic field. We also evaluate these expressions for the homogeneous case in equilibrium, where all time dependence and external fields vanish. Unlike similar sum rules for the Fermi-Hubbard model, in the Bose-Hubbard model case, the sum rules often depend on expectation values that cannot be determined simply from parameters in the Hamiltonian like the interaction strength and chemical potential but require knowledge of equal-time many-body expectation values from some other source. We show how one can approximately evaluate these expectation values for the Mott-insulating phase in a systematic strong-coupling expansion in powers of the hopping divided by the interaction. We compare the exact moment relations to the calculated moments of spectral functions determined from a variety of different numerical approximations and use them to benchmark their accuracy. DOI: 10.1103/PhysRevA.87.013628

Toll-Like Receptors 7, 8, and 9 Expression and Function in Primary Human Cervical Cancer Langerhans Cells Evidence of Anergy

Objective: Human papillomavirus oncoproteins E6 and E7 down modulate Toll-like receptor (TLR) 9 expression in infected keratinocytes. We explored the status of expression and function of TLR7, TLR8, and TLR9 in primary human Langerhans cells (LCs) isolated from cervical tumors. Methodology: Single-cell suspensions were made from fresh tissues of squamous cell carcinoma (International Federation of Gynecology and Obstetrics stage IB2); myeloid dendritic cells were purified using CD1c magnetic activated cell separation kits. Langerhans cells were further flow sorted into CD1a(+)CD207(+) cells. Acute monocytic leukemia cell line THP-1-derived LCs (moLCs) formed the controls. mRNA from flow-sorted LCs was reverse transcribed to cDNA and TLR7, TLR8, and TLR9 amplified. Monocyte-derived Langerhans cells and cervical tumor LCs were stimulated with TLR7, TLR8, and TLR9 ligands. Culture supernatants were assayed for interleukin (IL) 1 beta, IL-6, IL-10, IL-12p70, interferon (IFN) alpha, interferon gamma, and tumor necrosis factor (TNF) alpha by Luminex multiplex bead array. Human papillomavirus was genotyped. Results: We have for the first time demonstrated that the acute monocytic leukemia cell line THP-1 can be differentiated into LCs in vitro. Although these moLCs. expressed all the 3 TLRs, tumor LCs expressed TLR7 and TLR8, but uniformly lacked TLR9. Also, moLCs secreted IL-6, IL-1 beta, and tumor necrosis factor alpha to TLR8 ligand and interferon alpha in response to TLR9 ligand; in contrast, tumor LCs did not express any cytokine to any of the 3 TLR ligands. Human papillomavirus type 16 was one of the common human papillomavirus types in all cases. Conclusions: Cervical tumor LCs lacked TLR9 expression and were functionally anergic to all the 3: TLR7, TLR8, and TLR9 ligands, which may play a crucial role in immune tolerance. The exact location of block(s) in TLR7 and TLR8 signaling needs to be investigated, which would have important immunotherapeutic implications.