An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Nonlinear viscoelasticity of freestanding and polymer-anchored vertically aligned carbon nanotube foams

Vertical arrays of carbon nanotubes (VACNTs) show unique mechanical behavior in compression, with a highly nonlinear response similar to that of open cell foams and the ability to recover large deformations. Here, we study the viscoelastic response of both freestanding VACNT arrays and sandwich structures composed of a VACNT array partially embedded between two layers of poly(dimethylsiloxane) (PDMS) and bucky paper. The VACNTs tested are similar to 2 mm thick foams grown via an injection chemical vapor deposition method. Both freestanding and sandwich structures exhibit a time-dependent behavior under compression. A power-law function of time is used to describe the main features observed in creep and stress-relaxation tests. The power-law exponents show nonlinear viscoelastic behavior in which the rate of creep is dependent upon the stress level and the rate of stress relaxation is dependent upon the strain level. The results show a marginal effect of the thin PDMS/bucky paper layers on the viscoelastic responses. At high strain levels (epsilon - 0.8), the peak stress for the anchored CNTs reaches similar to 45 MPa, whereas it is only similar to 15MPa for freestanding CNTs, suggesting a large effect of PDMS on the structural response of the sandwich structures. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.3699184]

On the solution of bivariate population balance equations for aggregation: Pivotwise expansion of solution domain

The solution of a bivariate population balance equation (PBE) for aggregation of particles necessitates a large 2-d domain to be covered. A correspondingly large number of discretized equations for particle populations on pivots (representative sizes for bins) are solved, although at the end only a relatively small number of pivots are found to participate in the evolution process. In the present work, we initiate solution of the governing PBE on a small set of pivots that can represent the initial size distribution. New pivots are added to expand the computational domain in directions in which the evolving size distribution advances. A self-sufficient set of rules is developed to automate the addition of pivots, taken from an underlying X-grid formed by intersection of the lines of constant composition and constant particle mass. In order to test the robustness of the rule-set, simulations carried out with pivotwise expansion of X-grid are compared with those obtained using sufficiently large fixed X-grids for a number of composition independent and composition dependent aggregation kernels and initial conditions. The two techniques lead to identical predictions, with the former requiring only a fraction of the computational effort. The rule-set automatically reduces aggregation of particles of same composition to a 1-d problem. A midway change in the direction of expansion of domain, effected by the addition of particles of different mean composition, is captured correctly by the rule-set. The evolving shape of a computational domain carries with it the signature of the aggregation process, which can be insightful in complex and time dependent aggregation conditions. (c) 2012 Elsevier Ltd. All rights reserved.

The force distribution function of an oscillator model of polymer stretching at constant velocity

Recent simulations of the stretching of tethered biopolymers at a constant speed v (Ponmurugan and Vemparala, 2011 Phys. Rev. E 84 060101(R)) have suggested that for any time t, the distribution of the fluctuating forces f responsible for chain deformation is governed by a relation of the form P(+ f)/ P(- f) = expgamma f], gamma being a coefficient that is solely a function of v and the temperature T. This result, which is reminiscent of the fluctuation theorems applicable to stochastic trajectories involving thermodynamic variables, is derived in this paper from an analytical calculation based on a generalization of Mazonka and Jarzynski's classic model of dragged particle dynamics Mazonka and Jarzynski, 1999 arXiv:cond-\textbackslashmat/9912121v1]. However, the analytical calculations suggest that the result holds only if t >> 1 and the force fluctuations are driven by white rather than colored noise; they further suggest that the coefficient gamma in the purported theorem varies not as v(0.15)T-(0.7), as indicated by the simulations, but as vT(-1).

Synthesis of Cholic Acid Oligomer-Taurine Conjugates: A Study of Their Aggregation and Cholesterol Solubilization

Taurine conjugates of two cholic acid derived oligomers with different spacers between the cholic acid units were synthesized. These molecules self-assemble in aqueous media. The critical micelle concentration (CMC) values were measured by using fluorescence spectroscopic analysis and the aggregates were characterized by dynamic light scattering and electron microscopy. The cooperativity of the cholic acid units in these tetramers to solubilize cholesterol was investigated. The ability of these molecules to act as nanocarriers for liphophilic dyes was also studied.

Enhanced J-protein interaction and compromised protein stability of mtHsp70 variants lead to mitochondrial dysfunction in Parkinsons disease

Parkinsons disease (PD) is the second most prevalent progressive neurological disorder commonly associated with impaired mitochondrial function in dopaminergic neurons. Although familial PD is multifactorial in nature, a recent genetic screen involving PD patients identified two mitochondrial Hsp70 variants (P509S and R126W) that are suggested in PD pathogenesis. However, molecular mechanisms underlying how mtHsp70 PD variants are centrally involved in PD progression is totally elusive. In this article, we provide mechanistic insights into the mitochondrial dysfunction associated with human mtHsp70 PD variants. Biochemically, the R126W variant showed severely compromised protein stability and was found highly susceptible to aggregation at physiological conditions. Strikingly, on the other hand, the P509S variant exhibits significantly enhanced interaction with J-protein cochaperones involved in folding and import machinery, thus altering the overall regulation of chaperone-mediated folding cycle and protein homeostasis. To assess the impact of mtHsp70 PD mutations at the cellular level, we developed yeast as a model system by making analogous mutations in Ssc1 ortholog. Interestingly, PD mutations in yeast (R103W and P486S) exhibit multiple in vivo phenotypes, which are associated with omitochondrial dysfunction', including compromised growth, impairment in protein translocation, reduced functional mitochondrial mass, mitochondrial DNA loss, respiratory incompetency and increased susceptibility to oxidative stress. In addition to that, R103W protein is prone to aggregate in vivo due to reduced stability, whereas P486S showed enhanced interaction with J-proteins, thus remarkably recapitulating the cellular defects that are observed in human PD variants. Taken together, our findings provide evidence in favor of direct involvement of mtHsp70 as a susceptibility factor in PD.

MS_RHII-RSD, a Dual-Function RNase HII-(p)ppGpp Synthetase from Mycobacterium smegmatis

In the noninfectious soil saprophyte Mycobacterium smegmatis, intracellular levels of the stress alarmones guanosine tetraphosphate and guanosine pentaphosphate, together termed (p)ppGpp, are regulated by the enzyme Rel(Msm). This enzyme consists of a single, bifunctional polypeptide chain that is capable of both synthesizing and hydrolyzing (p)ppGpp. The rel(Msm), knockout strain of M. smegmatis (Delta rel(Msm)) is expected to show a (p)ppGpp null (p)ppGpp(0)] phenotype. Contrary to this expectation, the strain is capable of synthesizing (p)ppGpp in vivo. In this study, we identify and functionally characterize the open reading frame (ORF), MSMEG_5849, that encodes a second functional (p)ppGpp synthetase in M. smegmatis. In addition to (p)ppGpp synthesis, the 567-amino-acid-long protein encoded by this gene is capable of hydrolyzing RNA(.)DNA hybrids and bears similarity to the conventional RNase HII enzymes. We have classified this protein as actRel(Msm) in accordance with the recent nomenclature proposed and have named it MS_RHII-RSD, indicating the two enzymatic activities present RHII, RNase HII domain, originally identified as (d) under bar omain of (u) under bar nknown (f) under bar unction 429 (DUF429), and RSD, RelA_SpoT nucleotidyl transferase domain, the SYNTH domain responsible for (p)ppGpp synthesis activity]. MS_RHII-RSD is expressed and is constitutively active in vivo and behaves like a monofunctional (p)ppGpp synthetase in vitro. The occurrence of the RNase HII and (p)ppGpp synthetase domains together on the same polypeptide chain is suggestive of an in vivo role for this novel protein as a link connecting the essential life processes of DNA replication, repair, and transcription to the highly conserved stress survival pathway, the stringent response.

PocketAnnotate: towards site-based function annotation

A computational pipeline PocketAnnotate for functional annotation of proteins at the level of binding sites has been proposed in this study. The pipeline integrates three in-house algorithms for site-based function annotation: PocketDepth, for prediction of binding sites in protein structures; PocketMatch, for rapid comparison of binding sites and PocketAlign, to obtain detailed alignment between pair of binding sites. A novel scheme has been developed to rapidly generate a database of non-redundant binding sites. For a given input protein structure, putative ligand-binding sites are identified, matched in real time against the database and the query substructure aligned with the promising hits, to obtain a set of possible ligands that the given protein could bind to. The input can be either whole protein structures or merely the substructures corresponding to possible binding sites. Structure-based function annotation at the level of binding sites thus achieved could prove very useful for cases where no obvious functional inference can be obtained based purely on sequence or fold-level analyses. An attempt has also been made to analyse proteins of no known function from Protein Data Bank. PocketAnnotate would be a valuable tool for the scientific community and contribute towards structure-based functional inference. The web server can be freely accessed at http://proline.biochem.iisc.ernet.in/pocketannotate/.

Relation between the work function and Young's modulus of RhSi and estimate of Schottky-barrier height at RhSi/Si interface: An ab-initio study

Density-functional calculations are performed to explore the relationship between the work function and Young's modulus of RhSi, and to estimate the p-Schottky-barrier height (SBH) at the Si/RhSi(010) interface. It is shown that the Young's modulus and the workfunction of RhSi satisfy the generic sextic relation, proposed recently for elemental metals. The calculated p-SBH at the Si/RhSi interface is found to differ only by 0.04 eV in opposite limits, viz., no-pinning and strong pinning. We find that the p-SBH is reduced as much as by 0.28 eV due to vacancies at the interface. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4761994]

Unusual Salt-Induced Color Modulation through Aggregation-Induced Emission Switching of a Bis-cationic Phenylenedivinylene-Based pi Hydrogelator

The synthesis, hydrogelation, and aggregation-induced emission switching of the phenylenedivinylene bis-N-octyl pyridinium salt is described. Hydrogelation occurs as a consequence of pi-stacking, van der Waals, and electrostatic interactions that lead to a high gel melting temperature and significant mechanical properties at a very low weight percentage of the gelator. A morphology transition from fiber-to-coil-to-tube was observed depending on the concentration of the gelator. Variation in the added salt type, salt concentrations, or temperature profoundly influenced the order of aggregation of the gelator molecules in aqueous solution. Formation of a novel chromophore assembly in this way leads to an aggregation-induced switch of the emission colors. The emission color switches from sky blue to white to orange depending upon the extent of aggregation through mere addition of external inorganic salts. Remarkably, the salt effect on the assembly of such cationic phenylenedivinylenes in water follow the behavior predicted from the well-known Hofmeister effects. Mechanistic insights for these aggregation processes were obtained through the counterion exchange studies. The aggregation-induced emission switching that leads to a room-temperature white-light emission from a single chromophore in a single solvent (water) is highly promising for optoelectronic applications.

Perturbative expansion of the QCD Adler function improved by renormalization-group summation and analytic continuation in the Borel plane

We examine the large-order behavior of a recently proposed renormalization-group-improved expansion of the Adler function in perturbative QCD, which sums in an analytically closed form the leading logarithms accessible from renormalization-group invariance. The expansion is first written as an effective series in powers of the one-loop coupling, and its leading singularities in the Borel plane are shown to be identical to those of the standard ``contour-improved'' expansion. Applying the technique of conformal mappings for the analytic continuation in the Borel plane, we define a class of improved expansions, which implement both the renormalization-group invariance and the knowledge about the large-order behavior of the series. Detailed numerical studies of specific models for the Adler function indicate that the new expansions have remarkable convergence properties up to high orders. Using these expansions for the determination of the strong coupling from the hadronic width of the tau lepton we obtain, with a conservative estimate of the uncertainty due to the nonperturbative corrections, alpha(s)(M-tau(2)) = 0.3189(-0.0151)(+0.0173), which translates to alpha(s)(M-Z(2)) = 0.1184(-0.0018)(+0.0021). DOI: 10.1103/PhysRevD.87.014008

Excellent chirality transcription in two-component photochromic organogels assembled through J-aggregation

Organogels made of pyridine-end oligo-p-phenylenevinylenes with tartaric acid exhibit remarkable J-aggregation induced red-shifts (Dk = 55 nm) and notable chirality transcription. Induction of liquid-crystalline behavior is also tuned in the supramolecular assembly.

Spectral moment sum rules for the retarded Green's function and self-energy of the inhomogeneous Bose-Hubbard model in equilibrium and nonequilibrium

We derive exact expressions for the zeroth and the first three spectral moment sum rules for the retarded Green's function and for the zeroth and the first spectral moment sum rules for the retarded self-energy of the inhomogeneous Bose-Hubbard model in nonequilibrium, when the local on-site repulsion and the chemical potential are time-dependent, and in the presence of an external time-dependent electromagnetic field. We also evaluate these expressions for the homogeneous case in equilibrium, where all time dependence and external fields vanish. Unlike similar sum rules for the Fermi-Hubbard model, in the Bose-Hubbard model case, the sum rules often depend on expectation values that cannot be determined simply from parameters in the Hamiltonian like the interaction strength and chemical potential but require knowledge of equal-time many-body expectation values from some other source. We show how one can approximately evaluate these expectation values for the Mott-insulating phase in a systematic strong-coupling expansion in powers of the hopping divided by the interaction. We compare the exact moment relations to the calculated moments of spectral functions determined from a variety of different numerical approximations and use them to benchmark their accuracy. DOI: 10.1103/PhysRevA.87.013628

Toll-Like Receptors 7, 8, and 9 Expression and Function in Primary Human Cervical Cancer Langerhans Cells Evidence of Anergy

Objective: Human papillomavirus oncoproteins E6 and E7 down modulate Toll-like receptor (TLR) 9 expression in infected keratinocytes. We explored the status of expression and function of TLR7, TLR8, and TLR9 in primary human Langerhans cells (LCs) isolated from cervical tumors. Methodology: Single-cell suspensions were made from fresh tissues of squamous cell carcinoma (International Federation of Gynecology and Obstetrics stage IB2); myeloid dendritic cells were purified using CD1c magnetic activated cell separation kits. Langerhans cells were further flow sorted into CD1a(+)CD207(+) cells. Acute monocytic leukemia cell line THP-1-derived LCs (moLCs) formed the controls. mRNA from flow-sorted LCs was reverse transcribed to cDNA and TLR7, TLR8, and TLR9 amplified. Monocyte-derived Langerhans cells and cervical tumor LCs were stimulated with TLR7, TLR8, and TLR9 ligands. Culture supernatants were assayed for interleukin (IL) 1 beta, IL-6, IL-10, IL-12p70, interferon (IFN) alpha, interferon gamma, and tumor necrosis factor (TNF) alpha by Luminex multiplex bead array. Human papillomavirus was genotyped. Results: We have for the first time demonstrated that the acute monocytic leukemia cell line THP-1 can be differentiated into LCs in vitro. Although these moLCs. expressed all the 3 TLRs, tumor LCs expressed TLR7 and TLR8, but uniformly lacked TLR9. Also, moLCs secreted IL-6, IL-1 beta, and tumor necrosis factor alpha to TLR8 ligand and interferon alpha in response to TLR9 ligand; in contrast, tumor LCs did not express any cytokine to any of the 3 TLR ligands. Human papillomavirus type 16 was one of the common human papillomavirus types in all cases. Conclusions: Cervical tumor LCs lacked TLR9 expression and were functionally anergic to all the 3: TLR7, TLR8, and TLR9 ligands, which may play a crucial role in immune tolerance. The exact location of block(s) in TLR7 and TLR8 signaling needs to be investigated, which would have important immunotherapeutic implications.

Implications of program phase behavior on timing analysis

Knowledge about program worst case execution time (WCET) is essential in validating real-time systems and helps in effective scheduling. One popular approach used in industry is to measure execution time of program components on the target architecture and combine them using static analysis of the program. Measurements need to be taken in the least intrusive way in order to avoid affecting accuracy of estimated WCET. Several programs exhibit phase behavior, wherein program dynamic execution is observed to be composed of phases. Each phase being distinct from the other, exhibits homogeneous behavior with respect to cycles per instruction (CPI), data cache misses etc. In this paper, we show that phase behavior has important implications on timing analysis. We make use of the homogeneity of a phase to reduce instrumentation overhead at the same time ensuring that accuracy of WCET is not largely affected. We propose a model for estimating WCET using static worst case instruction counts of individual phases and a function of measured average CPI. We describe a WCET analyzer built on this model which targets two different architectures. The WCET analyzer is observed to give safe estimates for most benchmarks considered in this paper. The tightness of the WCET estimates are observed to be improved for most benchmarks compared to Chronos, a well known static WCET analyzer.