An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

The force distribution function of an oscillator model of polymer stretching at constant velocity

Recent simulations of the stretching of tethered biopolymers at a constant speed v (Ponmurugan and Vemparala, 2011 Phys. Rev. E 84 060101(R)) have suggested that for any time t, the distribution of the fluctuating forces f responsible for chain deformation is governed by a relation of the form P(+ f)/ P(- f) = expgamma f], gamma being a coefficient that is solely a function of v and the temperature T. This result, which is reminiscent of the fluctuation theorems applicable to stochastic trajectories involving thermodynamic variables, is derived in this paper from an analytical calculation based on a generalization of Mazonka and Jarzynski's classic model of dragged particle dynamics Mazonka and Jarzynski, 1999 arXiv:cond-\textbackslashmat/9912121v1]. However, the analytical calculations suggest that the result holds only if t >> 1 and the force fluctuations are driven by white rather than colored noise; they further suggest that the coefficient gamma in the purported theorem varies not as v(0.15)T-(0.7), as indicated by the simulations, but as vT(-1).

Effect of addition of aluminum on the evolution of microstructure in HITPERM class Fe44Co44Zr7B4Cu1 alloy

The paper reports the effect of the addition of small amount of Al on the microstructure and properties of HITPERM class rapidly solidified Fe44Co44Zr7B4Cu1 glassy alloy. Using three dimensional atom probe measurements we present evidence for the formation of Cu clusters on annealing in the metallic glass matrix of the Al containing alloy Fe43Co43Al2Zr7B4Cu1. Such clusters are otherwise absent in the parent alloy under similar conditions. The Cu clusters provides heterogeneous nucleation sites for the formation of bcc alpha'-FeCo phase leading to an increase in number density of this nanocrystalline phase and thereby enhancing the magnetic properties. (C) 2012 Elsevier B.V. All rights reserved.

Synthesis of benzofuran based 1,3,5-substituted pyrazole derivatives: As a new class of potent antioxidants and antimicrobials-A novel accost to amend biocompatibility

In search for a new antioxidant and antimicrobial agent with improved potency, we synthesized a series of benzofuran based 1,3,5-substituted pyrazole analogues (5a-l) in five step reaction. Initially, o-alkyl derivative of salicyaldehyde readily furnish corresponding 2-acetyl benzofuran 2 in good yield, on treatment with 1,8-diaza bicyclo5.4.0]undec-7-ene (DBU) in the presence of molecular sieves. Further, aldol condensation with vanillin, Claisen-Schmidt condensation reaction with hydrazine hydrate followed by coupling of substituted anilines afforded target compounds. The structures of newly synthesized compounds were confirmed by IR, H-1 NMR, C-13 NMR, mass, elemental analysis and further screened for their antioxidant and antimicrobial activities. Among the tested compounds 5d and 5f exhibited good antioxidant property with 50% inhibitory concentration higher than that of reference while compounds 5h and 5l exhibited good antimicrobial activity at concentration 1.0 and 0.5 mg/mL compared with standard, streptomycin and fluconazole respectively. (C) 2012 Elsevier Ltd. All rights reserved.

Local Structural Differences in Homologous Proteins: Specificities in Different SCOP Classes

The constant increase in the number of solved protein structures is of great help in understanding the basic principles behind protein folding and evolution. 3-D structural knowledge is valuable in designing and developing methods for comparison, modelling and prediction of protein structures. These approaches for structure analysis can be directly implicated in studying protein function and for drug design. The backbone of a protein structure favours certain local conformations which include alpha-helices, beta-strands and turns. Libraries of limited number of local conformations (Structural Alphabets) were developed in the past to obtain a useful categorization of backbone conformation. Protein Block (PB) is one such Structural Alphabet that gave a reasonable structure approximation of 0.42 angstrom. In this study, we use PB description of local structures to analyse conformations that are preferred sites for structural variations and insertions, among group of related folds. This knowledge can be utilized in improving tools for structure comparison that work by analysing local structure similarities. Conformational differences between homologous proteins are known to occur often in the regions comprising turns and loops. Interestingly, these differences are found to have specific preferences depending upon the structural classes of proteins. Such class-specific preferences are mainly seen in the all-beta class with changes involving short helical conformations and hairpin turns. A test carried out on a benchmark dataset also indicates that the use of knowledge on the class specific variations can improve the performance of a PB based structure comparison approach. The preference for the indel sites also seem to be confined to a few backbone conformations involving beta-turns and helix C-caps. These are mainly associated with short loops joining the regular secondary structures that mediate a reversal in the chain direction. Rare beta-turns of type I' and II' are also identified as preferred sites for insertions.

MS_RHII-RSD, a Dual-Function RNase HII-(p)ppGpp Synthetase from Mycobacterium smegmatis

In the noninfectious soil saprophyte Mycobacterium smegmatis, intracellular levels of the stress alarmones guanosine tetraphosphate and guanosine pentaphosphate, together termed (p)ppGpp, are regulated by the enzyme Rel(Msm). This enzyme consists of a single, bifunctional polypeptide chain that is capable of both synthesizing and hydrolyzing (p)ppGpp. The rel(Msm), knockout strain of M. smegmatis (Delta rel(Msm)) is expected to show a (p)ppGpp null (p)ppGpp(0)] phenotype. Contrary to this expectation, the strain is capable of synthesizing (p)ppGpp in vivo. In this study, we identify and functionally characterize the open reading frame (ORF), MSMEG_5849, that encodes a second functional (p)ppGpp synthetase in M. smegmatis. In addition to (p)ppGpp synthesis, the 567-amino-acid-long protein encoded by this gene is capable of hydrolyzing RNA(.)DNA hybrids and bears similarity to the conventional RNase HII enzymes. We have classified this protein as actRel(Msm) in accordance with the recent nomenclature proposed and have named it MS_RHII-RSD, indicating the two enzymatic activities present RHII, RNase HII domain, originally identified as (d) under bar omain of (u) under bar nknown (f) under bar unction 429 (DUF429), and RSD, RelA_SpoT nucleotidyl transferase domain, the SYNTH domain responsible for (p)ppGpp synthesis activity]. MS_RHII-RSD is expressed and is constitutively active in vivo and behaves like a monofunctional (p)ppGpp synthetase in vitro. The occurrence of the RNase HII and (p)ppGpp synthetase domains together on the same polypeptide chain is suggestive of an in vivo role for this novel protein as a link connecting the essential life processes of DNA replication, repair, and transcription to the highly conserved stress survival pathway, the stringent response.

PocketAnnotate: towards site-based function annotation

A computational pipeline PocketAnnotate for functional annotation of proteins at the level of binding sites has been proposed in this study. The pipeline integrates three in-house algorithms for site-based function annotation: PocketDepth, for prediction of binding sites in protein structures; PocketMatch, for rapid comparison of binding sites and PocketAlign, to obtain detailed alignment between pair of binding sites. A novel scheme has been developed to rapidly generate a database of non-redundant binding sites. For a given input protein structure, putative ligand-binding sites are identified, matched in real time against the database and the query substructure aligned with the promising hits, to obtain a set of possible ligands that the given protein could bind to. The input can be either whole protein structures or merely the substructures corresponding to possible binding sites. Structure-based function annotation at the level of binding sites thus achieved could prove very useful for cases where no obvious functional inference can be obtained based purely on sequence or fold-level analyses. An attempt has also been made to analyse proteins of no known function from Protein Data Bank. PocketAnnotate would be a valuable tool for the scientific community and contribute towards structure-based functional inference. The web server can be freely accessed at http://proline.biochem.iisc.ernet.in/pocketannotate/.

Application of a Class of Nano fluids to improve the Loadability of Power Transformers

It is generally known that addition of conducting or insulating particles to mineral transformer oil, lowers its breakdown strength, E-d. However, if the particulates are of molecular dimensions, or nanoparticles, (NPs), as they are called, the breakdown strength is seen to increase considerably. Recent experiments by the authors on oil cooled power equipment such as transformers showed that, nanofluids comprising NPs of selected oxides of iron, such as Fe(3)o(4), called magnetite, added to transformer oil increased the breakdown voltage of the virgin oil and more importantly a remarkable enhancement in the thermal conductivity and the viscosity and hence an increased loadability of the transformer for a given top oil temperature (TOT).

Nonuniform random geometric graphs with location-dependent radii

We propose a distribution-free approach to the study of random geometric graphs. The distribution of vertices follows a Poisson point process with intensity function n f(center dot), where n is an element of N, and f is a probability density function on R-d. A vertex located at x connects via directed edges to other vertices that are within a cut-off distance r(n)(x). We prove strong law results for (i) the critical cut-off function so that almost surely, the graph does not contain any node with out-degree zero for sufficiently large n and (ii) the maximum and minimum vertex degrees. We also provide a characterization of the cut-off function for which the number of nodes with out-degree zero converges in distribution to a Poisson random variable. We illustrate this result for a class of densities with compact support that have at most polynomial rates of decay to zero. Finally, we state a sufficient condition for an enhanced version of the above graph to be almost surely connected eventually.

Relation between the work function and Young's modulus of RhSi and estimate of Schottky-barrier height at RhSi/Si interface: An ab-initio study

Density-functional calculations are performed to explore the relationship between the work function and Young's modulus of RhSi, and to estimate the p-Schottky-barrier height (SBH) at the Si/RhSi(010) interface. It is shown that the Young's modulus and the workfunction of RhSi satisfy the generic sextic relation, proposed recently for elemental metals. The calculated p-SBH at the Si/RhSi interface is found to differ only by 0.04 eV in opposite limits, viz., no-pinning and strong pinning. We find that the p-SBH is reduced as much as by 0.28 eV due to vacancies at the interface. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4761994]

Serodiagnosis of Tuberculosis in Asian Elephants (Elephas maximus) in Southern India: A Latent Class Analysis

Background: Mycobacterium tuberculosis, a causative agent of chronic tuberculosis disease, is widespread among some animal species too. There is paucity of information on the distribution, prevalence and true disease status of tuberculosis in Asian elephants (Elephas maximus). The aim of this study was to estimate the sensitivity and specificity of serological tests to diagnose M. tuberculosis infection in captive elephants in southern India while simultaneously estimating sero-prevalence. Methodology/Principal Findings: Health assessment of 600 elephants was carried out and their sera screened with a commercially available rapid serum test. Trunk wash culture of select rapid serum test positive animals yielded no animal positive for M. tuberculosis isolation. Under Indian field conditions where the true disease status is unknown, we used a latent class model to estimate the diagnostic characteristics of an existing (rapid serum test) and new (four in-house ELISA) tests. One hundred and seventy nine sera were randomly selected for screening in the five tests. Diagnostic sensitivities of the four ELISAs were 91.3-97.6% (95% Credible Interval (CI): 74.8-99.9) and diagnostic specificity were 89.6-98.5% (95% CI: 79.4-99.9) based on the model we assumed. We estimate that 53.6% (95% CI: 44.6-62.8) of the samples tested were free from infection with M. tuberculosis and 15.9% (97.5% CI: 9.8 - to 24.0) tested positive on all five tests. Conclusions/Significance: Our results provide evidence for high prevalence of asymptomatic M. tuberculosis infection in Asian elephants in a captive Indian setting. Further validation of these tests would be important in formulating area-specific effective surveillance and control measures.

A UNIFIED APPROACH FOR OPTIMIZATION OF SNAKUSCULES AND OVUSCULES

Automated image segmentation techniques are useful tools in biological image analysis and are an essential step in tracking applications. Typically, snakes or active contours are used for segmentation and they evolve under the influence of certain internal and external forces. Recently, a new class of shape-specific active contours have been introduced, which are known as Snakuscules and Ovuscules. These contours are based on a pair of concentric circles and ellipses as the shape templates, and the optimization is carried out by maximizing a contrast function between the outer and inner templates. In this paper, we present a unified approach to the formulation and optimization of Snakuscules and Ovuscules by considering a specific form of affine transformations acting on a pair of concentric circles. We show how the parameters of the affine transformation may be optimized for, to generate either Snakuscules or Ovuscules. Our approach allows for a unified formulation and relies only on generic regularization terms and not shape-specific regularization functions. We show how the calculations of the partial derivatives may be made efficient thanks to the Green's theorem. Results on synthesized as well as real data are presented.

Spectral moment sum rules for the retarded Green's function and self-energy of the inhomogeneous Bose-Hubbard model in equilibrium and nonequilibrium

We derive exact expressions for the zeroth and the first three spectral moment sum rules for the retarded Green's function and for the zeroth and the first spectral moment sum rules for the retarded self-energy of the inhomogeneous Bose-Hubbard model in nonequilibrium, when the local on-site repulsion and the chemical potential are time-dependent, and in the presence of an external time-dependent electromagnetic field. We also evaluate these expressions for the homogeneous case in equilibrium, where all time dependence and external fields vanish. Unlike similar sum rules for the Fermi-Hubbard model, in the Bose-Hubbard model case, the sum rules often depend on expectation values that cannot be determined simply from parameters in the Hamiltonian like the interaction strength and chemical potential but require knowledge of equal-time many-body expectation values from some other source. We show how one can approximately evaluate these expectation values for the Mott-insulating phase in a systematic strong-coupling expansion in powers of the hopping divided by the interaction. We compare the exact moment relations to the calculated moments of spectral functions determined from a variety of different numerical approximations and use them to benchmark their accuracy. DOI: 10.1103/PhysRevA.87.013628

Toll-Like Receptors 7, 8, and 9 Expression and Function in Primary Human Cervical Cancer Langerhans Cells Evidence of Anergy

Objective: Human papillomavirus oncoproteins E6 and E7 down modulate Toll-like receptor (TLR) 9 expression in infected keratinocytes. We explored the status of expression and function of TLR7, TLR8, and TLR9 in primary human Langerhans cells (LCs) isolated from cervical tumors. Methodology: Single-cell suspensions were made from fresh tissues of squamous cell carcinoma (International Federation of Gynecology and Obstetrics stage IB2); myeloid dendritic cells were purified using CD1c magnetic activated cell separation kits. Langerhans cells were further flow sorted into CD1a(+)CD207(+) cells. Acute monocytic leukemia cell line THP-1-derived LCs (moLCs) formed the controls. mRNA from flow-sorted LCs was reverse transcribed to cDNA and TLR7, TLR8, and TLR9 amplified. Monocyte-derived Langerhans cells and cervical tumor LCs were stimulated with TLR7, TLR8, and TLR9 ligands. Culture supernatants were assayed for interleukin (IL) 1 beta, IL-6, IL-10, IL-12p70, interferon (IFN) alpha, interferon gamma, and tumor necrosis factor (TNF) alpha by Luminex multiplex bead array. Human papillomavirus was genotyped. Results: We have for the first time demonstrated that the acute monocytic leukemia cell line THP-1 can be differentiated into LCs in vitro. Although these moLCs. expressed all the 3 TLRs, tumor LCs expressed TLR7 and TLR8, but uniformly lacked TLR9. Also, moLCs secreted IL-6, IL-1 beta, and tumor necrosis factor alpha to TLR8 ligand and interferon alpha in response to TLR9 ligand; in contrast, tumor LCs did not express any cytokine to any of the 3 TLR ligands. Human papillomavirus type 16 was one of the common human papillomavirus types in all cases. Conclusions: Cervical tumor LCs lacked TLR9 expression and were functionally anergic to all the 3: TLR7, TLR8, and TLR9 ligands, which may play a crucial role in immune tolerance. The exact location of block(s) in TLR7 and TLR8 signaling needs to be investigated, which would have important immunotherapeutic implications.

Performance of a class of multi-robot deploy and search strategies based on centroidal voronoi configurations

This article considers a class of deploy and search strategies for multi-robot systems and evaluates their performance. The application framework used is deployment of a system of autonomous mobile robots equipped with required sensors in a search space to gather information. The lack of information about the search space is modelled as an uncertainty density distribution. The agents are deployed to maximise single-step search effectiveness. The centroidal Voronoi configuration, which achieves a locally optimal deployment, forms the basis for sequential deploy and search (SDS) and combined deploy and search (CDS) strategies. Completeness results are provided for both search strategies. The deployment strategy is analysed in the presence of constraints on robot speed and limit on sensor range for the convergence of trajectories with corresponding control laws responsible for the motion of robots. SDS and CDS strategies are compared with standard greedy and random search strategies on the basis of time taken to achieve reduction in the uncertainty density below a desired level. The simulation experiments reveal several important issues related to the dependence of the relative performances of the search strategies on parameters such as the number of robots, speed of robots and their sensor range limits.