An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

The force distribution function of an oscillator model of polymer stretching at constant velocity

Recent simulations of the stretching of tethered biopolymers at a constant speed v (Ponmurugan and Vemparala, 2011 Phys. Rev. E 84 060101(R)) have suggested that for any time t, the distribution of the fluctuating forces f responsible for chain deformation is governed by a relation of the form P(+ f)/ P(- f) = expgamma f], gamma being a coefficient that is solely a function of v and the temperature T. This result, which is reminiscent of the fluctuation theorems applicable to stochastic trajectories involving thermodynamic variables, is derived in this paper from an analytical calculation based on a generalization of Mazonka and Jarzynski's classic model of dragged particle dynamics Mazonka and Jarzynski, 1999 arXiv:cond-\textbackslashmat/9912121v1]. However, the analytical calculations suggest that the result holds only if t >> 1 and the force fluctuations are driven by white rather than colored noise; they further suggest that the coefficient gamma in the purported theorem varies not as v(0.15)T-(0.7), as indicated by the simulations, but as vT(-1).

Electromechanical dynamics and optimization of pectoral fin-based ionic polymer-metal composite underwater propulsor

Ionic polymer-metal composites are soft artificial muscle-like bending actuators, which can work efficiently in wet environments such as water. Therefore, there is significant motivation for research on the development and design analysis of ionic polymer-metal composite based biomimetic underwater propulsion systems. Among aquatic animals, fishes are efficient swimmers with advantages such as high maneuverability, high cruising speed, noiseless propulsion, and efficient stabilization. Fish swimming mechanisms provide biomimetic inspiration for underwater propulsor design. Fish locomotion can be broadly classified into body and/or caudal fin propulsion and median and/or paired pectoral fin propulsion. In this article, the paired pectoral fin-based oscillatory propulsion using ionic polymer-metal composite for aquatic propulsor applications is studied. Beam theory and the concept of hydrodynamic function are used to describe the interaction between the beam and water. Furthermore, a quasi-steady blade element model that accounts for unsteady phenomena such as added mass effects, dynamic stall, and the cumulative Wagner effect is used to obtain hydrodynamic performance of the ionic polymer-metal composite propulsor. Dynamic characteristics of ionic polymer-metal composite fin are analyzed using numerical simulations. It is shown that the use of optimization methods can lead to significant improvement in performance of the ionic polymer-metal composite fin.

MS_RHII-RSD, a Dual-Function RNase HII-(p)ppGpp Synthetase from Mycobacterium smegmatis

In the noninfectious soil saprophyte Mycobacterium smegmatis, intracellular levels of the stress alarmones guanosine tetraphosphate and guanosine pentaphosphate, together termed (p)ppGpp, are regulated by the enzyme Rel(Msm). This enzyme consists of a single, bifunctional polypeptide chain that is capable of both synthesizing and hydrolyzing (p)ppGpp. The rel(Msm), knockout strain of M. smegmatis (Delta rel(Msm)) is expected to show a (p)ppGpp null (p)ppGpp(0)] phenotype. Contrary to this expectation, the strain is capable of synthesizing (p)ppGpp in vivo. In this study, we identify and functionally characterize the open reading frame (ORF), MSMEG_5849, that encodes a second functional (p)ppGpp synthetase in M. smegmatis. In addition to (p)ppGpp synthesis, the 567-amino-acid-long protein encoded by this gene is capable of hydrolyzing RNA(.)DNA hybrids and bears similarity to the conventional RNase HII enzymes. We have classified this protein as actRel(Msm) in accordance with the recent nomenclature proposed and have named it MS_RHII-RSD, indicating the two enzymatic activities present RHII, RNase HII domain, originally identified as (d) under bar omain of (u) under bar nknown (f) under bar unction 429 (DUF429), and RSD, RelA_SpoT nucleotidyl transferase domain, the SYNTH domain responsible for (p)ppGpp synthesis activity]. MS_RHII-RSD is expressed and is constitutively active in vivo and behaves like a monofunctional (p)ppGpp synthetase in vitro. The occurrence of the RNase HII and (p)ppGpp synthetase domains together on the same polypeptide chain is suggestive of an in vivo role for this novel protein as a link connecting the essential life processes of DNA replication, repair, and transcription to the highly conserved stress survival pathway, the stringent response.

PocketAnnotate: towards site-based function annotation

A computational pipeline PocketAnnotate for functional annotation of proteins at the level of binding sites has been proposed in this study. The pipeline integrates three in-house algorithms for site-based function annotation: PocketDepth, for prediction of binding sites in protein structures; PocketMatch, for rapid comparison of binding sites and PocketAlign, to obtain detailed alignment between pair of binding sites. A novel scheme has been developed to rapidly generate a database of non-redundant binding sites. For a given input protein structure, putative ligand-binding sites are identified, matched in real time against the database and the query substructure aligned with the promising hits, to obtain a set of possible ligands that the given protein could bind to. The input can be either whole protein structures or merely the substructures corresponding to possible binding sites. Structure-based function annotation at the level of binding sites thus achieved could prove very useful for cases where no obvious functional inference can be obtained based purely on sequence or fold-level analyses. An attempt has also been made to analyse proteins of no known function from Protein Data Bank. PocketAnnotate would be a valuable tool for the scientific community and contribute towards structure-based functional inference. The web server can be freely accessed at http://proline.biochem.iisc.ernet.in/pocketannotate/.

The antibodies against the computationally designed mimic of the glycoprotein hormone receptor transmembrane domain provide insights into receptor activation and suppress the constitutively activated receptor Mutants

The exoloops of glycoprotein hormone receptors (GpHRs) transduce the signal generated by the ligand-ectodomain interactions to the transmembrane helices either through direct hormonal contact and/or by modulating the interdomain interactions between the hinge region (HinR) and the transmembrane domain (TMD). The ligand-induced conformational alterations in the HinRs and the interhelical loops of luteinizing hormone receptor/follicle stimulating hormone receptor/thyroid stimulating hormone receptor were mapped using exoloop-specific antibodies generated against a mini-TMD protein designed to mimic the native exoloop conformations that were created by joining the thyroid stimulating hormone receptor exoloops constrained through helical tethers and library-derived linkers. The antibody against the mini-TMD specifically recognized all three GpHRs and inhibited the basal and hormone-stimulated cAMP production without affecting hormone binding. Interestingly, binding of the antibody to all three receptors was abolished by prior incubation of the receptors with the respective hormones, suggesting that the exoloops are buried in the hormone-receptor complexes. The antibody also suppressed the high basal activities of gain-of-function mutations in the HinRs, exoloops, and TMDs such as those involved in precocious puberty and thyroid toxic adenomas. Using the antibody and point/deletion/chimeric receptor mutants, we demonstrate that changes in the HinR-exoloop interactions play an important role in receptor activation. Computational analysis suggests that the mini-TMD antibodies act by conformationally locking the transmembrane helices by means of restraining the exoloops and the juxta-membrane regions. Using GpHRs as a model, we describe a novel computational approach of generating soluble TMD mimics that can be used to explain the role of exoloops during receptor activation and their interplay with TMDs.

Relation between the work function and Young's modulus of RhSi and estimate of Schottky-barrier height at RhSi/Si interface: An ab-initio study

Density-functional calculations are performed to explore the relationship between the work function and Young's modulus of RhSi, and to estimate the p-Schottky-barrier height (SBH) at the Si/RhSi(010) interface. It is shown that the Young's modulus and the workfunction of RhSi satisfy the generic sextic relation, proposed recently for elemental metals. The calculated p-SBH at the Si/RhSi interface is found to differ only by 0.04 eV in opposite limits, viz., no-pinning and strong pinning. We find that the p-SBH is reduced as much as by 0.28 eV due to vacancies at the interface. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4761994]

Perturbative expansion of the QCD Adler function improved by renormalization-group summation and analytic continuation in the Borel plane

We examine the large-order behavior of a recently proposed renormalization-group-improved expansion of the Adler function in perturbative QCD, which sums in an analytically closed form the leading logarithms accessible from renormalization-group invariance. The expansion is first written as an effective series in powers of the one-loop coupling, and its leading singularities in the Borel plane are shown to be identical to those of the standard ``contour-improved'' expansion. Applying the technique of conformal mappings for the analytic continuation in the Borel plane, we define a class of improved expansions, which implement both the renormalization-group invariance and the knowledge about the large-order behavior of the series. Detailed numerical studies of specific models for the Adler function indicate that the new expansions have remarkable convergence properties up to high orders. Using these expansions for the determination of the strong coupling from the hadronic width of the tau lepton we obtain, with a conservative estimate of the uncertainty due to the nonperturbative corrections, alpha(s)(M-tau(2)) = 0.3189(-0.0151)(+0.0173), which translates to alpha(s)(M-Z(2)) = 0.1184(-0.0018)(+0.0021). DOI: 10.1103/PhysRevD.87.014008

SURE-FAST BILATERAL FILTERS

Edge-preserving smoothing is widely used in image processing and bilateral filtering is one way to achieve it. Bilateral filter is a nonlinear combination of domain and range filters. Implementing the classical bilateral filter is computationally intensive, owing to the nonlinearity of the range filter. In the standard form, the domain and range filters are Gaussian functions and the performance depends on the choice of the filter parameters. Recently, a constant time implementation of the bilateral filter has been proposed based on raisedcosine approximation to the Gaussian to facilitate fast implementation of the bilateral filter. We address the problem of determining the optimal parameters for raised-cosine-based constant time implementation of the bilateral filter. To determine the optimal parameters, we propose the use of Stein's unbiased risk estimator (SURE). The fast bilateral filter accelerates the search for optimal parameters by faster optimization of the SURE cost. Experimental results show that the SURE-optimal raised-cosine-based bilateral filter has nearly the same performance as the SURE-optimal standard Gaussian bilateral filter and the Oracle mean squared error (MSE)-based optimal bilateral filter.

Spectral moment sum rules for the retarded Green's function and self-energy of the inhomogeneous Bose-Hubbard model in equilibrium and nonequilibrium

We derive exact expressions for the zeroth and the first three spectral moment sum rules for the retarded Green's function and for the zeroth and the first spectral moment sum rules for the retarded self-energy of the inhomogeneous Bose-Hubbard model in nonequilibrium, when the local on-site repulsion and the chemical potential are time-dependent, and in the presence of an external time-dependent electromagnetic field. We also evaluate these expressions for the homogeneous case in equilibrium, where all time dependence and external fields vanish. Unlike similar sum rules for the Fermi-Hubbard model, in the Bose-Hubbard model case, the sum rules often depend on expectation values that cannot be determined simply from parameters in the Hamiltonian like the interaction strength and chemical potential but require knowledge of equal-time many-body expectation values from some other source. We show how one can approximately evaluate these expectation values for the Mott-insulating phase in a systematic strong-coupling expansion in powers of the hopping divided by the interaction. We compare the exact moment relations to the calculated moments of spectral functions determined from a variety of different numerical approximations and use them to benchmark their accuracy. DOI: 10.1103/PhysRevA.87.013628

Toll-Like Receptors 7, 8, and 9 Expression and Function in Primary Human Cervical Cancer Langerhans Cells Evidence of Anergy

Objective: Human papillomavirus oncoproteins E6 and E7 down modulate Toll-like receptor (TLR) 9 expression in infected keratinocytes. We explored the status of expression and function of TLR7, TLR8, and TLR9 in primary human Langerhans cells (LCs) isolated from cervical tumors. Methodology: Single-cell suspensions were made from fresh tissues of squamous cell carcinoma (International Federation of Gynecology and Obstetrics stage IB2); myeloid dendritic cells were purified using CD1c magnetic activated cell separation kits. Langerhans cells were further flow sorted into CD1a(+)CD207(+) cells. Acute monocytic leukemia cell line THP-1-derived LCs (moLCs) formed the controls. mRNA from flow-sorted LCs was reverse transcribed to cDNA and TLR7, TLR8, and TLR9 amplified. Monocyte-derived Langerhans cells and cervical tumor LCs were stimulated with TLR7, TLR8, and TLR9 ligands. Culture supernatants were assayed for interleukin (IL) 1 beta, IL-6, IL-10, IL-12p70, interferon (IFN) alpha, interferon gamma, and tumor necrosis factor (TNF) alpha by Luminex multiplex bead array. Human papillomavirus was genotyped. Results: We have for the first time demonstrated that the acute monocytic leukemia cell line THP-1 can be differentiated into LCs in vitro. Although these moLCs. expressed all the 3 TLRs, tumor LCs expressed TLR7 and TLR8, but uniformly lacked TLR9. Also, moLCs secreted IL-6, IL-1 beta, and tumor necrosis factor alpha to TLR8 ligand and interferon alpha in response to TLR9 ligand; in contrast, tumor LCs did not express any cytokine to any of the 3 TLR ligands. Human papillomavirus type 16 was one of the common human papillomavirus types in all cases. Conclusions: Cervical tumor LCs lacked TLR9 expression and were functionally anergic to all the 3: TLR7, TLR8, and TLR9 ligands, which may play a crucial role in immune tolerance. The exact location of block(s) in TLR7 and TLR8 signaling needs to be investigated, which would have important immunotherapeutic implications.

Smoothed functional and Quasi-Newton algorithms for routing in multi-stage queueing network with constraints

We consider the problem of optimal routing in a multi-stage network of queues with constraints on queue lengths. We develop three algorithms for probabilistic routing for this problem using only the total end-to-end delays. These algorithms use the smoothed functional (SF) approach to optimize the routing probabilities. In our model all the queues are assumed to have constraints on the average queue length. We also propose a novel quasi-Newton based SF algorithm. Policies like Join Shortest Queue or Least Work Left work only for unconstrained routing. Besides assuming knowledge of the queue length at all the queues. If the only information available is the expected end-to-end delay as with our case such policies cannot be used. We also give simulation results showing the performance of the SF algorithms for this problem.

A nested linear codes approach to distributed function computation over subspaces

In this paper, we consider a distributed function computation setting, where there are m distributed but correlated sources X1,...,Xm and a receiver interested in computing an s-dimensional subspace generated by [X1,...,Xm]Γ for some (m × s) matrix Γ of rank s. We construct a scheme based on nested linear codes and characterize the achievable rates obtained using the scheme. The proposed nested-linear-code approach performs at least as well as the Slepian-Wolf scheme in terms of sum-rate performance for all subspaces and source distributions. In addition, for a large class of distributions and subspaces, the scheme improves upon the Slepian-Wolf approach. The nested-linear-code scheme may be viewed as uniting under a common framework, both the Korner-Marton approach of using a common linear encoder as well as the Slepian-Wolf approach of employing different encoders at each source. Along the way, we prove an interesting and fundamental structural result on the nature of subspaces of an m-dimensional vector space V with respect to a normalized measure of entropy. Here, each element in V corresponds to a distinct linear combination of a set {Xi}im=1 of m random variables whose joint probability distribution function is given.

Quasi-based hierarchical clustering for land cover mapping using satellite images

This paper presents an improved hierarchical clustering algorithm for land cover mapping problem using quasi-random distribution. Initially, Niche Particle Swarm Optimization (NPSO) with pseudo/quasi-random distribution is used for splitting the data into number of cluster centers by satisfying Bayesian Information Criteria (BIC). Themain objective is to search and locate the best possible number of cluster and its centers. NPSO which highly depends on the initial distribution of particles in search space is not been exploited to its full potential. In this study, we have compared more uniformly distributed quasi-random with pseudo-random distribution with NPSO for splitting data set. Here to generate quasi-random distribution, Faure method has been used. Performance of previously proposed methods namely K-means, Mean Shift Clustering (MSC) and NPSO with pseudo-random is compared with the proposed approach - NPSO with quasi distribution(Faure). These algorithms are used on synthetic data set and multi-spectral satellite image (Landsat 7 thematic mapper). From the result obtained we conclude that use of quasi-random sequence with NPSO for hierarchical clustering algorithm results in a more accurate data classification.

Fish inspired biomimetic ionic polymer metal composite pectoral fins using labriform propulsion

Ionic polymer metal composites (IPMC) are a new class of smart materials that have attractive characteristics such as muscle like softness, low voltage and power consumption, and good performance in aqueous environments. Thus, IPMC’s provide promising application for biomimetic fish like propulsion systems. In this paper, we design and analyze IPMC underwater propulsor inspired from swimming of Labriform fishes. Different fish species in nature are source of inspiration for different biomimetic flapping IPMC fin design. Here, three fish species with high performance flapping pectoral fin locomotion is chosen and performance analysis of each fin design is done to discover the better configurations for engineering applications. In order to describe the behavior of an active IPMC fin actuator in water, a complex hydrodynamic function is used and structural model of the IPMC fin is obtained by modifying the classical dynamic equation for a slender beam. A quasi-steady blade element model that accounts for unsteady phenomena such as added mass effects, dynamic stall, and the cumulative Wagner effect is used to estimate the hydrodynamic performance of the flapping rectangular shape fin. Dynamic characteristics of IPMC actuated flapping fins having the same size as the actual fins of three different fish species, Gomphosus varius, Scarus frenatus and Sthethojulis trilineata, are analyzed with numerical simulations. Finally, a comparative study is performed to analyze the performance of three different biomimetic IPMC flapping pectoral fins.