A novel Q-learning algorithm with function approximation for constrained Markov decision processes

We present a novel multi-timescale Q-learning algorithm for average cost control in a Markov decision process subject to multiple inequality constraints. We formulate a relaxed version of this problem through the Lagrange multiplier method. Our algorithm is different from Q-learning in that it updates two parameters - a Q-value parameter and a policy parameter. The Q-value parameter is updated on a slower time scale as compared to the policy parameter. Whereas Q-learning with function approximation can diverge in some cases, our algorithm is seen to be convergent as a result of the aforementioned timescale separation. We show the results of experiments on a problem of constrained routing in a multistage queueing network. Our algorithm is seen to exhibit good performance and the various inequality constraints are seen to be satisfied upon convergence of the algorithm.

Design and analysis of novel compression fracture specimen with constant form factor: Edge cracked semicircular disk (ECSD)

Inspired by the Brazilian disk geometry we examine the utility of an edge cracked semicircular disk (ECSD) specimen for rapid assessment of fracture toughness of brittle materials using compressive loading. It is desirable to optimize the geometry towards a constant form factor F for evaluating K-I. In this investigation photoelastic and finite element results for K-I evaluation highlight the effect of loading modeled using a Hertzian. A Hertzian loading subtending 4 degrees at the center leads to a surprisingly constant form factor of 1.36. This special case is further analyzed by applying uniform pressure over a chord for facilitating testing.

Genome sequencing unveils a novel sea enterotoxin-carrying PVL phage in staphylococcus aureus ST772 from India

Staphylococcus aureus is a major human pathogen, first recognized as a leading cause of hospital-acquired infections. Community-associated S. aureus (CA-SA) pose a greater threat due to increase in severity of infection and disease among children and healthy adults. CA-SA strains in India are genetically diverse, among which is the sequence type (ST) 772, which has now spread to Australia, Europe and Japan. Towards understanding the genetic characteristics of ST772, we obtained draft genome sequences of five relevant clinical isolates and studied the properties of their PVL-carrying prophages, whose presence is a defining hallmark of CA-SA. We show that this is a novel prophage, which carries the structural genes of the hlb-carrying prophage and includes the sea enterotoxin. This architecture probably emerged early within the ST772 lineage, at least in India. The sea gene, unique to ST772 PVL, despite having promoter sequence characteristics typical of low expression, appears to be highly expressed during early phase of growth in laboratory conditions. We speculate that this might be a consequence of its novel sequence context. The crippled nature of the hlb-converting prophage in ST772. suggests that widespread mobility of the sea enterotoxin might be a selective force behind its `transfer' to the PVL prophage. Wild type ST772 strains induced strong proliferative responses as well as high cytotoxic activity against neutrophils, likely mediated by superantigen SEA and the PVL toxin respectively. Both proliferation and cytotoxicity were markedly reduced in a cured ST772 strain indicating the impact of the phage on virulence. The presence of SEA alongside he genes for the immune system-modulating PVL toxin may contribute to the success and virulence of ST772.

A novel adeno-associated virus serotype (AAV)-8 capsid mutant improves human coagulation factor IX expression in vivo

Recombinant AAV-8 vectors have shown significant promise for hepatic gene therapy of hemophilia B. However, the theme of AAV vector dose dependent immunotoxicity seen with AAV2 vectors earlier seem to re-emerge with AAV8 vectors as well. It is therefore important to develop novel AAV8 vectors that provide enhanced gene expression at significantly less vector doses. We hypothesized that AAV8 during its intracellular trafficking, are targeted for destruction in the cytoplasm by the host-cellular kinase/ubiquitination/proteasomal degradation machinery and modification of specific serine/threonine kinase or ubiquitination targets on AAV8 capsid (Fig.1A) may improve its transduction efficiency. To test this, point mutations at specific serine (S)/threonine (T) > alanine (A) or lysine (K)>arginine (R) residues were generated on AAV8 capsid. scAAV8-EGFP vectors containing the wild-type (WT) and each one of the 5 S/T/K-mutant(S276A, S501A, S671A, T251A and K137R) capsids were evaluated for their liver transduction efficiency at a dose of 5 X 1010 vgs/ animal in C57BL/6 mice in vivo. The best performing mutant was found to be the K137R vector in terms of either the gene expression (46-fold) or the vector copy numbers in the hepatocytes (22-fold) compared to WT-AAV8 (Fig.1B). The K137R-AAV8 vector that showed significantly decreased ubiquitination of the viral capsid had reduced activation of markers of innate immune response [IL-6, IL-12, tumor necrosis factor α, Kupffer cells and TLR-9]. In addition, animals injected with the K137R mutant also demonstrated decreased (2-fold) levels of cross-neutralizing antibodies when compared to animals that received the WT-AAV8 vector. To study further the utility of the novel AAV8-K137R mutant in a therapeutic setting, we delivered human coagulation factor IX (h.FIX) under the control of liver specific promoters (LP1 or hAAT) at two different doses (2.5x10^10 and 1x10^11 vgs per mouse) in 8-12 weeks old male C57BL/6 mice. As can be seen in Fig.1C/D, the circulating levels of h.FIX were higher in all the K137R-AAV8 treated groups as compared to the WT-AAV8 treated groups either at 2 weeks (62% vs 37% for hAAT constructs and 47% vs 21% for LP1 constructs) or 4 weeks (78% vs 56% for hAAT constructs and 64% vs 30% for LP1 constructs) post hepatic gene transfer. These studies demonstrate the feasibility of the use of this novel vector for potential gene therapy of hemophilia B.

A novel filtering framework through Girsanov correction for the identification of nonlinear dynamical systems

Using a Girsanov change of measures, we propose novel variations within a particle-filtering algorithm, as applied to the inverse problem of state and parameter estimations of nonlinear dynamical systems of engineering interest, toward weakly correcting for the linearization or integration errors that almost invariably occur whilst numerically propagating the process dynamics, typically governed by nonlinear stochastic differential equations (SDEs). Specifically, the correction for linearization, provided by the likelihood or the Radon-Nikodym derivative, is incorporated within the evolving flow in two steps. Once the likelihood, an exponential martingale, is split into a product of two factors, correction owing to the first factor is implemented via rejection sampling in the first step. The second factor, which is directly computable, is accounted for via two different schemes, one employing resampling and the other using a gain-weighted innovation term added to the drift field of the process dynamics thereby overcoming the problem of sample dispersion posed by resampling. The proposed strategies, employed as add-ons to existing particle filters, the bootstrap and auxiliary SIR filters in this work, are found to non-trivially improve the convergence and accuracy of the estimates and also yield reduced mean square errors of such estimates vis-a-vis those obtained through the parent-filtering schemes.

Composite cyclodextrin-calcium carbonate porous microparticles and modified multilayer capsules: novel carriers for encapsulation of hydrophobic drugs

Novel composite cyclodextrin (CD)-CaCO3 spherical porous microparticles have been synthesized through Ca2+-CD complex formation, which influences the crystal growth of CaCO3. The CDs are entrapped and distributed uniformly in the matrix of CaCO3 microparticles during crystallization. The hydrophobic fluorescent molecules coumarin and Nile red (NR) are efficiently encapsulated into these composite CD-CaCO3 porous particles through supramolecular inclusion complexation between entrapped CDs and hydrophobic molecules. Thermogravimetric (TGA) and infrared spectroscopy (IR) analysis of composite CD-CaCO3 particles reveals the presence of large CDs and their strong interaction with calcium carbonate nanoparticles. The resulting composite CD-CaCO3 microparticles are utilized as sacrificial templates for preparation of CD-modified layer-by-layer (LbL) capsules. After dissolution of the carbonate core, CDs are retained in the interior of the capsules in a network fashion and assist in the encapsulation of hydrophobic molecules. The efficient encapsulation of the hydrophobic fluorescent dye, coumarin, was successfully demonstrated using CD-modified capsules. In vitro release of the encapsulated coumarin from the CD-CaCO3 and CD-modified capsules has been demonstrated.

Fermions in synthetic non-Abelian gauge potentials: rashbon condensates to novel Hamiltonians

Recent advances in the generation of synthetic gauge fields in cold atomic systems have stimulated interest in the physics of interacting bosons and fermions in them. In this paper, we discuss interacting two-component fermionic systems in uniform non-Abelian gauge fields that produce a spin-orbit interaction and uniform spin potentials. Two classes of gauge fields discussed include those that produce a Rashba spin-orbit interaction and the type of gauge fields (SM gauge fields) obtained in experiments by the Shanxi and MIT groups. For high symmetry Rashba gauge fields, a two-particle bound state exists even for a vanishingly small attractive interaction described by a scattering length. Upon increasing the strength of a Rashba gauge field, a finite density of weakly interacting fermions undergoes a crossover from a BCS like ground state to a BEC state of a new kind of boson called the rashbon whose properties are determined solely by the gauge field and not by the interaction between the fermions. The rashbon Bose-Einstein condensate (RBEC) is a quite intriguing state with the rashbon-rashbon interactions being independent of the fermion-fermion interactions (scattering length). Furthermore, we show that the RBEC has a transition temperature of the order of the Fermi temperature, suggesting routes to enhance the transition temperatures of weakly interacting superfluids by tuning the spin-orbit coupling. For the SM gauge fields, we show that in a regime of parameters, a pair of particles with finite centre-of-mass momentum is the most strongly bound. In other regimes of centre-of-mass momenta, there is no two-body bound state, but a resonance like feature appears in the scattering continuum. In the many-body setting, this results in flow enhanced pairing. Also, strongly interacting normal states utilizing the scattering resonance can be created opening the possibility of studying properties of helical Fermi liquids. This paper contains a general discussion of the physics of Feshbach resonance in a non-Abelian gauge field, where several novel features such as centre-of-mass-momentum-dependent effective interactions are shown. It is also shown that a uniform non-Abelian gauge field in conjunction with a spatial potential can be used to generate novel Hamiltonians; we discuss an explicit example of the generation of a monopole Hamiltonian.

A Novel Anticancer Agent, 8-Methoxypyrimido4 `,5 `: 4,5]thieno(2,3-b) Quinoline-4(3H)-One Induces Neuro 2a Neuroblastoma Cell Death through p53-Dependent, Caspase-Dependent and Independent Apoptotic Pathways

Neuroblastoma is the most common cancer in infants and fourth most common cancer in children. Despite recent advances in cancer treatments, the prognosis of stage-IV neuroblastoma patients continues to be dismal which warrant new pharmacotherapy. A novel tetracyclic condensed quinoline compound, 8-methoxypyrimido 4 `,5 `: 4,5] thieno(2,3-b) quinoline-4(3H)-one (MPTQ) is a structural analogue of an anticancer drug ellipticine and has been reported to posses anticancer property. Study on MPTQ on neuroblastoma cells is very limited and mechanisms related to its cytotoxicity on neuroblastoma cells are completely unknown. Here, we evaluated the anticancer property of MPTQ on mouse neuro 2a and human SH-SY5Y neuroblastoma cells and investigated the mechanisms underlying MPTQ-mediated neuro 2a cell death. MPTQ-mediated neuro 2a and SH-SY5Y cell deaths were found to be dose and time dependent. Moreover, MPTQ induced cell death reached approximately 99.8% and 90% in neuro 2a and SH-SY5Y cells respectively. Nuclear oligonucleosomal DNA fragmentation and Terminal dUTP Nick End Labelling assays indicated MPTQ-mediated neuro 2a cell death involved apoptosis. MPTQ-mediated apoptosis is associated with increased phosphorylation of p53 at Ser15 and Ser20 which correlates with the hyperphosphorylation of Ataxia-Telangiectasia mutated protein (ATM). Immunocytochemical analysis demonstrated the increased level of Bax protein in MPTQ treated neuro 2a cells. MPTQ-mediated apoptosis is also associated with increased activation of caspase-9, -3 and -7 but not caspase-2 and -8. Furthermore, increased level of caspase-3 and cleaved Poly ( ADP Ribose) polymerase were observed in the nucleus of MPTQ treated neuro 2a cells, suggesting the involvement of caspase-dependent intrinsic but not extrinsic apoptotic pathway. Increased nuclear translocation of apoptosis inducing factor suggests additional involvement of caspase-independent apoptosis pathway in MPTQ treated neuro 2a cells. Collectively, MPTQ-induced neuro 2a cell death is mediated by ATM and p53 activation, and Bax-mediated activation of caspase-dependent and caspase-independent mitochondrial apoptosis pathways.

Stabilization of diborane(4) by transition metal fragments and a novel metal to pi Dewar-Chatt-Duncanson model of back donation

The feasibility of using transition metal fragments to stabilize B2H4 in planar configuration by donating 2 electrons to the boron moiety is investigated. Building upon the existing theoretical and experimental data and aided by the isolobal analogy, the model transition metal complexes Cr(CO)(4)B2H4 (6), Mn(CO)-CpB2H4 (7), Fe(CO)(3)B2H4 (8) and CoCpB2H4 (9) are chosen to illustrate this unique bonding feature bond strengthening with pi-back donation. Other possible types of complexes with B2H4 and the metal fragment are also explored and the energies are compared. One of the low energy isomers wherein the planar B2H4 interacts with the metal fragment in an in-plane fashion represents a unique case study for the Dewar-Chatt-Duncanson model. In this complex the back-donation from the metal fills the p bonding orbital between the two boron atoms thus forming a B=B double bond.

Ellagic acid - a novel organic electrode material for high capacity lithium ion batteries

Ellagic acid, a naturally occurring polyphenol, extracted from pomegranate husk, is found to be a very good organic electrode material for rechargeable lithium batteries with high reversible capacities of similar to 450 and 200 mA h g(-1) at C/10 and C/2.5 discharge rates, respectively; ex situ NMR studies reveal possible lithiation-delithiation modes at different stages of the charge-discharge process.

Novel piezoelectric thin film impact sensor: application in non-destructive material discrimination

We report on the design, development, and performance study of a packaged piezoelectric thin film impact sensor, and its potential application in non-destructive material discrimination. The impact sensing element employed was a thin circular diaphragm of flexible Phynox alloy. Piezoelectric ZnO thin film as an impact sensing layer was deposited on to the Phynox alloy diaphragm by RF reactive magnetron sputtering. Deposited ZnO thin film was characterized by X-ray diffraction (XRD), Atomic Force Microscopy (AFM), and Scanning Electron Microscopy (SEM) techniques. The d(31) piezoelectric coefficient value of ZnO thin film was 4.7 pm V-1, as measured by 4-point bending method. ZnO film deposited diaphragm based sensing element was properly packaged in a suitable housing made of High Density Polyethylene (HDPE) material. Packaged impact sensor was used in an experimental set-up, which was designed and developed in-house for non-destructive material discrimination studies. Materials of different densities (iron, glass, wood, and plastic) were used as test specimens for material discrimination studies. The analysis of output voltage waveforms obtained reveals lots of valuable information about the impacted material. Impact sensor was able to discriminate the test materials on the basis of the difference in their densities. The output response of packaged impact sensor shows high linearity and repeatability. The packaged impact sensor discussed in this paper is highly sensitive, reliable, and cost-effective.

Novel procedure for the estimation of swelling pressures of compacted bentonites based on diffuse double layer theory

Bentonite clays are proven to be attractive as buffer and backfill material in high-level nuclear waste repositories around the world. A quick estimation of swelling pressures of the compacted bentonites for different clay-water-electrolyte interactions is essential in the design of buffer and backfill materials. The theoretical studies on the swelling behavior of bentonites are based on diffuse double layer (DDL) theory. To establish theoretical relationship between void ratio and swelling pressure (e versus P), evaluation of elliptic integral and inverse analysis are unavoidable. In this paper, a novel procedure is presented to establish theoretical relationship of e versus P based on the Gouy-Chapman method. The proposed procedure establishes a unique relationship between electric potentials of interacting and non-interacting diffuse clay-water-electrolyte systems. A procedure is, thus, proposed to deduce the relation between swelling pressures and void ratio from the established relation between electric potentials. This approach is simple and alleviates the need for elliptic integral evaluation and also the inverse analysis. Further, application of the proposed approach to estimate swelling pressures of four compacted bentonites, for example, MX 80, Febex, Montigel and Kunigel V1, at different dry densities, shows that the method is very simple and predicts solutions with very good accuracy. Moreover, the proposed procedure provides continuous distributions of e versus P and thus it is computationally efficient when compared with the existing techniques.

Novel co-evaporation approach for the growth of Sb doped SnO2 nanowires

Antimony doped tin oxide (Sb:SnO2) nanowires were grown by thermal and e-beam assisted co-evaporation of Sb and Sn in the presence of oxygen at a low substrate temperature of 450 degrees C. The field emission scanning electron microscopy study revealed that the nanowires had a length and diameter of 2-4 mu m and 20-60 nm respectively. Transmission electron microscopy study revealed the single crystalline nature of the nanowires; energy dispersive X-ray spectroscopy (EDS) and EDS mapping on the nanowires confirmed the presence of Sb doping in the nanowires. UV light detection study on the doped SnO2 nanowire films exhibited fast response and recovery time compared to undoped SnO2 nanowire films. This is an innovative and simple method to grow doped SnO2 nanowires.

A Novel Monte-Carlo-Sampling-Based Receiver for Large-Scale Uplink Multiuser MIMO Systems

In this paper, we propose low-complexity algorithms based on Monte Carlo sampling for signal detection and channel estimation on the uplink in large-scale multiuser multiple-input-multiple-output (MIMO) systems with tens to hundreds of antennas at the base station (BS) and a similar number of uplink users. A BS receiver that employs a novel mixed sampling technique (which makes a probabilistic choice between Gibbs sampling and random uniform sampling in each coordinate update) for detection and a Gibbs-sampling-based method for channel estimation is proposed. The algorithm proposed for detection alleviates the stalling problem encountered at high signal-to-noise ratios (SNRs) in conventional Gibbs-sampling-based detection and achieves near-optimal performance in large systems with M-ary quadrature amplitude modulation (M-QAM). A novel ingredient in the detection algorithm that is responsible for achieving near-optimal performance at low complexity is the joint use of a mixed Gibbs sampling (MGS) strategy coupled with a multiple restart (MR) strategy with an efficient restart criterion. Near-optimal detection performance is demonstrated for a large number of BS antennas and users (e. g., 64 and 128 BS antennas and users). The proposed Gibbs-sampling-based channel estimation algorithm refines an initial estimate of the channel obtained during the pilot phase through iterations with the proposed MGS-based detection during the data phase. In time-division duplex systems where channel reciprocity holds, these channel estimates can be used for multiuser MIMO precoding on the downlink. The proposed receiver is shown to achieve good performance and scale well for large dimensions.

A Novel Prevailing Torque Threaded Fastener and Its Analysis

We present a novel concept of a threaded fastener that is resistant to loosening under vibration. The anti-loosening feature does not use any additional element and is based on modifying the geometry of the thread in the bolt. In a normal nut and bolt combination, the axial motion of the nut or bolt is linearly related to the rotation by a constant pitch. In the proposed concept, the axial motion in the bolt is chosen to be a cubic function of the rotation, while for the nut, the axial motion remains linearly related to the rotation. This mismatch results in interference during the tightening process and additional torque required to overcome this interference gives rise to the enhanced anti-loosening property. In addition, the cubic curve is designed to ensure that the mismatch results in stresses and deformation in the elastic region of the chosen material. This ensures that the nut can be removed and reused while maintaining a repeatable anti-loosening property in the threaded fastener. A finite element analysis demonstrates the feasibility of this concept.