Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Effect of surface on the conductance characteristics of Au‐Bi2Sr2CaCu2O8+δ (single crystal) point contact junctions

We report the effect of surface treatments on the dynamic conductance curves (G=dI/dV‐V) of Au‐Bi2Sr2CaCu2O8+δ (single crystal) point contact junctions of variable junction conductances (100 mS≳G≳100 μS). We find that if the crystal surface is cleaved freshly just prior to making contacts, all irreproducible sharp multiple features often observed in tunneling data of Bi(2212) oxide superconductors disappear. If the cleaved crystal surfaces are left under ambient conditions for a few days and the tunneling experiments are repeated, these multiple features reappear. We also find that if the current in the junction is made to pass predominantly through the bulk (and not along the surface), gap features are sharper. The observed conductance curves are fitted to a modified model [G. E. Blonder et al., Phys. Rev. B 25, 4515 (1982)] and estimated gap values are Δ≂28 to 30 meV corresponding to the ratio 2Δ/kBTc ≂ 7.5 with lifetime broadening Γ/Δ≂0.2. We conclude that the sharp multiple features observed in Bi(2212) tunneling curves has no intrinsic origin in the bulk and they arise from the surface only.

Characteristics of j.f.e.t.s. taking the separation of gate electrode from the source and drain electrodes into account

The performance characteristics of a junction field-effect transistor (j.f.e.t.) are evaluated considering the presence of the gap between the gate electrode and the source and drain terminals. It is concluded that the effect of the gap is to demand a higher drain voltage to maintain the same drain current. So long as the device is operated at the same drain current, the presence of the gap does not change the performance of the device as an amplifier. The nature of the performance of the device as a variable resistor is not affected by the gap if it is less than or equal to the physical height of the channel. For gap lengths larger than the channel height, the effect of the gap is to add a series resistance in the drain.

Functional Analysis of DNA Replication Fork Reversal Catalyzed by Mycobacterium tuberculosis RuvAB Proteins

Initially discovered in Escherichia coli, RuvAB proteins are ubiquitous in bacteria and play a dual role as molecular motor proteins responsible for branch migration of the Holliday junction(s) and reversal of stalled replication forks. Despite mounting genetic evidence for a crucial role of RuvA and RuvB proteins in reversal of stalled replication forks, the mechanistic aspects of this process are still not fully understood. Here, we elucidate the ability of Mycobacterium tuberculosis RuvAB (MtRuvAB) complex to catalyze the reversal of replication forks using a range of DNA replication fork substrates. Our studies show that MtRuvAB, unlike E. coli RuvAB, is able to drive replication fork reversal via the formation of Holliday junction intermediates, suggesting that RuvAB-catalyzed fork reversal involves concerted unwinding and annealing of nascent leading and lagging strands. We also demonstrate the reversal of replication forks carrying hemi-replicated DNA, indicating that MtRuvAB complex-catalyzed fork reversal is independent of symmetry at the fork junction. The fork reversal reaction catalyzed by MtRuvAB is coupled to ATP hydrolysis, is processive, and culminates in the formation of an extended reverse DNA arm. Notably, we found that sequence heterology failed to impede the fork reversal activity of MtRuvAB. We discuss the implications of these results in the context of recognition and processing of varied types of replication fork structures by RuvAB proteins.

Selective synthesis of metallic and semiconducting single-walled carbon nanotubes

As-prepared single-walled carbon nanotubes (SWNTs) are generally mixtures of semiconducting and metallic species, the proportion of the former being around 67%. Since most applications of SWNTs are best served by semiconducting or metallic nanotubes, rather than by mixtures of the two, methods which would directly yield semiconducting and metallic SWNTs in pure form are desirable. In this article, we present the available methods for the direct synthesis of such SWNTs along with the methods available to separate semiconducting and metallic SWNTs from mixtures. We also discuss the synthesis of Y-junction carbon nanotubes.

Free Energy Barriers for Escape of Water Molecules from Protein Hydration Layer

Free energy barriers separating interfacial water molecules from the hydration layer at the surface of a protein to the bulk are obtained by using the umbrella sampling method of free energy calculation. We consider hydration layer of chicken villin head piece (HP-36) which has been studied extensively by molecular dynamics simulations. The free energy calculations reveal a strong sensitivity to the secondary structure. In particular, we find a region near the junction of first and second helix that contains a cluster of water molecules which are slow in motion, characterized by long residence times (of the order of 100 ps or more) and separated by a large free energy barrier from the bulk water. However, these ``slow'' water molecules constitute only about 5-10% of the total number of hydration layer water molecules. Nevertheless, they play an important role in stabilizing the protein conformation. Water molecules near the third helix (which is the important helix for biological function) are enthalpically least stable and exhibit the fastest dynamics. Interestingly, barrier height distributions of interfacial water are quite broad for water surrounding all the three helices (and the three coils), with the smallest barriers found for those near the helix-3. For the quasi-bound water molecules near the first and second helices, we use well-known Kramers' theory to estimate the residence time from the free energy surface, by estimating the friction along the reaction coordinate from the diffusion coefficient by using Einstein relation. The agreement found is satisfactory. We discuss the possible biological function of these slow, quasi-bound (but transient) water molecules on the surface.

Bose-Hubbard model in a strong effective magnetic field: Emergence of a chiral Mott insulator ground state

Motivated by experiments on Josephson junction arrays, and cold atoms in an optical lattice in a synthetic magnetic field, we study the ``fully frustrated'' Bose-Hubbard model with half a magnetic flux quantum per plaquette. We obtain the phase diagram of this model on a two-leg ladder at integer filling via the density matrix renormalization group approach, complemented by Monte Carlo simulations on an effective classical XY model. The ground state at intermediate correlations is consistently shown to be a chiral Mott insulator (CMI) with a gap to all excitations and staggered loop currents which spontaneously break time-reversal symmetry. We characterize the CMI state as a vortex supersolid or an indirect exciton condensate, and discuss various experimental implications.

Viral Kinetics Suggests a Reconciliation of the Disparate Observations of the Modulation of Claudin-1 Expression on Cells Exposed to Hepatitis C Virus

The tight junction protein claudin-1 (CLDN1) is necessary for hepatitis C virus (HCV) entry into target cells. Recent studies have made disparate observations of the modulation of the expression of CLDN1 on cells following infection by HCV. In one study, the mean CLDN1 expression on cells exposed to HCV declined, whereas in another study HCV infected cells showed increased CLDN1 expression compared to uninfected cells. Consequently, the role of HCV in modulating CLDN1 expression, and hence the frequency of cellular superinfection, remains unclear. Here, we present a possible reconciliation of these disparate observations. We hypothesized that viral kinetics and not necessarily HCV-induced receptor modulation underlies these disparate observations. To test this hypothesis, we constructed a mathematical model of viral kinetics in vitro that mimicked the above experiments. Model predictions provided good fits to the observed evolution of the distribution of CLDN1 expression on cells following exposure to HCV. Cells with higher CLDN1 expression were preferentially infected and outgrown by cells with lower CLDN1 expression, resulting in a decline of the mean CLDN1 expression with time. At the same time, because the susceptibility of cells to infection increased with CLDN1 expression, infected cells tended to have higher CLDN1 expression on average than uninfected cells. Our study thus presents an explanation of the disparate observations of CLDN1 expression following HCV infection and points to the importance of considering viral kinetics in future studies of receptor expression on cells exposed to HCV.

Study of n-ZnO/p-Si (100) thin film heterojunctions by pulsed laser deposition without buffer layer

ZnO/Si heterojunctions were fabricated by growing ZnO thin films on p-type Si (100) substrate by pulsed laser deposition without buffer layers. The crystallinity of the heterojunction was analyzed by high resolution X-ray diffraction and atomic force microscopy. The optical quality of the film was analyzed by room temperature (RT) photoluminescence measurements. The high intense band to band emission confirmed the high quality of the ZnO thin films on Si. The electrical properties of the junction were studied by temperature dependent current-voltage measurements and RT capacitance-voltage (C-V) analysis. The charge carrier concentration and the barrier height (BH) were calculated, to be 5.6x10(19) cm(-3) and 0.6 eV respectively from the C-V plot. The BH and ideality factor, calculated using the thermionic emission (TE) model, were found to be highly temperature dependent. We observed a much lower value in Richardson constant, 5.19x10(-7)A/cm(2) K-2 than the theoretical value (32 A/cm(2) K-2) for ZnO. This analysis revealed the existence of a Gaussian distribution (GD) with a standard deviation of sigma(2)=0.035 V. By implementing the GD to the TE, the values of BH and Richardson constant were obtained as 1.3 eV and 39.97 A/cm(2) K-2 respectively from the modified Richardson plot. The obtained Richardson constant value is close to the theoretical value for n-ZnO. These high quality heterojunctions can be used for solar cell applications. (C) 2012 Elsevier B.V. All rights reserved.

Analysis of current voltage characteristics of MWIR homojunction photodiodes for uncooled operation

Dark currents n(+)/v/p(+) Hg0.69Cd0.Te-31 mid wave infrared photodiodes were measured at room temperature. The diodes exhibited negative differential resistance at room-temperature, but with increasing leakage currents as a function of reverse bias. The current-voltage characteristics were simulated and fitted by incorporating trap assisted tunneling via traps and Shockley-Read-Hall generation recombination process due to dislocations in the carrier transport equations. The thermal suppression of carriers was simulated by taking energy level of trap (E-t), trap density (N-t) and the doping concentrations of n(+) and v regions as fitting parameters. Values of E-t and N-t were 0.78E(g) and similar to 6-9 x 10(14) cm(-3) respectively for most of the diodes. Variable temperature current voltage measurements on variable area diode array (VADA) structures confirmed the fact that variation in zero bias resistance area product (R(0)A) is related to g-r processes originating from variation in concentration and kind of defects that intersect a junction area. (C) 2012 Elsevier B.V. All rights reserved.

The Hinge Region of Human Thyroid-Stimulating Hormone (TSH) Receptor Operates as a Tunable Switch between Hormone Binding and Receptor Activation

The mechanism by which the hinge regions of glycoprotein hormone receptors couple hormone binding to activation of downstream effecters is not clearly understood. In the present study, agonistic (311.62) and antagonistic (311.87) monoclonal antibodies (MAbs) directed against the TSH receptor extracellular domain were used to elucidate role of the hinge region in receptor activation. MAb 311.62 which identifies the LRR/Cb-2 junction (aa 265-275), increased the affinity of TSHR for the hormone while concomitantly decreasing its efficacy, whereas MAb 311.87 recognizing LRR 7-9 (aa 201-259) acted as a non-competitive inhibitor of Thyroid stimulating hormone (TSH) binding. Binding of MAbs was sensitive to the conformational changes caused by the activating and inactivating mutations and exhibited differential effects on hormone binding and response of these mutants. By studying the effects of these MAbs on truncation and chimeric mutants of thyroid stimulating hormone receptor (TSHR), this study confirms the tethered inverse agonistic role played by the hinge region and maps the interactions between TSHR hinge region and exoloops responsible for maintenance of the receptor in its basal state. Mechanistic studies on the antibody-receptor interactions suggest that MAb 311.87 is an allosteric insurmountable antagonist and inhibits initiation of the hormone induced conformational changes in the hinge region, whereas MAb 311.62 acts as a partial agonist that recognizes a conformational epitope critical for coupling of hormone binding to receptor activation. The hinge region, probably in close proximity with the alpha-subunit in the hormone-receptor complex, acts as a tunable switch between hormone binding and receptor activation.

Directional motion of impacting drops on dual-textured surfaces

In this work, we analyze the directional movement of impacting liquid drops on dual-textured solid surfaces comprising two different surface morphologies: a textured surface and a smooth surface. The dynamics of liquid drops impacting onto the junction line between the two parts of the dual-textured surfaces is studied experimentally for varying drop impact velocity. The dual-textured surfaces used here featured a variation in their textures' geometrical parameters as well as their surface chemistry. Two types of liquid drop differing in their surface tension were used. The impact process develops a net horizontal drop velocity towards the higher-wettability surface portion and results in a bulk movement of the impacting drop liquid. The final distance moved by the impacting drop from the junction line decreases with increasing impacting drop Weber number We. A fully theoretical model, employing a balance of forces acting at the drop contact line as well as energy conservation, is formulated to determine the variation, with We, of net horizontal drop velocity and subsequent movement of the impacting drop on the dual-textured surfaces.

Role of Specific Cations and Water Entropy on the Stability of Branched DNA Motif Structures

DNA three-way junctions (TWJs) are important intermediates in various cellular processes and are the simplest of a family of branched nucleic acids being considered as scaffolds for biomolecular nanotechnology. Branched nucleic acids are stabilized by divalent cations such as Mg2+, presumably due to condensation and neutralization of the negatively charged DNA backbone. However, electrostatic screening effects point to more complex solvation dynamics and a large role of interfacial waters in thermodynamic stability. Here, we report extensive computer simulations in explicit water and salt on a model TWJ and use free energy calculations to quantify the role of ionic character and strength on stability. We find that enthalpic stabilization of the first and second hydration shells by Mg2+ accounts for 1/3 and all of the free energy gain in 50% and pure MgCl2 solutions, respectively. The more distorted DNA molecule is actually destabilized in pure MgCl2 compared to pure NaCl. Notably, the first shell, interfacial waters have very low translational and rotational entropy (i.e., mobility) compared to the bulk, an entropic loss that is overcompensated by increased enthalpy from additional electrostatic interactions with Mg2+. In contrast, the second hydration shell has anomalously high entropy as it is trapped between an immobile and bulklike layer. The nonmonotonic entropic signature and long-range perturbations of the hydration shells to Mg2+ may have implications in the molecular recognition of these motifs. For example, we find that low salt stabilizes the parallel configuration of the three-way junction, whereas at normal salt we find antiparallel configurations deduced from the NMR. We use the 2PT analysis to follow the thermodynamics of this transition and find that the free energy barrier is dominated by entropic effects that result from the decreased surface area of the antiparallel form which has a smaller number of low entropy waters in the first monolayer.

Spin injection into a metal from a topological insulator

We study a junction of a topological insulator with a thin two-dimensional nonmagnetic or partially polarized ferromagnetic metallic film deposited on a three-dimensional insulator. We show, by deriving generic boundary conditions applicable to electrons traversing the junction, that there is a finite spin-current injection into the film whose magnitude can be controlled by tuning a voltage V applied across the junction. For ferromagnetic films, the direction of the component of the spin current along the film magnetization can also be tuned by tuning the barrier potential V-0 at the junction. We point out the role of the chiral spin-momentum locking of the Dirac electrons behind this phenomenon and suggest experiments to test our theory.

Cloning, expression, purification, crystallization and preliminary X-ray crystallographic study of thymidylate kinase (TTHA1607) from Thermus thermophilus HB8

Nucleotide biosynthesis plays a key role in cell survival and cell proliferation. Thymidylate kinase is an enzyme that catalyses the conversion of dTMP to dTDP using ATP-Mg2+ as a phosphoryl-donor group. This enzyme is present at the junction of the de novo and salvage pathways; thus, any inhibitor designed against it will result in cell death. This highlights the importance of this enzyme as a drug target. Thymidylate kinase from the extremely thermophilic organism Thermus thermophilus HB8 has been expressed, purified and crystallized using the microbatch method. The crystals diffracted to a resolution of 1.83 angstrom and belonged to the orthorhombic space group P2(1)2(1)2(1), with unit-cell parameters a = 39.50, b = 80.29, c = 122.55 angstrom. Preliminary studies revealed the presence of a dimer in the asymmetric unit with a Matthews coefficient (V-M) of 2.18 angstrom(3) Da(-1).