Two-timescale Q-learning Algorithms with an Application to Routing in Networks

We propose two variants of the Q-learning algorithm that (both) use two timescales. One of these updates Q-values of all feasible state-action pairs at each instant while the other updates Q-values of states with actions chosen according to the ‘current ’ randomized policy updates. A sketch of convergence of the algorithms is shown. Finally, numerical experiments using the proposed algorithms for routing on different network topologies are presented and performance comparisons with the regular Q-learning algorithm are shown.

Ethyl 6-methyl-2-sulfanylidene-4-[4(trifluoromethyl)phenyl]-1,2,3,4-tetrahyd ropyrimidine-5-carboxylate

The title compound, C(15)H(15)F(3)N(2)O(2)S, adopts a conformation with an intramolecular C-H center dot center dot center dot pi interaction. The dihedral angles between the planes of the 4-(trifluoromethyl) phenyl and ester groups with the plane of the six-membered tetrahydropyrimidine ring are 81.8 (1) and 16.0 (1)degrees, respectively. In the crystal structure, intermolecular N-H center dot center dot center dot S hydrogen bonds link pairs of molecules into dimers and N-H center dot center dot center dot O interactions generate hydrogen-bonded molecular chains along the crystallographic a axis.

Optimal location of centre of gravity for swashplateless helicopter UAV and MAV

To investigate the use of centre of gravity location on reducing cyclic pitch control for helicopter UAV's (unmanned air vehicles) and MAV's (micro air vehicles). Low cyclic pitch is a necessity to implement the swashplateless rotor concept using trailing edge flaps or active twist using current generation low authority piezoceramic actuators. Design/methodology/approach – An aeroelastic analysis of the helicopter rotor with elastic blades is used to perform parametric and sensitivity studies of the effects of longitudinal and lateral center of gravity (cg) movements on the main rotor cyclic pitch. An optimization approach is then used to find cg locations which reduce the cyclic pitch at a given forward speed. Findings – It is found that the longitudinal cyclic pitch and lateral cyclic pitch can be driven to zero at a given forward speed by shifting the cg forward and to the port side, respectively. There also exist pairs of numbers for the longitudinal and lateral cg locations which drive both the cyclic pitch components to zero at a given forward speed. Based on these results, a compromise optimal cg location is obtained such that the cyclic pitch is bounded within ±5° for a BO105 helicopter rotor. Originality/value – The reduction in the cyclic pitch due to helicopter cg location is found to significantly reduce the maximum magnitudes of the control angles in flight, facilitating the swashplateless rotor concept. In addition, the existence of cg locations which drive the cyclic pitches to zero allows for the use of active cg movement as a way to replace the cyclic pitch control for helicopter MAV's.

Effect of pairing fluctuations on low-energy electronic spectra in cuprate superconductors

We describe here a minimal theory of tight-binding electrons moving on the square planar Cu lattice of the hole-doped cuprates and mixed quantum mechanically with their own Cooper pairs. The superconductivity occurring at the transition temperature T(c) is the long-range, d-wave symmetry phase coherence of these Cooper pairs. Fluctuations, necessarily associated with incipient long-range superconducting order, have a generic large-distance behavior near T(c). We calculate the spectral density of electrons coupled to such Cooper-pair fluctuations and show that features observed in angle resolved photoemission spectroscopy (ARPES) experiments on different cuprates above T(c) as a function of doping and temperature emerge naturally in this description. These include ``Fermi arcs'' with temperature-dependent length and an antinodal pseudogap, which fills up linearly as the temperature increases toward the pseudogap temperature. Our results agree quantitatively with experiment. Below T(c), the effects of nonzero superfluid density and thermal fluctuations are calculated and compared successfully with some recent ARPES experiments, especially the observed bending or deviation of the superconducting gap from the canonical d-wave form.

Relay Precoder Optimization in MIMO-Relay Networks With Imperfect CSI

In this paper, we consider robust joint designs of relay precoder and destination receive filters in a nonregenerative multiple-input multiple-output (MIMO) relay network. The network consists of multiple source-destination node pairs assisted by a MIMO-relay node. The channel state information (CSI) available at the relay node is assumed to be imperfect. We consider robust designs for two models of CSI error. The first model is a stochastic error (SE) model, where the probability distribution of the CSI error is Gaussian. This model is applicable when the imperfect CSI is mainly due to errors in channel estimation. For this model, we propose robust minimum sum mean square error (SMSE), MSE-balancing, and relay transmit power minimizing precoder designs. The next model for the CSI error is a norm-bounded error (NBE) model, where the CSI error can be specified by an uncertainty set. This model is applicable when the CSI error is dominated by quantization errors. In this case, we adopt a worst-case design approach. For this model, we propose a robust precoder design that minimizes total relay transmit power under constraints on MSEs at the destination nodes. We show that the proposed robust design problems can be reformulated as convex optimization problems that can be solved efficiently using interior-point methods. We demonstrate the robust performance of the proposed design through simulations.

Nature and Properties of a Repulsive Fermi Gas in the Upper Branch of the Energy Spectrum

We generalize the Nozieres-Schmitt-Rink method to study the repulsive Fermi gas in the absence of molecule formation, i.e., in the so-called ``upper branch.'' We find that the system remains stable except close to resonance at sufficiently low temperatures. With increasing scattering length, the energy density of the system attains a maximum at a positive scattering length before resonance. This is shown to arise from Pauli blocking which causes the bound states of fermion pairs of different momenta to disappear at different scattering lengths. At the point of maximum energy, the compressibility of the system is substantially reduced, leading to a sizable uniform density core in a trapped gas. The change in spin susceptibility with increasing scattering length is moderate and does not indicate any magnetic instability. These features should also manifest in Fermi gases with unequal masses and/or spin populations.

Fast-Group-Decodable STBCs via Codes over GF(4): Further Results

Recently in, a framework was given to construct low ML decoding complexity Space-Time Block Codes (STBCs) via codes over the finite field F4. In this paper, we construct new full-diversity STBCs with cubic shaping property and low ML decoding complexity via codes over F4 for number of transmit antennas N = 2m, m >; 1, and rates R >; 1 complex symbols per channel use. The new codes have the least ML decoding complexity among all known codes for a large set of (N, R) pairs. The new full-rate codes of this paper (R = N) are not only information-lossless and fully diverse but also have the least known ML decoding complexity in the literature. For N ≥ 4, the new full-rate codes are the first instances of full-diversity, information-lossless STBCs with low ML decoding complexity. We also give a sufficient condition for STBCs obtainable from codes over F4 to have cubic shaping property, and a sufficient condition for any design to give rise to a full-diversity STBC when the symbols are encoded using rotated square QAM constellations.

Additions to the z transform table

The ztransform method is a widely used tool for the analysis and synthesis of discrete systems. In this note a table of ztransform pairs when F(z) is an irrational function of z is given. The table is also useful for obtaining closed-form sums for some infinite series.

Topological analysis of general linear networks

A new method of network analysis, a generalization in several different senses of existing methods and applicable to all networks for which a branch-admittance (or impedance) matrix can be formed, is presented. The treatment of network determinants is very general and essentially four terminal rather than three terminal, and leads to simple expressions based on trees of a simple graph associated with the network and matrix, and involving products of low-order, usually(2 times 2)determinants of tree-branch admittances, in addition to tree-branch products as in existing methods. By comparison with existing methods, the total number of trees and of tree pairs is usually considerably reduced, and this fact, together with an easy method of tree-pair sign determination which is also presented, makes the new method simpler in general. The method can be very easily adapted, by the use of infinite parameters, to accommodate ideal transformers, operational amplifiers, and other forms of network constraint; in fact, is thought to be applicable to all linear networks.

Asymptotically-Optimal, Fast-Decodable, Full-Diversity STBCs

For a family/sequence of Space-Time Block Codes (STBCs) C1, C2,⋯, with increasing number of transmit antennas Ni, with rates Ri complex symbols per channel use (cspcu), i = 1,2,⋯, the asymptotic normalized rate is defined as limi→∞ Ri/Ni. A family of STBCs is said to be asymptotically-good if the asymptotic normalized rate is non-zero, i.e., when the rate scales as a non-zero fraction of the number of transmit antennas, and the family of STBCs is said to be asymptotically-optimal if the asymptotic normalized rate is 1, which is the maximum possible value. In this paper, we construct a new class of full-diversity STBCs that have the least maximum-likelihood (ML) decoding complexity among all known codes for any number of transmit antennas N>;1 and rates R>;1 cspcu. For a large set of (R,N) pairs, the new codes have lower ML decoding complexity than the codes already available in the literature. Among the new codes, the class of full-rate codes (R=N) are asymptotically-optimal and fast-decodable, and for N>;5 have lower ML decoding complexity than all other families of asymptotically-optimal, fast-decodable, full-diversity STBCs available in the literature. The construction of the new STBCs is facilitated by the following further contributions of this paper: (i) Construction of a new class of asymptotically-good, full-diversity multigroup ML decodable codes, that not only includes STBCs for a larger set of antennas, but also either matches in rate or contains as a proper subset all other high-rate or asymptotically-good, delay-optimal, multigroup ML decodable codes available in the literature. (ii) Construction of a new class of fast-group-decodable codes (codes that combine the low ML decoding complexity properties of multigroup ML decodable codes and fast-decodable codes) for all even number of transmit antennas and rates 1 <; R ≤ 5/4.- - (iii) Given a design with full-rank linear dispersion matrices, we show that a full-diversity STBC can be constructed from this design by encoding the real symbols independently using only regular PAM constellations.

Low ML Decoding Complexity STBCs via Codes Over the Klein Group

In this paper, we give a new framework for constructing low ML decoding complexity space-time block codes (STBCs) using codes over the Klein group K. Almost all known low ML decoding complexity STBCs can be obtained via this approach. New full- diversity STBCs with low ML decoding complexity and cubic shaping property are constructed, via codes over K, for number of transmit antennas N = 2(m), m >= 1, and rates R > 1 complex symbols per channel use. When R = N, the new STBCs are information- lossless as well. The new class of STBCs have the least knownML decoding complexity among all the codes available in the literature for a large set of (N, R) pairs.

An Algorithm for Testing 2-Asummability of Boolean Functions

Simple algorithms have been developed to generate pairs of minterms forming a given 2-sum and thereby to test 2-asummability of switching functions. The 2-asummability testing procedure can be easily implemented on the computer. Since 2-asummability is a necessary and sufficient condition for a switching function of upto eight variables to be linearly separable (LS), it can be used for testing LS switching functions of upto eight variables.

Investigation on the High Vacuum Tribological Characteristics of Surface Treated Nuclear Grade Stainless Steel Type AISI 316 LN at 25 to 500 degrees C

Although some researchers have published friction and wear data of Plasma Nitride (PN) coatings, the tribological behavior of PN/PN Pairs in high vacuum environment has not been published so far In order to bridge this knowledge gap, tribological tests under dry conditions have been conducted on PN/PN Pairs for varying temperatures of 25, 200, 400 and 500 degrees C in high vacuum (1.6 x 10(-4) bar) environment. The PN coatings showed good wear resistance layer on the ring surface. The PN coatings were removed only from the pin surface for all the tests since it contacts at a point. The friction and wear were low at lower temperatures and it eliminated adhesion between the contact surfaces until the coating was completely removed from the pin surface. (C) 2011 Journal of Mechanical Engineering. All rights reserved.

Structure of tandem and its implication for bifunctional intercalation into dna

Quinoxaline antibiotics (Fig. 1a, b) form a useful group of compounds for the study of drug–nucleic acid interactions1,2. They consist of a cross-bridged cyclic octadepsipeptide, variously modified, bearing two quinoxaline chromophores. These antibiotics intercalate bifunctionally into DNA2,3 probably via the narrow groove, forming a complex in which, most probably, two base pairs are sandwiched between the chromophores4,5. Depending on the nature of their sulphur-containing cross-bridge and modifications to their amino acid side chains, they display characteristic patterns of nucleotide sequence selectivity when binding to DNAs of different base composition and to synthetic polydeoxynucleotides4,6,7. This specificity has been tentatively ascribed to specific hydrogen-bonding interactions between functional groups in the DNA and complementary moieties on the peptide ring2,4,5. Variations in selectivity have been attributed both to changes in the conformation of the peptide backbone6 and no modifications of the cross-bridge7. These suggestions were made, however, in the absence of firm knowledge about the three-dimensional structure and conformation of the antibiotic molecules. We now report the X-ray structure analysis of the synthetic analogue of the antibiotic triostin A, TANDEM (des-N-tetramethyl triostin A) (Fig. 1c), which binds preferentially to alternating adenine-thymine sequences7. The X-ray structure provides a starting point for exploring the origin of this specificity and suggests possible models for the binding of other members of the quinoxaline series.

Unzipping and binding of small interfering RNA with single walled carbon nanotube: A platform for small interfering RNA delivery

In an effort to design efficient platform for siRNA delivery, we combine all atom classical and quantum simulations to study the binding of small interfering RNA (siRNA) by pristine single wall carbon nanotube (SWCNT). Our results show that siRNA strongly binds to SWCNT surface via unzipping its base-pairs and the propensity of unzipping increases with the increase in the diameter of the SWCNTs. The unzipping and subsequent wrapping events are initiated and driven by van der Waals interactions between the aromatic rings of siRNA nucleobases and the SWCNT surface. However, molecular dynamics (MD) simulations of double strand DNA (dsDNA) of the same sequence show that the dsDNA undergoes much less unzipping and wrapping on the SWCNT in the simulation time scale of 70 ns. This interesting difference is due to smaller interaction energy of thymidine of dsDNA with the SWCNT compared to that of uridine of siRNA, as calculated by dispersion corrected density functional theory (DFT) methods. After the optimal binding of siRNA to SWCNT, the complex is very stable which serves as one of the major mechanisms of siRNA delivery for biomedical applications. Since siRNA has to undergo unwinding process with the effect of RNA-induced silencing complex, our proposed delivery mechanism by SWCNT possesses potential advantages in achieving RNA interference. (C) 2012 American Institute of Physics. [http://dx.doi.org/10.1063/1.3682780]