Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

The role of lysine-256 in the structure and function of sheep liver recombinant serine hydroxymethyltransferase

The active site lysine residue, K256, involved in Schiffs base linkage with pyridoxal-5'-phosphate (PEP) in sheep liver recombinant serine hydroxymethyltransferase (rSHMT) was changed to glutamine or arginine by site-directed mutagenesis. The purified K256Q and K256R SHMTs had less than 0.1% of catalytic activity with serine and H(4)folate as substrates compared to rSHMT. The mutant enzymes also failed to exhibit the characteristic visible absorbance spectrum (lambda(max) 425 nm) and did not produce the quinonoid intermediate (lambda(max) 495 nm) upon the addition of glycine and H(4)folate. The mutant enzymes were unable to catalyze aldol cleavage of beta-phenylserine and transamination of D-alanine. These results suggested that the mutation of the lysine had resulted in the inability of the enzyme to bind to the cofactor. Therefore, the K256Q SHMT was isolated as a dimer and the K256R SHMT as a mixture of dimers and tetramers which were converted to dimers slowly. On the other hand, rSHMT was stable as a tetramer for several months, further confirming the role of PLP in maintenance of oligomeric structure. The mutant enzymes also failed to exhibit the increased thermal stability upon the addition of serine, normally observed with rSHMT. The enhanced thermal stability has been attributed to a change in conformation of the enzyme from open to closed form leading to reaction specificity. The mutant enzymes were unable to undergo this conformational change probably because of the absence of bound cofactor.

Velocity distribution function for a dilute granular material in shear flow

The velocity distribution function for the steady shear flow of disks (in two dimensions) and spheres (in three dimensions) in a channel is determined in the limit where the frequency of particle-wall collisions is large compared to particle-particle collisions. An asymptotic analysis is used in the small parameter epsilon, which is naL in two dimensions and na(2)L in three dimensions, where; n is the number density of particles (per unit area in two dimensions and per unit volume in three dimensions), L is the separation of the walls of the channel and a is the particle diameter. The particle-wall collisions are inelastic, and are described by simple relations which involve coefficients of restitution e(t) and e(n) in the tangential and normal directions, and both elastic and inelastic binary collisions between particles are considered. In the absence of binary collisions between particles, it is found that the particle velocities converge to two constant values (u(x), u(y)) = (+/-V, O) after repeated collisions with the wall, where u(x) and u(y) are the velocities tangential and normal to the wall, V = (1 - e(t))V-w/(1 + e(t)), and V-w and -V-w, are the tangential velocities of the walls of the channel. The effect of binary collisions is included using a self-consistent calculation, and the distribution function is determined using the condition that the net collisional flux of particles at any point in velocity space is zero at steady state. Certain approximations are made regarding the velocities of particles undergoing binary collisions :in order to obtain analytical results for the distribution function, and these approximations are justified analytically by showing that the error incurred decreases proportional to epsilon(1/2) in the limit epsilon --> 0. A numerical calculation of the mean square of the difference between the exact flux and the approximate flux confirms that the error decreases proportional to epsilon(1/2) in the limit epsilon --> 0. The moments of the velocity distribution function are evaluated, and it is found that [u(x)(2)] --> V-2, [u(y)(2)] similar to V-2 epsilon and -[u(x)u(y)] similar to V-2 epsilon log(epsilon(-1)) in the limit epsilon --> 0. It is found that the distribution function and the scaling laws for the velocity moments are similar for both two- and three-dimensional systems.

Mean first passage time approach to the problem of optimal barrier subdivision for Kramer's escape rate

We consider the effect of subdividing the potential barrier along the reaction coordinate on Kramer's escape rate for a model potential, Using the known supersymmetric potential approach, we show the existence of an optimal number of subdivisions that maximizes the rate, We cast the problem as a mean first passage time problem of a biased random walker and obtain equivalent results, We briefly summarize the results of our investigation on the increase in the escape rate by placing a blow-torch in the unstable part of one of the potential wells. (C) 1999 Elsevier Science B.V. All rights reserved.

The prop-2-ynyloxy carbonyl function (POC): A new amino-protecting group removable from sulfur-containing peptides by ultrasonic irradiation with tetrathiomolybdate under mild and neutral conditions

The prop-2-ynyloxy carbonyl function (POC) which can be cleaved under mild and neutral conditions in the presence of benzyltriethylammonium tetrathiomolybdate has been developed as a new protecting group for amines. (C) 1999 Elsevier Science Ltd. All rights reserved.

Thermal barrier coating application of zircon sand

Naturally occurring zircon sand was plasma spray coated on steel substrates previously coated with NiCrAlY bond coat. The coatings were characterized for their microstructure, chemical composition, thermal shock resistance, and the nature of structural phases present, The as-sprayed coatings consisted of t-ZrO2 (major phase), m-ZrO2, ZrSiO4 (minor phases), and amorphous SiO2. These coatings, when annealed at 1200 degrees C/1.44 x 10(4) s yielded a ZrSiO4 phase as a result of the reaction between ZrO2 and SiO2, Dramatic changes occurred in the characteristics of the coatings when a mixture of zircon sand and Y2O3 was plasma spray coated and annealed at 1400 degrees C/1.44 x 10(4) s, The t-ZrO2 phase was completely stabilized, and these coatings were found to have considerable potential for thermal barrier applications.

Physics based basis function for vibration analysis of high speed rotating beams

The natural frequencies of continuous systems depend on the governing partial differential equation and can be numerically estimated using the finite element method. The accuracy and convergence of the finite element method depends on the choice of basis functions. A basis function will generally perform better if it is closely linked to the problem physics. The stiffness matrix is the same for either static or dynamic loading, hence the basis function can be chosen such that it satisfies the static part of the governing differential equation. However, in the case of a rotating beam, an exact closed form solution for the static part of the governing differential equation is not known. In this paper, we try to find an approximate solution for the static part of the governing differential equation for an uniform rotating beam. The error resulting from the approximation is minimized to generate relations between the constants assumed in the solution. This new function is used as a basis function which gives rise to shape functions which depend on position of the element in the beam, material, geometric properties and rotational speed of the beam. The results of finite element analysis with the new basis functions are verified with published literature for uniform and tapered rotating beams under different boundary conditions. Numerical results clearly show the advantage of the current approach at high rotation speeds with a reduction of 10 to 33% in the degrees of freedom required for convergence of the first five modes to four decimal places for an uniform rotating cantilever beam.

Vesicle formation from dimeric ion-paired amphiphiles. Control over vesicular thermotropic and ion-transport properties as a function of intra-amphiphilic headgroup separation

A novel series of vesicle-forming ion-paired amphiphiles, bis(hexadecyldimethylammonium)alkane dipalmitate (1a-1h), containing four chains were synthesized with two isolated headgroups. In each of these amphiphiles, the two headgroup charges are separated by a flexible polymethylene spacer chain -[(CH2)(m)]- of varying lengths (m) such that the length and the conformation of the spacer chain determine the intra-"monomer" headgroup separation. Transmission electron microscopy indicated that each of these forms bilayer membranes upon dispersion in aqueous media. The vesicular properties of these aggregates have been examined by differential scanning calorimetry and temperature-dependent fluorescence anisotropy measurements. Interestingly, their T-m values decreased with the increase in the m value. Thus while the apparent T-m of the lipid with m = 2 (1a) is 74.1 degrees C, the corresponding value observed for the lipid with m = 12 (1h) is 38.9 degrees C. The fluorescence anisotropy values (r) for 1b-1g were quite high (r similar to 0.3) compared to that of 1h (r similar to 0.23) at 20-30 degrees C in their gel states. On the other hand, the r value for vesicular 1b beyond melting was higher (0.1) compared to any of those for 1c-1h (similar to 0.04-0.06). X-ray diffraction of the cast films was performed to understand the nature and the thickness of these membrane organizations. The membrane widths ranged from 30 to 51 A as the m values varied. The entrapment of a small water-soluble solute, riboflavin, by the individual vesicular aggregates, and their sustenance: under an imposed transmembrane pH gradient have also been examined. These results show that all lipid vesicles entrap riboflavin and that generally the resistance to OH- permeation decreases with the increase in m value. Finally,all the above observations were comparatively analyzed, and on the basis of the calculated structures of these lipids, it was possible to conclude that membrane propel-ties can be modulated by spacer chain length variation of the ion-paired amphiphiles.

Dielectric barrier discharge plasma interaction with the airflow over an airfoil

The effect of various parameters on the velocity of the induced jet produced by a dielectric barrier discharge (DBD) plasma was studied experimentally. The glow discharge was created at atmospheric conditions by using a high voltage RF power supply. Flow visualization and photographic studies of the plasma were performed. The parametric investigation of the characteristics of the plasma show that the width of the plasma in the uniform glow discharge regime was an indication of the velocity induced under stagnant conditions. It was observed that the spanwise overlap of the two electrodes, dielectric thickness, voltage and frequency of the applied voltage are the major parameters that govern the velocity and the extent of plasma in the streamwise direction.

Woody plant seedling distribution under invasive Lantana camara thickets in a dry-forest plot in Mudumalai, southern India

Lantana camara, a shrub of Central and South American origin, has become invasive across dry forests worldwide. The effect of the thicket-forming habit of L. camara as a dispersal and recruitment barrier in a community of native woody seedlings was examined in a 50-ha permanent plot located in the seasonally dry forest of Mudumalai, southern India. Sixty 100-m(2) plots were enumerated for native woody seedlings between 10-100 cm in height. Of these, 30 plots had no L. camara thickets, while the other 30 had dense thickets. The frequency of occurrence and abundance of seedlings were modelled as a function of dispersal mode (mammal, bird or mechanical) and affinities to forest habitats (dry forest, moist forest or ubiquitous) as well as presence or absence of dense L. camara thickets. Furthermore, frequency of occurrence and abundance of individual species were also compared between thickets and no L. camara. At the community level, L. camara density, dispersal mode and forest habitat affinities of species determined both frequency of occurrence and abundance of seedlings, with the abundance of dry-forest mammal-dispersed species and ubiquitous mechanically dispersed species being significantly lower under L. camara thickets. Phyllanthus emblica and Kydia calycina were found to be significantly less abundant under L. camara, whereas most other species were not affected by the presence of thickets. It was inferred that, by affecting the establishment of native tree seedlings, L. camara thickets could eventually alter the community composition of such forests.

Structure, function, and mechanism of HhaI DNA methyltransferases

A vast amount of literature has accumulated on the characterization of DNA methyltransferases. The HhaI DNA methyltransferase, a C5-cytosine methyltransferase, has been the subject of investigation for the last 2 decades. Biochemical and kinetic characterization have led to an understanding of the catalytic and kinetic mechanism of the methyltransfer reaction. The HhaI methyltransferase has also been subjected to extensive structural analysis, with the availability of 12 structures with or without a cofactor and a variety of DNA substrates. The mechanism of base flipping, first described for the HhaI methyltransferase, is conserved among all DNA methyltransferases and is also found to occur in numerous DNA repair enzymes. Studies with other methyltransferase reveal a significant structural and functional similarity among different types of methyltransferases. This review aims to summarize the available information on the HhaI DNA methyltransferase.

Localisation of discrete change in a transformer winding: a network-function-loci approach

An entirely different approach for localisation of winding deformation based on terminal measurements is presented. Within the context of this study, winding deformation means, a discrete and specific change externally imposed at a particular position on the winding. The proposed method is based on pre-computing and plotting the complex network-function loci e.g. driving-point impedance (DPI)] at a selected frequency, for a meaningful range of values for each element (increasing and decreasing) of the ladder network which represents the winding. This loci diagram is called the nomogram. After introducing a discrete change, amplitude and phase of DPI are measured. By plotting this single measurement on the nomogram, it is possible to estimate the location and identify the extent of change. In contrast to the existing approach, the proposed method is fast, non-iterative and yields reasonably good localisation. Experimental results for actual transformer windings (interleaved and continuous disc) are presented.

High-pressure studies on lithium fast-ion conductors

The variation of resistivity of the lithium fast-ion conductor Li3+y Ge1−yO4 (y = 0.25, 0.6, 0.72) has been studied with hydrostatic pressure up to 70 kbar and compared with that of Li16−2x Znx (GeO4)4(x = 1, 2). Both types showed pronounced resistivity maxima between 20–30 kbar and marked decrease thereafter. Measurements as a function of temperature between 120–300 K permitted the determination of activation energies and prefactors that also showed corresponding maxima. The activation volumes (ΔV) of the first type of compound varied between 4.34 to −4.90 cm3/mol at 300 K and decreased monotonically with increasing temperature. For the second type ΔV was much smaller, varied with pressure between 0.58 and −0.24 cm3/mol, and went through a maximum with increasing temperature. High-pressure studies were also conducted on aged samples, and the results are discussed in conjunction with results of impedance measurements and nuclear magnetic resonance (NMR) studies. The principal effect of pressure appears to be variations of the sum of interatomic potentials and hence barrier height, which also causes significant changes in entropy.

Epitaxial LaNiO3 thin films: A normal metal barrier for SNS junction

Epitaxial LaNiO3 thin films have been grown on SrTiO3 and several other substrates by pulsed laser deposition. The films are observed to be metallic down to 15 K, and the temperature dependence of resistivity is similar to that of bulk LaNiO3. Epitaxial, c-axis oriented YBa2Cu3O7-x films with good superconducting properties have been grown on the LaNiO3 (100) films. I-V characteristics of the YBa2Cu3O7-x-LaNiO3 junction are linear, indicating ohmic contact between them.