Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

The role of lysine-256 in the structure and function of sheep liver recombinant serine hydroxymethyltransferase

The active site lysine residue, K256, involved in Schiffs base linkage with pyridoxal-5'-phosphate (PEP) in sheep liver recombinant serine hydroxymethyltransferase (rSHMT) was changed to glutamine or arginine by site-directed mutagenesis. The purified K256Q and K256R SHMTs had less than 0.1% of catalytic activity with serine and H(4)folate as substrates compared to rSHMT. The mutant enzymes also failed to exhibit the characteristic visible absorbance spectrum (lambda(max) 425 nm) and did not produce the quinonoid intermediate (lambda(max) 495 nm) upon the addition of glycine and H(4)folate. The mutant enzymes were unable to catalyze aldol cleavage of beta-phenylserine and transamination of D-alanine. These results suggested that the mutation of the lysine had resulted in the inability of the enzyme to bind to the cofactor. Therefore, the K256Q SHMT was isolated as a dimer and the K256R SHMT as a mixture of dimers and tetramers which were converted to dimers slowly. On the other hand, rSHMT was stable as a tetramer for several months, further confirming the role of PLP in maintenance of oligomeric structure. The mutant enzymes also failed to exhibit the increased thermal stability upon the addition of serine, normally observed with rSHMT. The enhanced thermal stability has been attributed to a change in conformation of the enzyme from open to closed form leading to reaction specificity. The mutant enzymes were unable to undergo this conformational change probably because of the absence of bound cofactor.

Velocity distribution function for a dilute granular material in shear flow

The velocity distribution function for the steady shear flow of disks (in two dimensions) and spheres (in three dimensions) in a channel is determined in the limit where the frequency of particle-wall collisions is large compared to particle-particle collisions. An asymptotic analysis is used in the small parameter epsilon, which is naL in two dimensions and na(2)L in three dimensions, where; n is the number density of particles (per unit area in two dimensions and per unit volume in three dimensions), L is the separation of the walls of the channel and a is the particle diameter. The particle-wall collisions are inelastic, and are described by simple relations which involve coefficients of restitution e(t) and e(n) in the tangential and normal directions, and both elastic and inelastic binary collisions between particles are considered. In the absence of binary collisions between particles, it is found that the particle velocities converge to two constant values (u(x), u(y)) = (+/-V, O) after repeated collisions with the wall, where u(x) and u(y) are the velocities tangential and normal to the wall, V = (1 - e(t))V-w/(1 + e(t)), and V-w and -V-w, are the tangential velocities of the walls of the channel. The effect of binary collisions is included using a self-consistent calculation, and the distribution function is determined using the condition that the net collisional flux of particles at any point in velocity space is zero at steady state. Certain approximations are made regarding the velocities of particles undergoing binary collisions :in order to obtain analytical results for the distribution function, and these approximations are justified analytically by showing that the error incurred decreases proportional to epsilon(1/2) in the limit epsilon --> 0. A numerical calculation of the mean square of the difference between the exact flux and the approximate flux confirms that the error decreases proportional to epsilon(1/2) in the limit epsilon --> 0. The moments of the velocity distribution function are evaluated, and it is found that [u(x)(2)] --> V-2, [u(y)(2)] similar to V-2 epsilon and -[u(x)u(y)] similar to V-2 epsilon log(epsilon(-1)) in the limit epsilon --> 0. It is found that the distribution function and the scaling laws for the velocity moments are similar for both two- and three-dimensional systems.

The prop-2-ynyloxy carbonyl function (POC): A new amino-protecting group removable from sulfur-containing peptides by ultrasonic irradiation with tetrathiomolybdate under mild and neutral conditions

The prop-2-ynyloxy carbonyl function (POC) which can be cleaved under mild and neutral conditions in the presence of benzyltriethylammonium tetrathiomolybdate has been developed as a new protecting group for amines. (C) 1999 Elsevier Science Ltd. All rights reserved.

Physics based basis function for vibration analysis of high speed rotating beams

The natural frequencies of continuous systems depend on the governing partial differential equation and can be numerically estimated using the finite element method. The accuracy and convergence of the finite element method depends on the choice of basis functions. A basis function will generally perform better if it is closely linked to the problem physics. The stiffness matrix is the same for either static or dynamic loading, hence the basis function can be chosen such that it satisfies the static part of the governing differential equation. However, in the case of a rotating beam, an exact closed form solution for the static part of the governing differential equation is not known. In this paper, we try to find an approximate solution for the static part of the governing differential equation for an uniform rotating beam. The error resulting from the approximation is minimized to generate relations between the constants assumed in the solution. This new function is used as a basis function which gives rise to shape functions which depend on position of the element in the beam, material, geometric properties and rotational speed of the beam. The results of finite element analysis with the new basis functions are verified with published literature for uniform and tapered rotating beams under different boundary conditions. Numerical results clearly show the advantage of the current approach at high rotation speeds with a reduction of 10 to 33% in the degrees of freedom required for convergence of the first five modes to four decimal places for an uniform rotating cantilever beam.

Vesicle formation from dimeric ion-paired amphiphiles. Control over vesicular thermotropic and ion-transport properties as a function of intra-amphiphilic headgroup separation

A novel series of vesicle-forming ion-paired amphiphiles, bis(hexadecyldimethylammonium)alkane dipalmitate (1a-1h), containing four chains were synthesized with two isolated headgroups. In each of these amphiphiles, the two headgroup charges are separated by a flexible polymethylene spacer chain -[(CH2)(m)]- of varying lengths (m) such that the length and the conformation of the spacer chain determine the intra-"monomer" headgroup separation. Transmission electron microscopy indicated that each of these forms bilayer membranes upon dispersion in aqueous media. The vesicular properties of these aggregates have been examined by differential scanning calorimetry and temperature-dependent fluorescence anisotropy measurements. Interestingly, their T-m values decreased with the increase in the m value. Thus while the apparent T-m of the lipid with m = 2 (1a) is 74.1 degrees C, the corresponding value observed for the lipid with m = 12 (1h) is 38.9 degrees C. The fluorescence anisotropy values (r) for 1b-1g were quite high (r similar to 0.3) compared to that of 1h (r similar to 0.23) at 20-30 degrees C in their gel states. On the other hand, the r value for vesicular 1b beyond melting was higher (0.1) compared to any of those for 1c-1h (similar to 0.04-0.06). X-ray diffraction of the cast films was performed to understand the nature and the thickness of these membrane organizations. The membrane widths ranged from 30 to 51 A as the m values varied. The entrapment of a small water-soluble solute, riboflavin, by the individual vesicular aggregates, and their sustenance: under an imposed transmembrane pH gradient have also been examined. These results show that all lipid vesicles entrap riboflavin and that generally the resistance to OH- permeation decreases with the increase in m value. Finally,all the above observations were comparatively analyzed, and on the basis of the calculated structures of these lipids, it was possible to conclude that membrane propel-ties can be modulated by spacer chain length variation of the ion-paired amphiphiles.

Structure, function, and mechanism of HhaI DNA methyltransferases

A vast amount of literature has accumulated on the characterization of DNA methyltransferases. The HhaI DNA methyltransferase, a C5-cytosine methyltransferase, has been the subject of investigation for the last 2 decades. Biochemical and kinetic characterization have led to an understanding of the catalytic and kinetic mechanism of the methyltransfer reaction. The HhaI methyltransferase has also been subjected to extensive structural analysis, with the availability of 12 structures with or without a cofactor and a variety of DNA substrates. The mechanism of base flipping, first described for the HhaI methyltransferase, is conserved among all DNA methyltransferases and is also found to occur in numerous DNA repair enzymes. Studies with other methyltransferase reveal a significant structural and functional similarity among different types of methyltransferases. This review aims to summarize the available information on the HhaI DNA methyltransferase.

Localisation of discrete change in a transformer winding: a network-function-loci approach

An entirely different approach for localisation of winding deformation based on terminal measurements is presented. Within the context of this study, winding deformation means, a discrete and specific change externally imposed at a particular position on the winding. The proposed method is based on pre-computing and plotting the complex network-function loci e.g. driving-point impedance (DPI)] at a selected frequency, for a meaningful range of values for each element (increasing and decreasing) of the ladder network which represents the winding. This loci diagram is called the nomogram. After introducing a discrete change, amplitude and phase of DPI are measured. By plotting this single measurement on the nomogram, it is possible to estimate the location and identify the extent of change. In contrast to the existing approach, the proposed method is fast, non-iterative and yields reasonably good localisation. Experimental results for actual transformer windings (interleaved and continuous disc) are presented.

The role of competition and habitat in structuring small mammal communities in a tropical montane ecosystem in southern India

Small mammals were sampled in two natural habitats (montane stunted evergreen forests and montane grassland) and four anthropogenic habitats (tea, wattle, bluegum and pine plantation) in the Upper Nilgiris in southern India. Of the species trapped, eight were in montane evergreen forests and three were in other habitats. Habitat discrimination was studied in the rodents Rattus rattus and Mus famulus and the shrew Suncus montanus in the montane forest habitat. Multivariate tests on five variables (canopy cover, midstorey density, ground cover, tree density, canopy height) showed that R. rattus uses areas of higher tree density and lower canopy cover. Suncus montanus and M. famulus use habitat with higher tree density and ground cover and lower canopy height. Multivariate tests did not discriminate habitat use between the species. Univariate tests, however, showed that M. famulus uses areas of higher tree density than R. rattus and S. montanus. Rattus rattus was the dominant species in the montane forest, comprising 60.9% of total density, while the rodent Millardia meltada was the dominant species in the grassland. Studies of spatial interaction between these two species in habitats where they coexisted showed neither overlap nor avoidance between the species. Rattus rattus, however, did use areas of lower ground cover than did M. meltada. The analysis of spatial interactions between the species, habitat discrimination and use, and the removal experiments suggest that interspecific competition may not be a strong force in structuring these small mammal communities. There are distinct patterns in the use of different habitats by some species, but microhabitat selection and segregation is weak. Other factors such as intraspecific competition may play a more important role in these communities.

Special Issue: Engineering applications of representations of function, Part 1

In engineering design, the end goal is the creation of an artifact, product, system, or process that fulfills some functional requirements at some desired level of performance. As such, knowledge of functionality is essential in a wide variety of tasks in engineering activities, including modeling, generation, modification, visualization, explanation, evaluation, diagnosis, and repair of these artifacts and processes. A formal representation of functionality is essential for supporting any of these activities on computers. The goal of Parts 1 and 2 of this Special Issue is to bring together the state of knowledge of representing functionality in engineering applications from both the engineering and the artificial intelligence (AI) research communities.

The carboxy terminal WD domain of the pre-mRNA splicing factor Prp17p is critical for function

In Saccharomyces cerevisiae, Prp17p is required for the efficient completion of the second step of pre-mRNA splicing. The function and interacting factors for this protein have not been elucidated. We have performed a mutational analysis of yPrp17p to identify protein domains critical for function. A series of deletions were made throughout the region spanning the N-terminal 158 amino acids of the protein, which do not contain any identified structural motifs. The C-terminal portion (amino acids 160–455) contains a WD domain containing seven WD repeats. We determined that a minimal functional Prp17p consists of the WD domain and 40 amino acids N-terminal to it. We generated a three-dimensional model of the WD repeats in Prp17p based on the crystal structure of the [beta]-transducin WD domain. This model was used to identify potentially important amino acids for in vivo functional characterization. Through analysis of mutations in four different loops of Prp17p that lie between [beta] strands in the WD repeats, we have identified four amino acids, 235TETG238, that are critical for function. These amino acids are predicted to be surface exposed and may be involved in interactions that are important for splicing. Temperature-sensitive prp17 alleles with mutations of these four amino acids are defective for the second step of splicing and are synthetically lethal with a U5 snRNA loop I mutation, which is also required for the second step of splicing. These data reinforce the functional significance of this region within the WD domain of Prp17p in the second step of splicing.

Environment Education for Ecosystem Conservation

Conservation of natural resources through sustainable ecosystem management and development is the key to our secured future. The management of ecosystem involves inventorying and monitoring, and applying integrated technologies, methodologies and interdisciplinary approaches for its conservation. Hence, now it is even more critical than ever before for the humans to be environmentally literate. To realise this vision, both ecological and environmental education must become a fundamental part of the education system at all levels of education. Currently, it is even more critical than ever before for the humankind as a whole to have a clear understanding of environmental concerns and to follow sustainable development practices. The degradation of our environment is linked to continuing problems of pollution, loss of forest, solid waste disposal, and issues related to economic productivity and national as well as ecological security. Environmental management has gained momentum in the recent years with the initiatives focussing on managing environmental hazards and preventing possible disasters. Environmental issues make better sense, when one can understand them in the context of one’s own cognitive sphere. Environmental education focusing on real-world contexts and issues often begins close to home, encouraging learners to understand and forge connections with their immediate surroundings. The awareness, knowledge, and skills needed for these local connections and understandings provide a base for moving out into larger systems, broader issues, and a more sophisticated comprehension of causes, connections, and consequences. Environmental Education Programme at CES in collaboration with Karnataka Environment Research Foundation (KERF) referred as ‘Know your Ecosystem’ focuses on the importance of investigating the ecosystems within the context of human influences, incorporating an examination of ecology, economics, culture, political structure, and social equity as well as natural processes and systems. The ultimate goal of environment education is to develop an environmentally literate public. It needs to address the connection between our conception and practice of education and our relationship as human cultures to life-sustaining ecological systems. For each environmental issue there are many perspectives and much uncertainty. Environmental education cultivates the ability to recognise uncertainty, envision alternative scenarios, and adapt to changing conditions and information. These knowledge, skills, and mindset translate into a citizenry who is better equipped to address its common problems and take advantage of opportunities, whether environmental concerns are involved or not.

Nitrogen and carbon flows through Belandur lake - Role of Bellandur lake as a natural wetland treating Bangalore wastewater

Much of the Bangalore sewage is treated in three streams namely Bellandur (K&C Valley),Vrishabhavati and Hebbal-Nagavara stream systems. Among these it is estimated that out of a total of about 500MLD of partially treated sewage is let into the Bellandur tank. We estimate that a total of about 77t N non-industrial anthropogenic nitrogen efflux (mainly urine and excreta) in Bangalore city. This is distributed between that handled by the three sewage streams, soak-pits and land deposition. About 17-24.5t N enters the Bellandur tank daily. This has been happening over few decades and our observations suggest that this approximately 380ha tank is functioning as a C and N removal system with reasonable efficiency. The ammoniacal and nitrate nitrogen content of the water at the discharge points were estimated and found that over 80% of the nitrogen influx and over 75% of the C influx is removed by this tank system. We observed that there are three nitrogen sinks namely bacterial, micro-algal and macrophytes. The micro-algal fraction is dominated by Microcystis and Euglenophyceae members and they appear to constitute a significant fraction. Water hyacinth represents the single largest representative of the macrophytes. This tank has been functioning in this manner for over three decades. We attempt to study this phenomenon from a material balance approach and show that it is functioning with a reasonable degree of satisfaction as a natural wetland. As the population served and concomitant influx into this wetland increases, there is a potential for the system to be overloaded and to collapse. Therefore a better understanding of its function and the need for maintenance is discussed in the paper.

Biomonitoring of Lotic habitats through Diatoms

Freshwater ecosystems vary in size and composition and contain a wide range of organisms which interact with each other and with the environment. These interactions are between organisms and the environment as nutrient cycling, biomass formation and transfer, maintenance of internal environment and interactions with the external environment. The range of organisms present in aquatic communities decides the generation and transfer function of biomass, which defines and characterises the system. These organisms have distinct roles as they occupy particular trophic levels, forming an interconnected system in a food chain. Availability of resources and competition would primarily determine the balance of individual species within the food web, which in turn influences the variety and proportions of the different organisms, with important implications for the overall functioning of the system. This dynamic and diverse relationship decides the physical, chemical and biological elements across spatial and temporal scales in the aquatic ecosystem, which can be recorded by regular inventorying and monitoring to maintain the integrity and conserve the ecosystem. Regular environmental monitoring, particularly water quality monitoring allows us to detect, assess and manage the overall impacts on the rivers. The appreciation of water quality is in constant flux. Water quality assessments derived through the biotic indices, i.e. assessments based on observations of the resident floral and faunal communities has gained importance in recent years. Biological evaluations provide a description of the water quality that is often not achievable from elemental analyses alone. A biological indicator (or bioindicator) is a taxon or taxa selected based on its sensitivity to a particular attribute, and then assessed to make inferences about that attribute. In other words, they are a substitute for directly measuring abiotic features or other biota. Bioindicators are evaluated through presence or absence, condition, relative abundance, reproductive success, community structure (i.e. composition and diversity), community function (i.e. trophic structure), or any combination thereof.Biological communities reflect the overall ecological integrity by integrating various stresses, thus providing a broad measure of their synergistic impacts. Aquatic communities, both plants and animals, integrate and reflect the effects of chemical and physical disturbances that occur over extended periods of time. Monitoring procedures based on the biota measure the health of a river and the ability of aquatic ecosystems to support life as opposed to simply characterising the chemical and physical components of a particular system. This is the central purpose of assessing the biological condition of aquatic communities of a river.Diatoms (Bacillariophyceae), blue green algae (Cyanophyceae), green algae (Chlorophyceae), and red algae (Rhodphyceae) are the main groups of algae in flowing water. These organisms are widely used as biological indicators of environmental health in the aquatic ecosystem because algae occupy the most basic level in the transfer of energy through natural aquatic systems. The distribution of algae in an aquatic ecosystem is directly related to the fundamental factors such as physical, chemical and biological constituents. Soft algae (all the algal groups except diatoms) have also been used as indicators of biological integrity, but they may have less efficiency than diatoms in this respect due to their highly variable morphology. The diatoms (Bacillariophyceae) comprise a ubiquitous, highly successful and distinctive group of unicellular algae with the most obvious distinguishing characteristic feature being siliceous cell walls (frustules). The photosynthetic organisms living within its photic zone are responsible for about one-half of global primary productivity. The most successful organisms are thought to be photosynthetic prokaryotes (cyanobacteria and prochlorophytes) and a class of eukaryotic unicellular algae known as diatoms. Diatoms are likely to have arisen around 240 million years ago following an endosymbiotic event between a red eukaryotic alga and a heterotrophic flagellate related to the Oomycetes.The importance of algae to riverine ecology is easily appreciated when one considers that they are primary producers that convert inorganic nutrients into biologically active organic compounds while providing physical habitat for other organisms. As primary producers, algae transform solar energy into food from which many invertebrates obtain their energy. Algae also transform inorganic nutrients, such as atmospheric nitrogen into organic forms such as ammonia and amino acids that can be used by other organisms. Algae stabilises the substrate and creates mats that form structural habitats for fish and invertebrates. Algae are a source of organic matter and provide habitat for other organisms such as non-photosynthetic bacteria, protists, invertebrates, and fish. Algae's crucial role in stream ecosystems and their excellent indicator properties make them an important component of environmental studies to assess the effects of human activities on stream health. Diatoms are used as biological indicators for a number of reasons: 1. They occur in all types of aquatic ecosystems. 2. They collectively show a broad range of tolerance along a gradient of aquatic productivity, individual species have specific water chemistry requirements. 3. They have one of the shortest generation times of all biological indicators (~2 weeks). They reproduce and respond rapidly to environmental change and provide early measures of both pollution impacts and habitat restoration. 4. It takes two to three weeks before changes are reflected to a measurable extent in the assemblage composition.