Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Signal compression through spatial frequency-based motion estimation

The interest in low bit rate video coding has increased considerably. Despite rapid progress in storage density and digital communication system performance, demand for data-transmission bandwidth and storage capacity continue to exceed the capabilities of available technologies. The growth of data-intensive digital audio, video applications and the increased use of bandwidth-limited media such as video conferencing and full motion video have not only sustained the need for efficient ways to encode analog signals, but made signal compression central to digital communication and data-storage technology. In this paper we explore techniques for compression of image sequences in a manner that optimizes the results for the human receiver. We propose a new motion estimator using two novel block match algorithms which are based on human perception. Simulations with image sequences have shown an improved bit rate while maintaining ''image quality'' when compared to conventional motion estimation techniques using the MAD block match criteria.

Secondary compression factor

When the variation of secondary compression, with log(10) t is non-linear, the quantification of secondary settlement through the coefficient of secondary compression, C-alpha epsilon, becomes difficult which frequently leads to an underestimate of the settlement, Log(10) delta - log(10) t representation of such true-compression data has the distinct advantage of exhibiting linear secondary compression behaviour over an appreciably larger time span. The slope of the secondary compression portion of the log(10) e - log(10) t curve expressed as Delta(log e)/(log t) and called the 'secondary compression factor', m, proves to be a better alternative to C-alpha epsilon and the prediction of secondary settlement is improved.

On the characterisation of syntactic foam core sandwich composites for compressive properties

Sandwich structures, especially those with honeycomb and grid structures as the core material, are very commonly employed in aircraft structures. There is an increasing use of closed-pore rigid syntactic foams as core materials in sandwich constructions because they possess a number of favourable properties. The syntactic foams, owing to their structure and formation, behave differently under compression compared to other traditionally used core materials. In the present study, therefore, syntactic foam core sandwich constructions are evaluated for their behaviour under compression in both edgewise and flatwise orientations. Further, the work characterises the relative performance of two sets of sandwich materials, one containing glass-epoxy and the other, glass/carbon hybrid-epoxy skins. As non-standard geometry test specimens were involved, only a comparative evaluation was contemplated in this approach. The experiments indicate that the nature of the reinforcement fabric in the skin has a bearing on the test results in edgewise orientation. Thus, the tendency towards initiation of vertical crack in the central plane of the core material, which is a typical fracture event in this kind of material, was found to occur after a delay for the specimens containing the glass fabric in the skin. Attempts are made to establish the correlation between observations made on the test specimen visually during the course of testing and the post-compression microscopic examinations of the fracture features.

Starbursts triggered by cloud compression in interacting galaxies

We propose a physical mechanism for the triggering of starbursts in interacting spiral galaxies by shock compression of the pre-existing disk giant molecular clouds (GMCs). We show that as a disk GMC tumbles into the central region of a galaxy following a galactic tidal encounter, it undergoes a radiative shock compression by the pre-existing high pressure of the central molecular intercloud medium. The shocked outer shell of a GMC becomes gravitationally unstable, which results in a burst of star formation in the initially stable GMC. In the case of colliding galaxies with physical overlap such as Arp 244, the cloud compression is shown to occur due to the hot, high-pressure remnant gas resulting from the collisions of atomic hydrogen gas clouds from the two galaxies. The resulting values of infrared luminosity agree with observations. The main mode of triggered star formation is via clusters of stars, thus we can naturally explain the formation of young, luminous star clusters observed in starburst galaxies.

Studies on the variations in compression strengths of graphite powders bearing glass-epoxy composites exposed to humid conditions

The moisture absorption and changes in compression strengths in glass-epoxy (G-E composites without and with discrete quantities of graphite powders introduced into the resin mix prior to its spreading on specific glass fabric (layers) during the lay-up (stacking) sequence forms the subject matter of this report. The results point to higher moisture absorption for graphite bearing specimens. The strengths of graphite-free coupons show a continuous decrease, while the filler bearing ones show an initial rise followed by a drop for larger exposure times. Scanning Fractographic features were examined for an understanding of the process. The observations were explained invoking the effect of matrix plasticizing and the role of interfacial regions.

Wavelet based compression and denoising of optical tomography data

Two methods based on wavelet/wavelet packet expansion to denoise and compress optical tomography data containing scattered noise are presented, In the first, the wavelet expansion coefficients of noisy data are shrunk using a soft threshold. In the second, the data are expanded into a wavelet packet tree upon which a best basis search is done. The resulting coefficients are truncated on the basis of energy content. It can be seen that the first method results in efficient denoising of experimental data when scattering particle density in the medium surrounding the object was up to 12.0 x 10(6) per cm(3). This method achieves a compression ratio of approximate to 8:1. The wavelet packet based method resulted in a compression of up to 11:1 and also exhibited reasonable noise reduction capability. Tomographic reconstructions obtained from denoised data are presented. (C) 1999 Published by Elsevier Science B.V. All rights reserved,

A finite element assessment of flexural strength of prestressed concrete beams with fiber reinforcement

This paper presents an assessment of the flexural behavior of 15 fully/partially prestressed high strength concrete beams containing steel fibers investigated using three-dimensional nonlinear finite elemental analysis. The experimental results consisted of eight fully and seven partially prestressed beams, which were designed to be flexure dominant in the absence of fibers. The main parameters varied in the tests were: the levels of prestressing force (i.e, in partially prestressed beams 50% of the prestress was reduced with the introduction of two high strength deformed bars instead), fiber volume fractions (0%, 0.5%, 1.0% and 1.5%), fiber location (full depth and partial depth over full length and half the depth over the shear span only). A three-dimensional nonlinear finite element analysis was conducted using ANSYS 5.5 [Theory Reference Manual. In: Kohnke P, editor. Elements Reference Manual. 8th ed. September 1998] general purpose finite element software to study the flexural behavior of both fully and partially prestressed fiber reinforced concrete beams. Influence of fibers on the concrete failure surface and stress-strain response of high strength concrete and the nonlinear stress-strain curves of prestressing wire and deformed bar were considered in the present analysis. In the finite element model. tension stiffening and bond slip between concrete and reinforcement (fibers., prestressing wire, and conventional reinforcing steel bar) have also been considered explicitly. The fraction of the entire volume of the fiber present along the longitudinal axis of the prestressed beams alone has been modeled explicitly as it is expected that these fibers would contribute to the mobilization of forces required to sustain the applied loads across the crack interfaces through their bridging action. A comparison of results from both tests and analysis on all 15 specimens confirm that, inclusion of fibers over a partial depth in the tensile side of the prestressed flexural structural members was economical and led to considerable cost saving without sacrificing on the desired performance. However. beams having fibers over half the depth in only the shear span, did not show any increase in the ultimate load or deformational characteristics when compared to plain concrete beams. (C) 2002 Published by Elsevier Science Ltd.

A high performance scheme for EEG compression using a multichannel model

The amount of data contained in electroencephalogram (EEG) recordings is quite massive and this places constraints on bandwidth and storage. The requirement of online transmission of data needs a scheme that allows higher performance with lower computation. Single channel algorithms, when applied on multichannel EEG data fail to meet this requirement. While there have been many methods proposed for multichannel ECG compression, not much work appears to have been done in the area of multichannel EEG. compression. In this paper, we present an EEG compression algorithm based on a multichannel model, which gives higher performance compared to other algorithms. Simulations have been performed on both normal and pathological EEG data and it is observed that a high compression ratio with very large SNR is obtained in both cases. The reconstructed signals are found to match the original signals very closely, thus confirming that diagnostic information is being preserved during transmission.

A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.

Effect of strain rate on tensile and compression behaviour of Al-Si/graphite composite

Effects of strain rate (10(-4)-10(-2) s(-1)) on tensile and compressive strength of the Al-Si alloy and Al-Si/graphite composite are investigated. The strain hardening exponent value of the composite was more than that of the alloy for all strain rates during tensile and compressive loading. The yield stress of the composite was more than that of the ultimate tensile strength of the alloy for all strain rates. Tensile and compressive properties of the alloy and composite are dependent on strain rates. The negative strain rate sensitivity was observed for the composite and alloy at lower strain rates during the compression and tension loading respectively. (C) 2011 Elsevier B.V. All rights reserved.

Efficient statistical modeling for the compression of tree structured intermediate code

We propose a scheme for the compression of tree structured intermediate code consisting of a sequence of trees specified by a regular tree grammar. The scheme is based on arithmetic coding, and the model that works in conjunction with the coder is automatically generated from the syntactical specification of the tree language. Experiments on data sets consisting of intermediate code trees yield compression ratios ranging from 2.5 to 8, for file sizes ranging from 167 bytes to 1 megabyte.

Exergetic analysis of carbon dioxide vapour compression refrigeration cycle using the new fundamental equation of state

The purpose of this paper is to present exergy charts for carbon dioxide (CO2) based on the new fundamental equation of state and the results of a thermodynamic analysis of conventional and trans-critical vapour compression refrigeration cycles using the data thereof. The calculation scheme is anchored on the Mathematica platform. There exist upper and lower bounds for the high cycle pressure for a given set of evaporating and pre-throttling temperatures. The maximum possible exergetic efficiency for each case was determined. Empirical correlations for exergetic efficiency and COP, valid in the range of temperatures studied here, are obtained. The exergy losses have been quantified. (C) 2003 Elsevier Ltd. All rights reserved.

Guessing and compression subject to distortion

The problem of guessing a random string is revisited. The relation-ship between guessing without distortion and compression is extended to the case when source alphabet size is countably in¯nite. Further, similar relationship is established for the case when distortion allowed by establishing a tight relationship between rate distortion codes and guessing strategies.