Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Signal compression through spatial frequency-based motion estimation

The interest in low bit rate video coding has increased considerably. Despite rapid progress in storage density and digital communication system performance, demand for data-transmission bandwidth and storage capacity continue to exceed the capabilities of available technologies. The growth of data-intensive digital audio, video applications and the increased use of bandwidth-limited media such as video conferencing and full motion video have not only sustained the need for efficient ways to encode analog signals, but made signal compression central to digital communication and data-storage technology. In this paper we explore techniques for compression of image sequences in a manner that optimizes the results for the human receiver. We propose a new motion estimator using two novel block match algorithms which are based on human perception. Simulations with image sequences have shown an improved bit rate while maintaining ''image quality'' when compared to conventional motion estimation techniques using the MAD block match criteria.

Nonlinear electrical conduction and broad band noise in the charge-ordered rare earth manganate Nd0.5Ca0.5MnO3

Measurements of the dc transport properties and the low-frequency conductivity noise in films of charge-ordered Nd0.5Ca0.5MnO3 grown on Si substrate reveal the existence of a threshold field in the charge-ordered regime beyond which strong nonlinear conduction sets in along with a large broad band conductivity noise. Threshold-dependent conduction disappears as T --> T-CO, the charge-ordering temperature. This observation suggests that the charge-ordered state gets depinned at the onset of the nonlinear conduction. (C) 1999 American Institute of Physics. [S0003-6951(99)05247-X].

Resent radar observations of asteroid 1566 Icarus

We report Doppler-only radar observations of Icarus at Goldstone at a transmitter frequency of 8510 MHz (3.5 cm wavelength) during 8-10 June 1996, the first radar detection of the object since 1968. Optimally filtered and folded spectra achieve a maximum opposite-circular (OC) polarization signal-to-noise ratio of about 10 and help to constrain Icarus' physical properties. We obtain an OC radar cross section of 0.05 km(2) (with a 35% uncertainty), which is less than values estimated by Goldstein (1969) and by Pettengill et al. (1969), and a circular polarization (SC/OC) ratio of 0.5+/-0.2. We analyze the echo power spectrum with a model incorporating the echo bandwidth B and a spectral shape parameter it, yielding a coupled constraint between B and n. We adopt 25 Hz as the lower bound on B, which gives a lower bound on the maximum pole-on breadth of about 0.6 km and upper bounds on the radar and optical albedos that are consistent with Icarus' tentative QS classification. The observed circular polarization ratio indicates a very rough near-surface at spatial scales of the order of the radar wavelength. (C) 1999 Elsevier Science Ltd. All rights reserved.

On optimal timer-based distributed selection for rate-adaptive multi-user diversity systems

We develop an optimal, distributed, and low feedback timer-based selection scheme to enable next generation rate-adaptive wireless systems to exploit multi-user diversity. In our scheme, each user sets a timer depending on its signal to noise ratio (SNR) and transmits a small packet to identify itself when its timer expires. When the SNR-to-timer mapping is monotone non-decreasing, timers of users with better SNRs expire earlier. Thus, the base station (BS) simply selects the first user whose timer expiry it can detect, and transmits data to it at as high a rate as reliably possible. However, timers that expire too close to one another cannot be detected by the BS due to collisions. We characterize in detail the structure of the SNR-to-timer mapping that optimally handles these collisions to maximize the average data rate. We prove that the optimal timer values take only a discrete set of values, and that the rate adaptation policy strongly influences the optimal scheme's structure. The optimal average rate is very close to that of ideal selection in which the BS always selects highest rate user, and is much higher than that of the popular, but ad hoc, timer schemes considered in the literature.

Suppression of low-frequency noise in two-dimensional electron gas at degenerately doped Si:P delta layers

We report low-frequency 1/f-noise measurements of degenerately doped Si:P delta layers at 4.2 K. The noise was found to be over six orders of magnitude lower than that of bulk Si:P systems in the metallic regime and is one of the lowest values reported for doped semiconductors. The noise was nearly independent of magnetic field at low fields, indicating negligible contribution from universal conductance fluctuations. Instead, the interaction of electrons with very few active structural two-level systems may explain the observed noise magnitude.

Data rectification and detection of trend shifts in jet engine path measurements using median filters and fuzzy logic

Filtering methods are explored for removing noise from data while preserving sharp edges that many indicate a trend shift in gas turbine measurements. Linear filters are found to be have problems with removing noise while preserving features in the signal. The nonlinear hybrid median filter is found to accurately reproduce the root signal from noisy data. Simulated faulty data and fault-free gas path measurement data are passed through median filters and health residuals for the data set are created. The health residual is a scalar norm of the gas path measurement deltas and is used to partition the faulty engine from the healthy engine using fuzzy sets. The fuzzy detection system is developed and tested with noisy data and with filtered data. It is found from tests with simulated fault-free and faulty data that fuzzy trend shift detection based on filtered data is very accurate with no false alarms and negligible missed alarms.

Measurement of 1/f noise and its application in materials science

This is a review of the measurement of I If noise in certain classes of materials which have a wide range of potential applications. This includes metal films, semi-conductors, metallic oxides and inhomogeneous systems such as composites. The review contains a basic introduction to this field, the theories and models and follows it up with a discussion on measurement methods. There are discussions on specific examples of the application of noise spectroscopy in the field of materials science. (C) 2002 Elsevier Science Ltd. All rights reserved.

On the evidence for the effect of bubble interference on cavitation noise

Cavitation-noise measurements from an axisymmetric body with ‘controlled’ generation of cavitation are reported. The control was achieved by seeding artificial nuclei in the boundary layer by electrolysis. It was possible to alter the number density of nuclei by varying the electrolysis voltage, polarity and the geometry of the electrode. From the observed trend of cavitation-noise data it is postulated that there exists an ‘interference effect’ which influences cavitation noise. When the nucleus-number density is high and cavitation numbers are low this effect is strong. Under these conditions the properties of cavitation noise are found to differ considerably from those expected based on theories concerning noise from single-spherical-bubble cavitation.

Performance analysis of TCM with generalized selection combining on Rayleigh fading channels

We derive the computational cutoff rate, R-o, for coherent trellis-coded modulation (TCM) schemes on independent indentically distributed (i.i.d.) Rayleigh fading channels with (K, L) generalized selection combining (GSC) diversity, which combines the K paths with the largest instantaneous signal-to-noise ratios (SNRs) among the L available diversity paths. The cutoff rate is shown to be a simple function of the moment generating function (MGF) of the SNR at the output of the (K, L) GSC receiver. We also derive the union bound on the bit error probability of TCM schemes with (K, L) GSC in the form of a simple, finite integral. The effectiveness of this bound is verified through simulations.