Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Recommendations for driver licensing and traffic law enforcement in India aiming to improve road safety

During the last decade, developing countries such as India have been exhibiting rapid increase in human population and vehicles, and increase in road accidents. Inappropriate driving behaviour is considered one of the major causes of road accidents in India as compared to defective geometric design of pavement or mechanical defects in vehicles. It can result in conditions such as lack of lane discipline, disregard to traffic laws, frequent traffic violations, increase in crashes due to self-centred driving, etc. It also demotivates educated drivers from following good driving practices. Hence, improved driver behaviour can be an effective countermeasure to reduce the vulnerability of road users and inhibit crash risks. This article highlights improved driver behaviour through better driver education, driver training and licensing procedures along with good on-road enforcement; as an effective countermeasure to ensure road safety in India. Based on the review and analysis, the article also recommends certain measures pertaining to driver licensing and traffic law enforcement in India aimed at improving road safety.

Signal compression through spatial frequency-based motion estimation

The interest in low bit rate video coding has increased considerably. Despite rapid progress in storage density and digital communication system performance, demand for data-transmission bandwidth and storage capacity continue to exceed the capabilities of available technologies. The growth of data-intensive digital audio, video applications and the increased use of bandwidth-limited media such as video conferencing and full motion video have not only sustained the need for efficient ways to encode analog signals, but made signal compression central to digital communication and data-storage technology. In this paper we explore techniques for compression of image sequences in a manner that optimizes the results for the human receiver. We propose a new motion estimator using two novel block match algorithms which are based on human perception. Simulations with image sequences have shown an improved bit rate while maintaining ''image quality'' when compared to conventional motion estimation techniques using the MAD block match criteria.

Non-interferometric methods of phase estimation for application in optical tomography

We describe here two non-interferometric methods for the estimation of the phase of transmitted wavefronts through refracting objects. The phase of the wavefronts obtained is used to reconstruct either the refractive index distribution of the objects or their contours. Refraction corrected reconstructions are obtained by the application of an iterative loop incorporating digital ray tracing for forward propagation and a modified filtered back projection (FBP) for reconstruction. The FBP is modified to take into account non-straight path propagation of light through the object. When the iteration stagnates, the difference between the projection data and an estimate of it obtained by ray tracing through the final reconstruction is reconstructed using a diffraction tomography algorithm. The reconstruction so obtained, viewed as a correction term, is added to the estimate of the object from the loop to obtain an improved final refractive index reconstruction.

Performance analysis of a modified spatial smoothing technique for direction estimation

The statistical performance analysis of ESPRIT, root-MUSIC, minimum-norm methods for direction estimation, due to finite data perturbations, using the modified spatially smoothed covariance matrix, is developed. Expressions for the mean-squared error in the direction estimates are derived based on a common framework. Based on the analysis, the use of the modified smoothed covariance matrix improves the performance of the methods when the sources are fully correlated. Also, the performance is better even when the number of subarrays is large unlike in the case of the conventionally smoothed covariance matrix. However, the performance for uncorrelated sources deteriorates due to an artificial correlation introduced by the modified smoothing. The theoretical expressions are validated using extensive simulations. (C) 1999 Elsevier Science B.V. All rights reserved.

Novel subspace methods for blind estimation of multiple CFOs in MIMO-OFDM systems

The problem of estimating multiple Carrier Frequency Offsets (CFOs) in the uplink of MIMO-OFDM systems with Co-Channel (CC) and OFDMA based carrier allocation is considered. The tri-linear data model for generalized, multiuser OFDM system is formulated. Novel blind subspace based estimation of multiple CFOs in the case of arbitrary carrier allocation scheme in OFDMA systems and CC users in OFDM systems based on the Khatri-Rao product is proposed. The method works where the conventional subspace method fails. The performance of the proposed methods is compared with pilot based Least-Squares method.

Throughput Maximization of Delay-Constrained Traffic in Wireless Energy Harvesting Sensors

A wireless Energy Harvesting Sensor (EHS) needs to send data packets arriving in its queue over a fading channel at maximum possible throughput while ensuring acceptable packet delays. At the same time, it needs to ensure that energy neutrality is satisfied, i.e., the average energy drawn from a battery should equal the amount of energy deposited in it minus the energy lost due to the inefficiency of the battery. In this work, a framework is developed under which a system designer can optimize the performance of the EHS node using power control based on the current channel state information, when the EHS node employs a single modulation and coding scheme and the channel is Rayleigh fading. Optimal system parameters for throughput optimal, delay optimal and delay-constrained throughput optimal policies that ensure energy neutrality are derived. It is seen that a throughput optimal (maximal) policy is packet delay-unbounded and an average delay optimal (minimal) policy achieves negligibly small throughput. Finally, the influence of the harvested energy profile on the performance of the EHS is illustrated through the example of solar energy harvesting.

Application of an evolutionary fuzzy system for the estimation of workforce deployment and cross-training in an assembly environment

Owing to the increased customer demands for make-to-order products and smaller product life-cycles, today assembly lines are designed to ensure a quick switch-over from one product model to another for companies' survival in market place. The complexity associated with the decisions pertaining to the type of training and number of workers and their exposition to the different tasks especially in the current era of customized production is a serious problem that the managers and the HRD gurus are facing in industry. This paper aims to determine the amount of cross-training and dynamic deployment policy caused by workforce flexibility for a make-to-order assembly. The aforementioned issues have been dealt with by adopting the concept of evolutionary fuzzy system because of the linguistic nature of the attributes associated with product variety and task complexity. A fuzzy system-based methodology is proposed to determine the amount of cross-training and dynamic deployment policy. The proposed methodology is tested on 10 sample products of varying complexities and the results obtained are in line with the conclusions drawn by previous researchers.

Analytical models for energy consumption in infrastructure WLAN STAs carrying TCP traffic

We develop analytical models for estimating the energy spent by stations (STAs) in infrastructure WLANs when performing TCP controlled file downloads. We focus on the energy spent in radio communication when the STAs are in the Continuously Active Mode (CAM), or in the static Power Save Mode (PSM). Our approach is to develop accurate models for obtaining the fraction of times the STA radios spend in idling, receiving and transmitting. We discuss two traffic models for each mode of operation: (i) each STA performs one large file download, and (ii) the STAs perform short file transfers. We evaluate the rate of STA energy expenditure with long file downloads, and show that static PSM is worse than just using CAM. For short file downloads we compute the number of file downloads that can be completed with given battery capacity, and show that PSM performs better than CAM for this case. We provide a validation of our analytical models using the NS-2 simulator.

Estimation of elasticity map of soft biological tissue mimicking phantom using laser speckle contrast analysis

Scattering of coherent light from scattering particles causes phase shift to the scattered light. The interference of unscattered and scattered light causes the formation of speckles. When the scattering particles, under the influence of an ultrasound (US) pressure wave, vibrate, the phase shift fluctuates, thereby causing fluctuation in speckle intensity. We use the laser speckle contrast analysis (LSCA) to reconstruct a map of the elastic property (Young's modulus) of soft tissue-mimicking phantom. The displacement of the scatters is inversely related to the Young's modulus of the medium. The elastic properties of soft biological tissues vary, many fold with malignancy. The experimental results show that laser speckle contrast (LSC) is very sensitive to the pathological changes in a soft tissue medium. The experiments are carried out on a phantom with two cylindrical inclusions of sizes 6 mm in diameter, separated by 8 mm between them. Three samples are made. One inclusion has Young's modulus E of 40 kPa. The second inclusion has either a Young's modulus E of 20 kPa, or scattering coefficient of mu'(s), = 3.00 mm(-1) or absorption coefficient of mu(a) = 0.03 mm(-1). The optical absorption (mu(a)), reduced scattering (mu'(s)) coefficient, and the Young's modulus of the background are mu(a) = 0.01 mm(-1), mu'(s) = 1.00 mm(-1) and 12kPa, respectively. The experiments are carried out on all three phantoms. On a phantom with two inclusions of Young's modulus of 20 and 40 kPa, the measured relative speckle image contrasts are 36.55% and 63.72%, respectively. Experiments are repeated on phantoms with inclusions of mu(a) = 0.03 mm-1, E = 40 kPa and mu'(s) = 3.00 mm(-1). The results show that it is possible to detect inclusions with contrasts in optical absorption, optical scattering, and Young's modulus. Studies of the variation of laser speckle contrast with ultrasound driving force for various values of mu(a), mu'(s), and Young's modulus of the tissue mimicking medium are also carried out. (C) 2011 American Institute of Physics. doi:10.1063/1.3592352]

Identification of delamination in composite beams using spectral estimation and a genetic algorithm

An efficient strategy for identification of delamination in composite beams and connected structures is presented. A spectral finite-element model consisting of a damaged spectral element is used for model-based prediction of the damaged structural response in the frequency domain. A genetic algorithm (GA) specially tailored for damage identification is derived and is integrated with finite-element code for automation. For best application of the GA, sensitivities of various objective functions with respect to delamination parameters are studied and important conclusions are presented. Model-based simulations of increasing complexity illustrate some of the attractive features of the strategy in terms of accuracy as well as computational cost. This shows the possibility of using such strategies for the development of smart structural health monitoring softwares and systems.

Incremental-angle and angular velocity estimation using a star sensor

A method is described for estimating the incremental angle and angular velocity of a spacecraft using integrated rate parameters with the help of a star sensor alone. The chief advantage of this method is that the measured stars need not be identified, whereas the identification of the stars is necessary in earlier methods. This proposed estimation can be carried out with all of the available measurements by a simple linear Kalman filter, albeit with a time-varying sensitivity matrix. The residuals of estimated angular velocity by the proposed spacecraft incremental-angle and angular velocity estimation method are as accurate as the earlier methods. This method also enables the spacecraft attitude to be reconstructed for mapping the stars into an imaginary unit sphere in the body reference frame, which will preserve the true angular separation of the stars. This will pave the way for identification of the stars using any angular separation or triangle matching techniques applied to even a narrow field of view sensor that is made to sweep the sky. A numerical simulation for inertial as well as Earth pointing spacecraft is carried out to establish the results.

Estimation of surface free energy of pyrites by contact angle measurements

The surface properties of coal-pyrite play a major role in determining its separation from coal in processes such as flotation. The solution pH is an important parameter in determining the surface properties of both coal and coal-pyrite such as surface free energy and zeta-potential. In the present investigation, the effect of pH on the surface free energy of pyrites from different sources was studied. The surface free energy of solids is made up of two components, i.e. the dispersive surface free energy and the acid-base interaction energy. Various methods have been used by previous researchers to evaluate these two components for different solids. In the present study, a new approach was developed and used to study the surface free energy of pyrite surfaces. Results indicate that the dispersion surface free energy of various pyrites is independent of pH while the acid-base interaction energy is strongly dependent on the pH. The acid-base interaction energy is different for each pyrite sample and also the change with pH varies with the type of pyrite. Coal-pyrite was found to be more hydrophobic than ore-pyrite which may be attributed to the presence of carbon in coal-pyrites. The acid-base interaction energy varied little with pH for coal pyrites than ore-pyrite. Comparison of acid-base interaction energy with zeta-potential measurements shows a good correlation between the minimum in acid-base interaction energy and the pHpzc.