Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.

Effect of absorption in aqueous and hygrothermal media on the compressive properties of glass fiber reinforced syntactic foam

Absorption due to immersion in aqueous media consisting of either saline or seawater or due to exposure to water vapor conditions and the attendant effect on the compressive properties of syntactic foam reinforced with E-glass fibers in the form of chopped strands were studied. Whereas the compressive strength decreased in samples exposed to water vapor, the saline or seawater immersed samples showed increase when compared to the dry sample. The decrease in strength in the vapor-exposed case is ascribed to higher absorption of water and to debonding and damaged features for interfaces. The enhancement of strength values for the samples immersed in saltish media is traced to the larger size of the chloride ion and resultant changes in the stress state around the fiber-bearing regions. Recourse to an analysis of scanning electron microscopic pictures of the compression-failed samples is taken to explain the observed trends.

Experimental investigation of link between growth and decay of fiber Bragg gratings

We report here an experimental investigation for establishing and quantifying a link between the growth and decay characteristics of fiber Bragg gratings. One of the key aspects of our work is the determination of the defect energy distribution from the grating characteristics measured during their fabrication. We observe a strong correlation between the growth-based defect energy distribution and that obtained through accelerated aging experiments, paving the way for predicting the decay characteristics of fiber Bragg gratings from their growth data. Such a prediction is significant in simplifying the postfabrication steps required to enhance the thermal stability of fiber Bragg gratings. (c) 2011 Optical Society of America

Design of Geocell Reinforcement for Supporting Embankments on Soft Ground

The methods of design available for geocell-supported embankments are very few. Two of the earlier methods are considered in this paper and a third method is proposed and compared with them. The first method is the slip line method proposed by earlier researchers. The second method is based on slope stability analysis proposed by this author earlier and the new method proposed is based on the finite element analyses. In the first method, plastic bearing failure of the soil was assumed and the additional resistance due to geocell layer is calculated using a non-symmetric slip line field in the soft foundation soil. In the second method, generalpurpose slope stability program was used to design the geocell mattress of required strength for embankment using a composite model to represent the shear strength of geocell layer. In the third method proposed in this paper, geocell reinforcement is designed based on the plane strain finite element analysis of embankments. The geocell layer is modelled as an equivalent composite layer with modified strength and stiffness values. The strength and dimensions of geocell layer is estimated for the required bearing capacity or permissible deformations. These three design methods are compared through a design example.

Health Monitoring Of Composite Structures Based On Acoustic Wave Sensing Using Fiber Optic Sensors

Structural Health Monitoring has gained wide acceptance in the recent past as a means to monitor a structure and provide an early warning of an unsafe condition using real-time data. Utilization of structurally integrated, distributed sensors to monitor the health of a structure through accurate interpretation of sensor signals and real-time data processing can greatly reduce the inspection burden. The rapid improvement of the Fiber Optic Sensor technology for strain, vibration, ultrasonic and acoustic emission measurements in recent times makes it feasible alternative to the traditional strain gauges, PVDF and conventional Piezoelectric sensors used for Non Destructive Evaluation (NDE) and Structural Health Monitoring (SHM). Optical fiber-based sensors offer advantages over conventional strain gauges, and PZT devices in terms of size, ease of embedment, immunity from electromagnetic interference (EMI) and potential for multiplexing a number of sensors. The objective of this paper is to demonstrate the acoustic wave sensing using Extrinsic Fabry-Perot Interferometric (EFPI) sensor on a GFRP composite laminates. For this purpose experiments have been carried out initially for strain measurement with Fiber Optic Sensors on GFRP laminates with intentionally introduced holes of different sizes as defects. The results obtained from these experiments are presented in this paper. Numerical modeling has been carried out to obtain the relationship between the defect size and strain.

Modeling of active fiber composite for delamination sensing

In order to demonstrate the feasibility of Active Fiber Composites (AFC) as sensors for detecting damage, a pretwisted strip made of AFC with symmetric free-edge delamination is considered in this paper. The strain developed on the top/bottom of the strip is measured to detect and assess delamination. Variational Asymptotic Method (VAM) is used in the development of a non-classical non-linear cross sectional model of the strip. The original three dimensional (3D) problem is simplified by the decomposition into two simpler problems: a two-dimensional (2D) problem, which provides in a compact form the cross-sectional properties using VAM, and a non-linear one-dimensional (1D) problem along the length of the beam. This procedure gives the non-linear stiffnesses, which are very sensitive to damage, at any given cross-section of the strip. The developed model is used to study a special case of cantilevered laminated strip with antisymmetric layup, loaded only by an axial force at the tip. The charge generated in the AFC lamina is derived in closed form in terms of the 1D strain measures. It is observed that delamination length and location have a definite influence on the charge developed in the AFC lamina. Also, sensor voltage output distribution along the length of the beam is obtained using evenly distributed electrode strip. These data could in turn be used to detect the presence of damage.