Design of Geocell Reinforcement for Supporting Embankments on Soft Ground

The methods of design available for geocell-supported embankments are very few. Two of the earlier methods are considered in this paper and a third method is proposed and compared with them. The first method is the slip line method proposed by earlier researchers. The second method is based on slope stability analysis proposed by this author earlier and the new method proposed is based on the finite element analyses. In the first method, plastic bearing failure of the soil was assumed and the additional resistance due to geocell layer is calculated using a non-symmetric slip line field in the soft foundation soil. In the second method, generalpurpose slope stability program was used to design the geocell mattress of required strength for embankment using a composite model to represent the shear strength of geocell layer. In the third method proposed in this paper, geocell reinforcement is designed based on the plane strain finite element analysis of embankments. The geocell layer is modelled as an equivalent composite layer with modified strength and stiffness values. The strength and dimensions of geocell layer is estimated for the required bearing capacity or permissible deformations. These three design methods are compared through a design example.

Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Tensile force distribution along the reinforcement for reinforced soil foundations

In a reinforced soil bed system reinforcement layer is usually placed with or without end anchorage. Since soil is weak in tension reinforcement develop tension under the applied load or the displacement of the footing. This tensile force is distributed along the length of the reinforcement subjected to the end condition. The reinforccement tension helps in distributing the load over a wider area, and becomes more effective at large induced settlements. As a result, vertical componenent of tensile force generated becomes effective in reducing applied load. However, very few studies to quantify the tensile force along the reinforcement have been reported in the literature. In this paper an attempt has been made to obtain the true nature of tensile force distribution along the reinforcement. For a reinforced soil bed below a strip footing this paper brings out induced tensile force distribution along the reinforcement at different load levels and for different types of reinforcements.

A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.

Reinforcement learning controllers for automatic generation control in power systems having re-heat units with GRC and dead-band

A new automatic generation controller (AGC) design approach, adopting reinforcement learning (RL) techniques, was recently pro- posed [1]. In this paper we demonstrate the design and performance of controllers based on this RL approach for automatic generation control of systems consisting of units having complex dynamics—the reheat type of thermal units. For such systems, we also assess the capabilities of RL approach in handling realistic system features such as network changes, parameter variations, generation rate constraint (GRC), and governor deadband.

A new approach to the design of reinforcement schemes for learning automata

On introduit une nouvelle classe de schémas de renforcement des automates d'apprentissage utilisant les estimations des caractéristiques aléatoires de l'environnement. On montre que les algorithmes convergent en probabilité vers le choix optimal des actions. On présente les résultats de simulation et on suggère des applications à un environnement à plusieurs apprentissages

Measurement of effective permeability of reinforcement mats using sensitivity analysis

A methodology using sensitivity analysis is proposed to measure the effective permeability which includes the interaction of the resin and the reinforcement. Initially, mold-filling experiments were performed at isothermal conditions on the test specimen and the positions of the flow front were tracked with time using a flow visualization method. Following this, mold-filling experiments were simulated using a commercial software to obtain the positions of the flow front with time at the process conditions used for experiments. Several iterations were performed using different trial values of the permeability until the experimentally tracked and simulated positions of the flow front with time were matched. Finally, the value of the permeability thus obtained was validated by comparing the positions obtained by performing the experiments at different process conditions with the positions obtained by simulating the experiments. In this study, woven roving and chopped strand mats of E-class glass fiber and unsaturated polyester resin were used for the experiments. From the results, it was found that the measured permeabilities were consistent with varying process conditions. POLYM. COMPOS., 2012. (c) 2012 Society of Plastics Engineers

Feature Search in the Grassmanian in Online Reinforcement Learning

We consider the problem of finding the best features for value function approximation in reinforcement learning and develop an online algorithm to optimize the mean square Bellman error objective. For any given feature value, our algorithm performs gradient search in the parameter space via a residual gradient scheme and, on a slower timescale, also performs gradient search in the Grassman manifold of features. We present a proof of convergence of our algorithm. We show empirical results using our algorithm as well as a similar algorithm that uses temporal difference learning in place of the residual gradient scheme for the faster timescale updates.

Effect of TiN particulate reinforcement on corrosive behaviour of aluminium 6061 composites in chloride medium

In the present investigation, the corrosive behaviour of Al 6061-TiN particulate composites prepared by liquid metallurgy has been studied in chloride medium using electroanalytical techniques such as Tafel, cyclic polarization and electrochemical impedance spectroscopy (EIS). Surface morphology of the sample electrodes was examined using scanning electron micrography and energy dispersive X-ray methods. X-ray diffraction technique was used to confirm inclusion of TiN particulates in the matrix alloy and identify the alloying elements and intermetallic compounds in the Al 6061 composites. Polarization studies indicate an increase in the corrosion resistance in composites compared to the matrix alloy. EIS study reveals that the polarization resistance (R (p)) increases with increase in TiN content in composites, thus confirming improved corrosion resistance in composites. The observed decrease in corrosion rate in the case of composites is due to decoupling between TiN particles and Al 6061 alloy. It is understood that after the initiation of corrosion, interfacial corrosion products may have decoupled the conducting ceramic TiN from Al 6061 matrix alloy thus eliminating the galvanic effect between them.

Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

Fly ash cenospheres as reinforcement in different polymer composites - a comparative study of physical and mechanical properties

The work reports the preparation of fly ash cenospheres bearing polymer composites, using various polymer matrix materials namely, low density polyethylene, high density polyethylene, polystyrene and polymethylmethacrylate followed by evaluation of properties. The composites are synthesized by including about 18% by weight fly ash cenospheres, into various polymer matrices using brabender facility in the temperature range 120-160 degrees C and at a mixing pressure of 50 MPa. Subsequently, they are cast into sheets through compression moulding. The test samples, made from the sheets, are characterized for physical as well as mechanical properties such as density, hardness, compression strength, impact response, wear and friction. The investigation reveals that the addition of fly ash cenospheres to various polymer matrices results in reduction of density. Further, improvements in the slide wear resistance and decrease in the co-efficient of friction values are noticed. As for interpreting the slide wear data, recourse to examination under scanning electron microscope is made in this paper. As regards the mechanical properties, hardness increases while the compression strength and impact energy decreases with inclusion of cenospheres in all the four types of samples investigated.