Processing maps for hot-working of powder metallurgy 1100 Al-10 vol % SiC-particulate metal-matrix composite

The hot-working characteristics of the metal-matrix composite (MMC) Al-10 vol % SiC-particulate (SiCp) powder metallurgy compacts in as-sintered and in hot-extruded conditions were studied using hot compression testing. On the basis of the stress-strain data as a function of temperature and strain rate, processing maps depicting the variation in the efficiency of power dissipation, given by eegr = 2m/(m+1), where m is the strain rate sensitivity of flow stress, have been established and are interpreted on the basis of the dynamic materials model. The as-sintered MMC exhibited a domain of dynamic recrystallization (DRX) with a peak efficiency of about 30% at a temperature of about 500°C and a strain rate of 0.01 s�1. At temperatures below 350°C and in the strain rate range 0.001�0.01 s�1 the MMC exhibited dynamic recovery. The as-sintered MMC was extruded at 500°C using a ram speed of 3 mm s�1 and an extrusion ratio of 10ratio1. A processing map was established on the extruded product, and this map showed that the DRX domain had shifted to lower temperature (450°C) and higher strain rate (1 s�1). The optimum temperature and strain rate combination for powder metallurgy billet conditioning are 500°C and 0.01 s�1, and the secondary metal-working on the extruded product may be done at a higher strain rate of 1 s�1 and a lower temperature of 425°C.

A tutorial survey of reinforcement learning

This paper gives a compact, self-contained tutorial survey of reinforcement learning, a tool that is increasingly finding application in the development of intelligent dynamic systems. Research on reinforcement learning during the past decade has led to the development of a variety of useful algorithms. This paper surveys the literature and presents the algorithms in a cohesive framework.

Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

A finite element assessment of flexural strength of prestressed concrete beams with fiber reinforcement

This paper presents an assessment of the flexural behavior of 15 fully/partially prestressed high strength concrete beams containing steel fibers investigated using three-dimensional nonlinear finite elemental analysis. The experimental results consisted of eight fully and seven partially prestressed beams, which were designed to be flexure dominant in the absence of fibers. The main parameters varied in the tests were: the levels of prestressing force (i.e, in partially prestressed beams 50% of the prestress was reduced with the introduction of two high strength deformed bars instead), fiber volume fractions (0%, 0.5%, 1.0% and 1.5%), fiber location (full depth and partial depth over full length and half the depth over the shear span only). A three-dimensional nonlinear finite element analysis was conducted using ANSYS 5.5 [Theory Reference Manual. In: Kohnke P, editor. Elements Reference Manual. 8th ed. September 1998] general purpose finite element software to study the flexural behavior of both fully and partially prestressed fiber reinforced concrete beams. Influence of fibers on the concrete failure surface and stress-strain response of high strength concrete and the nonlinear stress-strain curves of prestressing wire and deformed bar were considered in the present analysis. In the finite element model. tension stiffening and bond slip between concrete and reinforcement (fibers., prestressing wire, and conventional reinforcing steel bar) have also been considered explicitly. The fraction of the entire volume of the fiber present along the longitudinal axis of the prestressed beams alone has been modeled explicitly as it is expected that these fibers would contribute to the mobilization of forces required to sustain the applied loads across the crack interfaces through their bridging action. A comparison of results from both tests and analysis on all 15 specimens confirm that, inclusion of fibers over a partial depth in the tensile side of the prestressed flexural structural members was economical and led to considerable cost saving without sacrificing on the desired performance. However. beams having fibers over half the depth in only the shear span, did not show any increase in the ultimate load or deformational characteristics when compared to plain concrete beams. (C) 2002 Published by Elsevier Science Ltd.

A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.

Effects of surface treatments and size of fly ash particles on the compressive properties of epoxy based particulate composites

Particulate reinforcements for polymers are selected with dual objective of improving composite properties and save on the total cost of the system. In the present study fly ash, an industrial waste with good properties is used as filler in epoxy and the compressive properties of such composites are studied. Particle surfaces are treated chemically using a silane-coupling agent to improve the compatibility with the matrix. The compressive properties of these are compared with those made of untreated fly ash particulates. Furthermore properties of fly ash composites with two different average particle sizes are first compared between themselves and then with those made using the as-received bimodal nature of particle size distribution. Microscopic observations of compression tested samples revealed a better adherence of the particles with the matrix in case of treated particles and regards the size effect the composites with lower average particle size showed improved strength at higher filler contents. Experimental values of strengths and modulii are compared with some of the theoretical models for composite properties. (C) 2002 Kluwer Academic Publishers.

Design of Geocell Reinforcement for Supporting Embankments on Soft Ground

The methods of design available for geocell-supported embankments are very few. Two of the earlier methods are considered in this paper and a third method is proposed and compared with them. The first method is the slip line method proposed by earlier researchers. The second method is based on slope stability analysis proposed by this author earlier and the new method proposed is based on the finite element analyses. In the first method, plastic bearing failure of the soil was assumed and the additional resistance due to geocell layer is calculated using a non-symmetric slip line field in the soft foundation soil. In the second method, generalpurpose slope stability program was used to design the geocell mattress of required strength for embankment using a composite model to represent the shear strength of geocell layer. In the third method proposed in this paper, geocell reinforcement is designed based on the plane strain finite element analysis of embankments. The geocell layer is modelled as an equivalent composite layer with modified strength and stiffness values. The strength and dimensions of geocell layer is estimated for the required bearing capacity or permissible deformations. These three design methods are compared through a design example.

Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Tensile force distribution along the reinforcement for reinforced soil foundations

In a reinforced soil bed system reinforcement layer is usually placed with or without end anchorage. Since soil is weak in tension reinforcement develop tension under the applied load or the displacement of the footing. This tensile force is distributed along the length of the reinforcement subjected to the end condition. The reinforccement tension helps in distributing the load over a wider area, and becomes more effective at large induced settlements. As a result, vertical componenent of tensile force generated becomes effective in reducing applied load. However, very few studies to quantify the tensile force along the reinforcement have been reported in the literature. In this paper an attempt has been made to obtain the true nature of tensile force distribution along the reinforcement. For a reinforced soil bed below a strip footing this paper brings out induced tensile force distribution along the reinforcement at different load levels and for different types of reinforcements.

A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.