Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Generalized method for assessment of compressibility of rockfill material

Examination of experimental data of the modelled rockfill materials using parallel gradation technique has revealed that the plots of logarithm of strain at failure against logarithm of confining pressure are linear. Also, a trend of increase in failure strain with increase in confining pressure and maximum size of the particle have been observed. The approach presented in this paper highlights the prediction of volume change properties of rockfill materials over a range of confining pressures and particle sizes based on the results of only two tests carried out at two different confining pressures for a maximum particle size of modelled material with the use of parallel gradation technique. Two test approach and its application in modelling of rockfill materials to estimate its volume change behaviour is illustrated by means of a selected experimental data available in the literature.

Tensile force distribution along the reinforcement for reinforced soil foundations

In a reinforced soil bed system reinforcement layer is usually placed with or without end anchorage. Since soil is weak in tension reinforcement develop tension under the applied load or the displacement of the footing. This tensile force is distributed along the length of the reinforcement subjected to the end condition. The reinforccement tension helps in distributing the load over a wider area, and becomes more effective at large induced settlements. As a result, vertical componenent of tensile force generated becomes effective in reducing applied load. However, very few studies to quantify the tensile force along the reinforcement have been reported in the literature. In this paper an attempt has been made to obtain the true nature of tensile force distribution along the reinforcement. For a reinforced soil bed below a strip footing this paper brings out induced tensile force distribution along the reinforcement at different load levels and for different types of reinforcements.

Is diatom earth a collapsible material?

Analysis of compressibility data of diatom earth and Ariake clay of similar water holding capacities has been made in this paper. Analysis suggests that in the case of clays with sheet minerals such as in Ariake clays, due to compression, cluster growth takes place, whereas with diatom earth the breakdown of cluster accounts for bilinear compression characteristics. It has been hypothesized that the interactive void ratio in the case of diatom earth is likely to be far smaller than that in the case of Ariake clay where most of the pore water is herd by micropores enclosed by clay particle clusters. In a way diatom earth reflects the behaviour of clay of very law physico-chemical potential with far reduced collapse potential. Even the compressibility at higher stress range both in undisturbed and remolded states are likely to be due to breakdown of clusters with little contribution from the physico - chemical potential. Diatom earth is not a collapsible material at stress levels of engineering interest despite the in -situ water content is at par or even higher than soft sensitive Ariake clay with comparatively low cementation consequently with pronounced collapsible potential.

A neural network based automatic generation control design through reinforcement learning

This paper presents the design and implementation of a learning controller for the Automatic Generation Control (AGC) in power systems based on a reinforcement learning (RL) framework. In contrast to the recent RL scheme for AGC proposed by us, the present method permits handling of power system variables such as Area Control Error (ACE) and deviations from scheduled frequency and tie-line flows as continuous variables. (In the earlier scheme, these variables have to be quantized into finitely many levels). The optimal control law is arrived at in the RL framework by making use of Q-learning strategy. Since the state variables are continuous, we propose the use of Radial Basis Function (RBF) neural networks to compute the Q-values for a given input state. Since, in this application we cannot provide training data appropriate for the standard supervised learning framework, a reinforcement learning algorithm is employed to train the RBF network. We also employ a novel exploration strategy, based on a Learning Automata algorithm,for generating training samples during Q-learning. The proposed scheme, in addition to being simple to implement, inherits all the attractive features of an RL scheme such as model independent design, flexibility in control objective specification, robustness etc. Two implementations of the proposed approach are presented. Through simulation studies the attractiveness of this approach is demonstrated.

Reinforcement learning controllers for automatic generation control in power systems having re-heat units with GRC and dead-band

A new automatic generation controller (AGC) design approach, adopting reinforcement learning (RL) techniques, was recently pro- posed [1]. In this paper we demonstrate the design and performance of controllers based on this RL approach for automatic generation control of systems consisting of units having complex dynamics—the reheat type of thermal units. For such systems, we also assess the capabilities of RL approach in handling realistic system features such as network changes, parameter variations, generation rate constraint (GRC), and governor deadband.

A new approach to the design of reinforcement schemes for learning automata

On introduit une nouvelle classe de schémas de renforcement des automates d'apprentissage utilisant les estimations des caractéristiques aléatoires de l'environnement. On montre que les algorithmes convergent en probabilité vers le choix optimal des actions. On présente les résultats de simulation et on suggère des applications à un environnement à plusieurs apprentissages

Synthesis, growth, and characterization of a new NLO material 3-(2,3-dimethoxyphenyl)-1-(pyridin-2-yl)prop-2-en-1-one

3-(2,3-Dimethoxyphenyl)-1-(pyridin-2-yl)prop-2-en-1-one (DMPP) a potential second harmonic generating (SHG) has been synthesized and grown as a single crystal by the slow evaporation technique at ambient temperature. The structure determination of the grown crystal was done by single crystal X-ray diffraction study. DMPP crystallizes with orthorhombic system with cell parameters a = 20.3106(8)angstrom, b = 4.9574(2)angstrom, c = 13.4863(5)angstrom, alpha = 90 degrees, beta = 90 degrees, gamma = 90 degrees and space group Pca2(1). The crystals were characterized by FT-IR, thermal analysis, UV-vis-NIR spectroscopy and SHG measurements. Various functional groups present in DMPP were ascertained by FTIR analysis. DMPP is thermally stable up to 80 degrees C and optically transparent in the visible region. The crystal exhibits SHG efficiency comparable to that of KDP. (C) 2011 Elsevier B.V. All rights reserved.

Structural, vibrational and thermal studies of a new nonlinear optical material: L-Asparagine-L-tartaric acid

Crystals of a new nonlinear optical (NLO) material, viz., L-asparagine-L-tartaric acid (LALT)(1) were grown by slow evaporation of an aqueous solution containing equimolar concentrations of L-asparagine and t-tartaric acid. The structure of the title compound which crystallizes in the non-centrosymmetric monoclinic space group P2(1) consists of a molecule of L-asparagine and a molecule of free L-tartaric acid both of which are interlinked by three varieties of H-bonding interactions namely O-H center dot center dot center dot O, N-H center dot center dot center dot O and C-H center dot center dot center dot O. The UV-Vis-NIR spectrum of 1 reveals its transparent nature while the vibrational spectra confirm the presence of the functional groups in 1. The thermal stability and second harmonic generation (SHG) conversion efficiency of 1 were investigated. (C) 2012 Elsevier B.V. All rights reserved.

Synthesis and Electrochemical Characterization of LiNi0.8Co0.2O2 as Cathode Material for Aqueous Rechargeable Lithium Batteries

LiNi0.8Co0.2O2 cathode material for lithium ion batteries is synthesized by reaction under autogenic pressure at elevated temperature (RAPET) method. The simple synthesis procedure is time and energy saving, and thus is promising for commercial application. The structure and stability of the material have been characterized by means of XRD and TG-DTA. The electrochemical properties of the LiNi0.8Co0.2O2 cathode are investigated in 2 M Li2SO4 aqueous electrolyte and they are compared to that in an organic electrolyte. A battery cell consisting of LiNi0.8Co0.2O2 as cathode in 2 M Li2SO4 solution is constructed in combination with LiTi2 (PO4)(3) as anode. The cell retained almost constant discharge capacity over hundred cycles. The electrochemical impedance spectral ( EIS) studies in aqueous and nonaqueous electrolytes revealed that the mechanism of lithium ion intercalation and deintercalation processes in LiNi0.8Co0.2O2 electrode follow almost similar mechanism in both aqueous and nonaqueous electrolytes. The chemical diffusion coefficient was calculated from slow scan rate cyclic voltammetry and EIS. (C) 2012 The Electrochemical Society. DOI: 10.1149/2.075205jes] All rights reserved.

Development of a composite material with high magnetic permeability and low loss factor for high frequency application

A flexible composite suitable for MHz frequency application has been developed by combining Fe3O4 and polyvinyl alcohol (PVA). The loss factor and the permeability have been evaluated. At an optimum weight percentage of Fe3O4 in the PVA matrix, the frequency at which the loss factor gives a minimum shifts to the MHz region. The loss factor has been found to be lower by one order of magnitude at 70 MHz compared to the presently used nickel zinc ferrite. The Henkel plot and the Cole-Cole plot have been obtained for the understanding of the high magnetic permeability and the low loss factor. (C) 2012 American Institute of Physics. doi:10.1063/1.3672867]

Measurement of effective permeability of reinforcement mats using sensitivity analysis

A methodology using sensitivity analysis is proposed to measure the effective permeability which includes the interaction of the resin and the reinforcement. Initially, mold-filling experiments were performed at isothermal conditions on the test specimen and the positions of the flow front were tracked with time using a flow visualization method. Following this, mold-filling experiments were simulated using a commercial software to obtain the positions of the flow front with time at the process conditions used for experiments. Several iterations were performed using different trial values of the permeability until the experimentally tracked and simulated positions of the flow front with time were matched. Finally, the value of the permeability thus obtained was validated by comparing the positions obtained by performing the experiments at different process conditions with the positions obtained by simulating the experiments. In this study, woven roving and chopped strand mats of E-class glass fiber and unsaturated polyester resin were used for the experiments. From the results, it was found that the measured permeabilities were consistent with varying process conditions. POLYM. COMPOS., 2012. (c) 2012 Society of Plastics Engineers

Sonochemical Synthesis of Pt Ion Substituted TiO2 (Ti0.9Pt0.1O2): A High Capacity Anode Material for Lithium Battery

Doping of TiO2 with a suitable metal ion where dopant redox potential couples with that of titanium (Ti4+) and act as catalyst for additional reduction of Ti4+ to Ti2+ (Ti4+ -> Ti3+ -> Ti2+) is envisaged here to enhance lithium storage even higher than one Li/TiO2. Accordingly, 10 atom% Pt ion substituted TiO2, Ti0.9Pt0.1O2 nanocrystallites was synthesized by sonochemical method using diethylenetriamine (DETA) as complexing agent. Powder X-ray diffraction pattern (XRD), Rietveld refinement and TEM study reveals that Ti0.9Pt0.1O2 nanocrystallites of similar to 4 nm size crystallize in anatase structure. X-ray photo-electron spectroscopy (XPS) study confirms that and both Ti and Pt are in 4+ oxidation state. Due to Pt4+ ion substitution in TiO2, reducibility of TiO2 was enhanced and Ti4+ was reduced up to Ti2+ state via coupling of Pt states (Pt4+/Pt2+/Pt-0) with Ti states (Ti4+/Ti3+/Ti2+). Galvanostatic cycling of Ti0.9Pt0.1O2 against lithium showed very high capacity of 430 mAhg(-1) or exchange of similar to 1.5Li/Ti0.9Pt0.1O2. (C) 2012 The Electrochemical Society. DOI: 10.1149/2.029208jes] All rights reserved.

Hydrothermal synthesis of a monoclinic VO2 nanotube-graphene hybrid for use as cathode material in lithium ion batteries

The hydrothermal reaction of a mixture of a colloidal dispersion of graphite oxide and ammonium vanadate yielded a hybrid made of graphene and a nanotubular metastable monoclinic polymorph of VO2, known as VO2(B). The formation of VO2(B) nanotubes is accompanied by the reduction of graphite oxide. Initially the partially scrolled graphite oxide layers act as templates for the crystallization of VO2(B) in the tubular morphology. This is followed by the reduction of graphite oxide to graphene resulting in a hybrid in which VO2(B) nanotubes are dispersed in graphene. Electron microscopic studies of the hybrid reveal that the VO2(B) nanotubes are wrapped by and trapped between graphene sheets. The hybrid shows potential to be a high capacity cathode material for lithium ion batteries. It exhibits a high capacity (similar to 450 mAh/g) and cycling stability. The high capacity of the hybrid is attributed to the interaction between the graphene sheets and the VO2(B) tubes which improves the charge-transfer. The graphene matrix prevents the aggregation of the VO2(B) nanotubes leading to high cycling stability. (C) 2012 Elsevier Ltd. All rights reserved.

Polymer microfabrication by scanning based microstereolithography: Optical design and material functionality

Several research groups have attempted to optimize photopolymerization parameters to increase the throughput of scanning based microstereolithography (MSL) systems through modified beam scanning techniques. Efforts in reducing the curing line width have been implemented through high numerical aperture (NA) optical setups. However, the intensity contour symmetry and the depth of field of focus have led to grossly non-vertical and non-uniform curing profiles. This work tries to review the photopolymerization process in a scanning based MSL system from the aspect of material functionality and optical design. The focus has been to exploit the rich potential of photoreactor scanning system in achieving desired fabrication modalities (minimum curing width, uniform depth profile, and vertical curing profile) even with a reduced NA optical setup and a single movable stage. The present study tries to manipulate to its advantage the effect of optimized lower c] (photoinitiator (PI) concentration) in reducing the minimum curing width to similar to 10-20 mu m even with a higher spot size (similar to 21.36 mu m) through a judiciously chosen ``monomer-PI'' system. Optimization on grounds of increasing E-max (maximum laser exposure energy at surface) by optimizing the scan rate provides enough time for the monomer or resin to get cured across the entire resist thickness (surface to substrate similar to 10-100 mu m), leading to uniform depth profiles along the entire scan lengths. (C) 2012 American Institute of Physics. http://dx.doi.org/10.1063/1.4750975]