Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Analysis,Insights and Generalization of a Fast Decentralized Relay Selection Mechanism

Relay selection for cooperative communications has attracted considerable research interest recently. While several criteria have been proposed for selecting one or more relays and analyzed, mechanisms that perform the selection in a distributed manner have received relatively less attention. In this paper, we analyze a splitting algorithm for selecting the single best relay amongst a known number of active nodes in a cooperative network. We develop new and exact asymptotic analysis for computing the average number of slots required to resolve the best relay. We then propose and analyze a new algorithm that addresses the general problem of selecting the best Q >= 1 relays. Regardless of the number of relays, the algorithm selects the best two relays within 4.406 slots and the best three within 6.491 slots, on average. Our analysis also brings out an intimate relationship between multiple access selection and multiple access control algorithms.

Sequential design of decentralized control systems

Our main result is a new sequential method for the design of decentralized control systems. Controller synthesis is conducted on a loop-by-loop basis, and at each step the designer obtains an explicit characterization of the class C of all compensators for the loop being closed that results in closed-loop system poles being in a specified closed region D of the s-plane, instead of merely stabilizing the closed-loop system. Since one of the primary goals of control system design is to satisfy basic performance requirements that are often directly related to closed-loop pole location (bandwidth, percentage overshoot, rise time, settling time), this approach immediately allows the designer to focus on other concerns such as robustness and sensitivity. By considering only compensators from class C and seeking the optimum member of that set with respect to sensitivity or robustness, the designer has a clearly-defined limited optimization problem to solve without concern for loss of performance. A solution to the decentralized tracking problem is also provided. This design approach has the attractive features of expandability, the use of only 'local models' for controller synthesis, and fault tolerance with respect to certain types of failure.

A modell of socially optimal hydro reservoir usage - yelding socially optimal outcomes in nordic electricity markets

An extensive electricity transmission network facilitates electricity trading between Finland, Sweden, Norway and Denmark. Currently most of the area's power generation is traded at NordPool, where the trading volumes have steadily increased since the early 1990's, when the exchange was founded. The Nordic electricity is expected to follow the current trend and further integrate with the other European electricity markets. Hydro power is the source for roughly a half of the supply in the Nordic electricity market and most of the hydro is generated in Norway. The dominating role of hydro power distinguishes the Nordic electricity market from most of the other market places. Production of hydro power varies mainly due to hydro reservoirs and demand for electricity. Hydro reservoirs are affected by water inflows that differ each year. The hydro reservoirs explain remarkably the behaviour of the Nordic electricity markets. Therefore among others, Kauppi and Liski (2008) have developed a model that analyzes the behaviour of the markets using hydro reservoirs as explanatory factors. Their model includes, for example, welfare loss due to socially suboptimal hydro reservoir usage, socially optimal electricity price, hydro reservoir storage and thermal reservoir storage; that are referred as outcomes. However, the model does not explain the real market condition but rather an ideal situation. In the model the market is controlled by one agent, i.e. one agent controls all the power generation reserves; it is referred to as a socially optimal strategy. Article by Kauppi and Liski (2008) includes an assumption where an individual agent has a certain fraction of market power, e.g. 20 % or 30 %. In order to maintain the focus of this thesis, this part of their paper is omitted. The goal of this thesis is two-fold. Firstly we expand the results from the socially optimal strategy for years 2006-08, as the earlier study finishes in 2005. The second objective is to improve on the methods from the previous study. This thesis results several outcomes (SPOT-price and welfare loss, etc.) due to socially optimal actions. Welfare loss is interesting as it describes the inefficiency of the market. SPOT-price is an important output for the market participants as it often has an effect on end users' electricity bills. Another function is to modify and try to improve the model by means of using more accurate input data, e.g. by considering pollution trade rights effect on input data. After modifications to the model, new welfare losses are calculated and compared with the same results before the modifications. The hydro reservoir has the higher explanatory significance in the model followed by thermal power. In Nordic markets, thermal power reserves are mostly nuclear power and other thermal sources (coal, natural gas, oil, peat). It can be argued that hydro and thermal reservoirs determine electricity supply. Roughly speaking, the model takes into account electricity demand and supply, and several parameters related to them (water inflow, oil price, etc.), yielding finally the socially optimal outcomes. The author of this thesis is not aware of any similar model being tested before. There have been some other studies that are close to the Kauppi and Liski (2008) model, but those have a somewhat different focus. For example, a specific feature in the model is the focus on long-run capacity usage that differs from the previous studies on short-run market power. The closest study to the model is from California's wholesale electricity markets that, however, uses different methodology. Work is constructed as follows.

Biomass gasification: Environmentally sound technology for decentralized power generation, a case study from India

This study aims at understanding the need for decentralized power generation systems and to explore the potential, feasibility and environmental implications of biomass gasifier-based electricity generation systems for village electrification. Electricity needs of villages are in the range of 5–20 kW depending on the size of the village. Decentralized power generation systems are desirable for low load village situations as the cost of power transmission lines is reduced and transmission and distribution losses are minimised. A biomass gasifier-based electricity generation system is one of the feasible options; the technology is readily available and has already been field tested. To meet the lighting and stationary power needs of 500,000 villages in India the land required is only 16 Mha compared to over 100 Mha of degraded land available for tree planting. In fact all the 95 Mt of woody biomass required for gasification could be obtained through biomass conservation programmes such as biogas and improved cook stoves. Thus dedication of land for energy plantations may not be required. A shift to a biomass gasifier-based power generation system leads to local benefits such as village self reliance, local employment and skill generation and promotion of in situ plant diversity plus global benefits like no net CO2 emission (as sustainable biomass harvests are possible) and a reduction in CO2 emissions (when used to substitute thermal power and diesel in irrigation pump sets).

Decentralized Parallel Operation of Inverters Sharing Unbalanced and Nonlinear Loads

In this paper, a wireless control strategy for parallel operation of three-phase four-wire inverters is proposed. A generalized situation is considered where the inverters are of unequal power ratings and the loads are nonlinear and unbalanced in nature. The proposed control algorithm exploits the potential of sinusoidal domain proportional+multiresonant controller ( in the inner voltage regulation loop) to make the system suitable for nonlinear and unbalanced loads with a simple and generalized structure of virtual output-impedance loop. The decentralized operation is achieved by using three-phase P/Q droop characteristics. The overall control algorithm helps to limit the harmonic contents and the degree of unbalance in the output-voltage waveform and to achieve excellent power-sharing accuracy in spite of mismatch in the inverter output impedances. Moreover, a synchronized turn on with consequent change over to the droop mode is applied for the new incoming unit in order to limit the circulating current completely. The simulation and experimental results from-1 kVA and -0.5 kVA paralleled units validate the effectiveness of the scheme.

Bayesian Decentralized Spectrum Sensing in Cognitive Radio Networks

This paper considers the problem of spectrum sensing, i.e., the detection of whether or not a primary user is transmitting data by a cognitive radio. The Bayesian framework is adopted, with the performance measure being the probability of detection error. A decentralized setup, where N sensors use M observations each to arrive at individual decisions that are combined at a fusion center to form the overall decision is considered. The unknown fading channel between the primary sensor and the cognitive radios makes the individual decision rule computationally complex, hence, a generalized likelihood ratio test (GLRT)-based approach is adopted. Analysis of the probabilities of false alarm and miss detection of the proposed method reveals that the error exponent with respect to M is zero. Also, the fusion of N individual decisions offers a diversity advantage, similar to diversity reception in communication systems, and a tight bound on the error exponent is presented. Through an analysis in the low power regime, the number of observations needed as a function of received power, to achieve a given probability of error is determined. Monte-Carlo simulations confirm the accuracy of the analysis.

Positional Consensus of Multi-Agent Systems Using Linear Programming Based Decentralized Control with Rectilinear Decision Domain

In this paper we develop a Linear Programming (LP) based decentralized algorithm for a group of multiple autonomous agents to achieve positional consensus. Each agent is capable of exchanging information about its position and orientation with other agents within their sensing region. The method is computationally feasible and easy to implement. Analytical results are presented. The effectiveness of the approach is illustrated with simulation results.

Decentralized Sustainable Energy Planning of Tumkur District, India

Present work shows the feasibility of decentralized energy options for the Tumkur district in India. Decentralized energy planning (DEP) involves scaling down energy planning to subnational or regional scales. The important aspect of the energy planning at decentralized level would be to prepare an area-based DEP to meet energy needs and development of alternate energy sources at least-cost to the economy and environment. The geographical coverage and scale reflects the level at which the analysis takes place, which is an important factor in determining the structure of models. In the present work, DEP modeling under different scenarios has been carried out for Tumkur district of India for the year 2020. DEP model is suitably scaled for obtaining the optimal mix of energy resources and technologies using a computer-based goal programming technique. The rural areas of the Tumkur district have different energy needs. Results show that electricity needs can be met by biomass gasifier technology, using biomass feedstock produced by allocating only 12% of the wasteland in the district at 8 t/ha/yr of biomass productivity. Surplus electricity can be produced by adopting the option of biomass power generation from energy plantations. The surplus electricity generated can be supplied to the grid. The sustainable development scenario is a least cost scenario apart from promoting self-reliance, local employment, and environmental benefits. (C) 2010 American Institute of Chemical Engineers Environ Prog, 30: 248-258, 2011

Decentralized control of linear multivariable systems

A strongly connected decentralized control system may be made single channel controllable and observable with respect to any channel by decentralized feedbacks. It is noted here that the system example considered by Corfmat and Morse to illustrate this fact is already single channel controllable and observable, with respect to one of the channels. An alternate example which fits into the situation is presented in this item.