Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Risk Minimizing Option Pricing in a Semi-Markov Modulated Market

We address risk minimizing option pricing in a semi-Markov modulated market where the floating interest rate depends on a finite state semi-Markov process. The growth rate and the volatility of the stock also depend on the semi-Markov process. Using the Föllmer–Schweizer decomposition we find the locally risk minimizing price for European options and the corresponding hedging strategy. We develop suitable numerical methods for computing option prices.

Portfolio Optimization in a Semi-Markov Modulated Market

We address a portfolio optimization problem in a semi-Markov modulated market. We study both the terminal expected utility optimization on finite time horizon and the risk-sensitive portfolio optimization on finite and infinite time horizon. We obtain optimal portfolios in relevant cases. A numerical procedure is also developed to compute the optimal expected terminal utility for finite horizon problem.

Asymptotic analysis of option pricing in a Markov modulated market

We address asymptotic analysis of option pricing in a regime switching market where the risk free interest rate, growth rate and the volatility of the stocks depend on a finite state Markov chain. We study two variations of the chain namely, when the chain is moving very fast compared to the underlying asset price and when it is moving very slow. Using quadratic hedging and asymptotic expansion, we derive corrections on the locally risk minimizing option price.

Risk Minimizing Option Pricing for a Class of Exotic Options in a Markov-Modulated Market

We address risk minimizing option pricing in a regime switching market where the floating interest rate depends on a finite state Markov process. The growth rate and the volatility of the stock also depend on the Markov process. Using the minimal martingale measure, we show that the locally risk minimizing prices for certain exotic options satisfy a system of Black-Scholes partial differential equations with appropriate boundary conditions. We find the corresponding hedging strategies and the residual risk. We develop suitable numerical methods to compute option prices.

Pricing Defaultable Bonds in a Markov Modulated Market

We address the problem of pricing defaultable bonds in a Markov modulated market. Using Merton's structural approach we show that various types of defaultable bonds are combination of European type contingent claims. Thus pricing a defaultable bond is tantamount to pricing a contingent claim in a Markov modulated market. Since the market is incomplete, we use the method of quadratic hedging and minimal martingale measure to derive locally risk minimizing derivative prices, hedging strategies and the corresponding residual risks. The price of defaultable bonds are obtained as solutions to a system of PDEs with weak coupling subject to appropriate terminal and boundary conditions. We solve the system of PDEs numerically and carry out a numerical investigation for the defaultable bond prices. We compare their credit spreads with some of the existing models. We observe higher spreads in the Markov modulated market. We show how business cycles can be easily incorporated in the proposed framework. We demonstrate the impact on spreads of the inclusion of rare states that attempt to capture a tight liquidity situation. These states are characterized by low risk-free interest rate, high payout rate and high volatility.

Jatropha cultivation in southern India: assessing farmers' experiences

Together with 106 farmers who started growing Jatropha (Jatropha curcas L.) in 20042006, this research sought to increase the knowledge around the real-life experience of Jatropha farming in the southern India states of Tamil Nadu and Andhra Pradesh. Launched as an alternative for diesel in India, Jatropha has been promoted as a non-edible plant that could grow on poor soils, yield oil-rich seeds for production of bio-diesel, and not compete directly with food production. Through interviews with the farmers, information was gathered regarding their socio-economic situation, the implementation and performance of their Jatropha plantations, and their reasons for continuing or discontinuing Jatropha cultivation. Results reveal that 82% of the farmers had substituted former cropland for their Jatropha cultivation. By 2010, 85% (n = 90) of the farmers who cultivated Jatropha in 2004 had stopped. Cultivating the crop did not give the economic returns the farmers anticipated, mainly due to a lack of information about the crop and its maintenance during cultivation and due to water scarcity. A majority of the farmers irrigated and applied fertilizer, and even pesticides. Many problems experienced by the farmers were due to limited knowledge about cultivating Jatropha caused by poor planning and implementation of the national Jatropha program. Extension services, subsidies, and other support were not provided as promised. The farmers who continued cultivation had means of income other than Jatropha and held hopes of a future Jatropha market. The lack of market structures, such as purchase agreements and buyers, as well as a low retail price for the seeds, were frequently stated as barriers to Jatropha cultivation. For Jatropha biodiesel to perform well, efforts are needed to improve yield levels and stability through genetic improvements and drought tolerance, as well as agriculture extension services to support adoption of the crop. Government programs will -probably be more effective if implementing biodiesel production is conjoined with stimulating the demand for Jatropha biodiesel. To avoid food-biofuel competition, additional measures may be needed such as land-use restrictions for Jatropha producers and taxes on biofuels or biofuel feedstocks to improve the competitiveness of the food sector compared to the bioenergy sector. (c) 2012 Society of Chemical Industry and John Wiley & Sons, Ltd