Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Optimal multi-layered congestion based pricing schemes for enhanced QoS

Pricing is an effective tool to control congestion and achieve quality of service (QoS) provisioning for multiple differentiated levels of service. In this paper, we consider the problem of pricing for congestion control in the case of a network of nodes with multiple queues and multiple grades of service. We present a closed-loop multi-layered pricing scheme and propose an algorithm for finding the optimal state dependent price levels for individual queues, at each node. This is different from most adaptive pricing schemes in the literature that do not obtain a closed-loop state dependent pricing policy. The method that we propose finds optimal price levels that are functions of the queue lengths at individual queues. Further, we also propose a variant of the above scheme that assigns prices to incoming packets at each node according to a weighted average queue length at that node. This is done to reduce frequent price variations and is in the spirit of the random early detection (RED) mechanism used in TCP/IP networks. We observe in our numerical results a considerable improvement in performance using both of our schemes over that of a recently proposed related scheme in terms of both throughput and delay performance. In particular, our first scheme exhibits a throughput improvement in the range of 67-82% among all routes over the above scheme. (C) 2011 Elsevier B.V. All rights reserved.

Pricing credit derivatives

The financial crisis set off by the default of Lehman Brothers in 2008 leading to disastrous consequences for the global economy has focused attention on regulation and pricing issues related to credit derivatives. Credit risk refers to the potential losses that can arise due to the changes in the credit quality of financial instruments. These changes could be due to changes in the ratings, market price (spread) or default on contractual obligations. Credit derivatives are financial instruments designed to mitigate the adverse impact that may arise due to credit risks. However, they also allow the investors to take up purely speculative positions. In this article we provide a succinct introduction to the notions of credit risk, the credit derivatives market and describe some of the important credit derivative products. There are two approaches to pricing credit derivatives, namely the structural and the reduced form or intensity-based models. A crucial aspect of the modelling that we touch upon briefly in this article is the problem of calibration of these models. We hope to convey through this article the challenges that are inherent in credit risk modelling, the elegant mathematics and concepts that underlie some of the models and the importance of understanding the limitations of the models.

A simulation-based algorithm for optimal pricing policy under demand uncertainty

We propose a simulation-based algorithm for computing the optimal pricing policy for a product under uncertain demand dynamics. We consider a parameterized stochastic differential equation (SDE) model for the uncertain demand dynamics of the product over the planning horizon. In particular, we consider a dynamic model that is an extension of the Bass model. The performance of our algorithm is compared to that of a myopic pricing policy and is shown to give better results. Two significant advantages with our algorithm are as follows: (a) it does not require information on the system model parameters if the SDE system state is known via either a simulation device or real data, and (b) as it works efficiently even for high-dimensional parameters, it uses the efficient smoothed functional gradient estimator.

Regulation of Off-Network Pricing in a Nonneutral Network

Representatives of several Internet service providers (ISPs) have expressed their wish to see a substantial change in the pricing policies of the Internet. In particular, they would like to see content providers (CPs) pay for use of the network, given the large amount of resources they use. This would be in clear violation of the ``network neutrality'' principle that had characterized the development of the wireline Internet. Our first goal in this article is to propose and study possible ways of implementing such payments and of regulating their amount. We introduce a model that includes the users' behavior, the utilities of the ISP and of the CPs, and, the monetary flow that involves the content users, the ISP and CP, and, in pUrticular, the CP's revenues from advertisements. We consider various game models and study the resulting equilibria; they are all combinations of a noncooperative game (in which the ISPs and CPs determine how much they will charge the users) with a ``cooperative'' one on how the CP and the ISP share the payments. We include in our model a possible asymmetric weighting parameter (that varies between zero to one). We also study equilibria that arise when one of the CPs colludes with the TSP. We also study two dynamic game models as well as the convergence of prices to the equilibrium values.

Pricing and Power allocation in Femtocells using Stackelberg Game Theory

We consider a system with multiple Femtocells operating in a Macrocell. The transmissions in one Femtocell interfere with its neighboring Femtocells as well as with the Macrocell Base Station. We model Femtocells as selfish nodes and the Macrocell Base Station protects itself by pricing subchannels for each usage. We use Stackelberg game model to study this scenario and obtain equilibrium policies that satisfy certain quality of service.

Contingent sensitivity to configuration in a nonlinear evolution system

A trans-scopic sensitivity of macroscopic failure to slight differentiation in the meso-scopic structure of a system with nonlinear evolution is reported. A periodical chain following a non-local load-sharing evolution was applied as a propotype in failure study. The results demonstrate that there is a transition region composed of globally stable (GS) and evolution induced catastrophic (EIC) modes. That is different from a critical threshold as predicted by percolation and renormalization group theories. Moreover, the EIC mode shows a distinctive sample specific behaviour. For instance, some neighbouring initial states may evolve into completely different final states, though different initial states can evolve into the same final states. As an example, a marginal configuration of EIC mode, a quasi-Fibonacci skeleton, is constructed.

The Impact of Regulation on Pricing Behavior in the Spanish Electricity Market

In this paper we measure the impact of regulatory measures which affected the Spanish electricity wholesale market in the period 2002-2005. Our approach is based on the fact that regulation changes firms' incentives and therefore their market behavior. In the absence of any regulation firms would choose profit- maximizing prices on their residual demands so that the observed gap between optimal and actual prices provides a measure of the effect of regulation. Our results indicate that regulation has decreased wholesale prices considerably, but became less effective at the end of the sample period which explains the change of regulatory regime introduced in 2006.

An empirical comparison of the performance of alternative option pricing models

Published as an article in: Investigaciones Economicas, 2005, vol. 29, issue 3, pages 483-523.

Asset pricing and systematic liquidity risk: an empirical investigation of the Spanish stock market

Systematic liquidity shocks should affect the optimal behavior of agents in financial markets. Indeed, fluctuations in various measures of liquidity are significantly correlated across common stocks. Accordingly, this paper empirically analyzes whether Spanish average returns vary cross-sectionally with betas estimated relative to two competing liquidity risk factors. The first one, proposed by Pastor and Stambaugh (2002), is associated with the strength of volume-related return reversals. Our marketwide liquidity factor is defined as the difference between returns highly sensitive to changes in the relative bid-ask spread and returns with low sensitivities to those changes. Our empirical results show that neither of these proxies for systematic liquidity risk seems to be priced in the Spanish stock market. Further international evidence is deserved.

Consumer Expertise or Credit Risk? An empirical analysis of mortgage pricing

Loan mortgage interest rates are usually the result of a bank-customer negotiation process. Credit risk, consumer cross-buying potential, bundling, financial market competition and other features affecting the bargaining power of the parties could affect price. We argue that, since mortgage loan is a complex product, consumer expertise could be a relevant factor for mortgage pricing. Using data on mortgage loan prices for a sample of 1055 households for the year 2005 (Bank of Spain Survey of Household Finances, EFF-2005), and including credit risk, costs, potential capacity of the consumer to generate future business and bank competition variables, the regression results indicate that consumer expertise-related metrics are highly significant as predictors of mortgage loan prices. Other factors such as credit risk and consumer cross-buying potential do not have such a significant impact on mortgage prices. Our empirical results are affected by the credit conditions prior to the financial crisis and could shed some light on this issue.

Environmental Concerns in Water Pricing Policy: an Application of Data Envelopment Analysis (DEA)

25 p.