Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Electrochemical capacitors: Technical challenges and prognosis for future markets

Electrochemical capacitors are potential devices that could help bringing about major advances in future energy storage. They are lightweight and their manufacture and disposal has no detrimental effects on the environment. A comprehensive description of fundamental science of electrochemical capacitors is presented. Similarities and differences between electrochemical capacitors and secondary batteries for electrical energy storage are highlighted and various types of electrochemical capacitors are discussed with special reference to lead-carbon hybrid ultracapacitors. Some envisaged applications of electrochemical capacitors are described along with the technical challenges and prognosis for future markets. (C) 2012 Published by Elsevier Ltd.

Dynamic mechanism design for markets with strategic resources

The assignment of tasks to multiple resources becomes an interesting game theoretic problem, when both the task owner and the resources are strategic. In the classical, nonstrategic setting, where the states of the tasks and resources are observable by the controller, this problem is that of finding an optimal policy for a Markov decision process (MDP). When the states are held by strategic agents, the problem of an efficient task allocation extends beyond that of solving an MDP and becomes that of designing a mechanism. Motivated by this fact, we propose a general mechanism which decides on an allocation rule for the tasks and resources and a payment rule to incentivize agents' participation and truthful reports. In contrast to related dynamic strategic control problems studied in recent literature, the problem studied here has interdependent values: the benefit of an allocation to the task owner is not simply a function of the characteristics of the task itself and the allocation, but also of the state of the resources. We introduce a dynamic extension of Mezzetti's two phase mechanism for interdependent valuations. In this changed setting, the proposed dynamic mechanism is efficient, within period ex-post incentive compatible, and within period ex-post individually rational.

Analysis of volatility transmissions in integrated and interconnected markets: The case of the Iberian and French markets

This paper models the mean and volatility spillovers of prices within the integrated Iberian and the interconnected Spanish and French electricity markets. Using the constant (CCC) and dynamic conditional correlation (DCC) bivariate models with three different specifications of the univariate variance processes, we study the extent to which increasing interconnection and harmonization in regulation have favoured price convergence. The data consist of daily prices calculated as the arithmetic mean of the hourly prices over a span from July 1st 2007 until February 29th 2012. The DCC model in which the variances of the univariate processes are specified with a VARMA(1,1) fits the data best for the integrated MIBEL whereas a CCC model with a GARCH(1,1) specification for the univariate variance processes is selected to model the price series in Spain and France. Results show that there are significant mean and volatility spillovers in the MIBEL, indicating strong interdependence between the two markets, while there is a weaker evidence of integration between the Spanish and French markets. We provide new evidence that the EU target of achieving a single electricity market largely depends on increasing trade between countries and homogeneous rules of market functioning.

Long memory in return structures from developed markets

[En]The present study aimed at investigating the existence of long memory properties in ten developed stock markets across the globe. When return series exhibit long memory, the series realizations are not independent over time and past returns can help predict future returns, thus violating the market efficiency hypothesis. It poses a serious challenge to the supporters of random walk behavior of the stock returns indicating a potentially predictable component in the series dynamics. We computed Hurst-Mandelbrot’s Classical R/S statistic, Lo’s statistic and semi parametric GPH statistic using spectral regression. The findings suggest existence of long memory in volatility and random walk for logarithmic return series in general for all the selected stock market indices. Findings are in line with the stylized facts of financial time series.

Markets and microstructure

This document contains three papers examining the microstructure of financial interaction in development and market settings. I first examine the industrial organization of financial exchanges, specifically limit order markets. In this section, I perform a case study of Google stock surrounding a surprising earnings announcement in the 3rd quarter of 2009, uncovering parameters that describe information flows and liquidity provision. I then explore the disbursement process for community-driven development projects. This section is game theoretic in nature, using a novel three-player ultimatum structure. I finally develop econometric tools to simulate equilibrium and identify equilibrium models in limit order markets.

In chapter two, I estimate an equilibrium model using limit order data, finding parameters that describe information and liquidity preferences for trading. As a case study, I estimate the model for Google stock surrounding an unexpected good-news earnings announcement in the 3rd quarter of 2009. I find a substantial decrease in asymmetric information prior to the earnings announcement. I also simulate counterfactual dealer markets and find empirical evidence that limit order markets perform more efficiently than do their dealer market counterparts.

In chapter three, I examine Community-Driven Development. Community-Driven Development is considered a tool empowering communities to develop their own aid projects. While evidence has been mixed as to the effectiveness of CDD in achieving disbursement to intended beneficiaries, the literature maintains that local elites generally take control of most programs. I present a three player ultimatum game which describes a potential decentralized aid procurement process. Players successively split a dollar in aid money, and the final player--the targeted community member--decides between whistle blowing or not. Despite the elite capture present in my model, I find conditions under which money reaches targeted recipients. My results describe a perverse possibility in the decentralized aid process which could make detection of elite capture more difficult than previously considered. These processes may reconcile recent empirical work claiming effectiveness of the decentralized aid process with case studies which claim otherwise.

In chapter four, I develop in more depth the empirical and computational means to estimate model parameters in the case study in chapter two. I describe the liquidity supplier problem and equilibrium among those suppliers. I then outline the analytical forms for computing certainty-equivalent utilities for the informed trader. Following this, I describe a recursive algorithm which facilitates computing equilibrium in supply curves. Finally, I outline implementation of the Method of Simulated Moments in this context, focusing on Indirect Inference and formulating the pseudo model.

An integrated design approach to power systems : from power flows to electricity markets

Power system is at the brink of change. Engineering needs, economic forces and environmental factors are the main drivers of this change. The vision is to build a smart electrical grid and a smarter market mechanism around it to fulfill mandates on clean energy. Looking at engineering and economic issues in isolation is no longer an option today; it needs an integrated design approach. In this thesis, I shall revisit some of the classical questions on the engineering operation of power systems that deals with the nonconvexity of power flow equations. Then I shall explore some issues of the interaction of these power flow equations on the electricity markets to address the fundamental issue of market power in a deregulated market environment. Finally, motivated by the emergence of new storage technologies, I present an interesting result on the investment decision problem of placing storage over a power network. The goal of this study is to demonstrate that modern optimization and game theory can provide unique insights into this complex system. Some of the ideas carry over to applications beyond power systems.

Financial Futures Markets and Their Impact on the Real Economy in Middle and Low Income Countries

This project analyses the influence of the futures market on middle and low income countries. In it, I attempt to show that investments made by large investment funds in this market, as well as by certain pension plans, bring major consequences whose effects are more evident in less developed countries. The cornerstones of the work are as follows; to attempt to see the existing relationship between the commodity futures market and its underlying assets; analysing products such as wheat, rice and corn in-depth, because these are the most basic foodstuffs at a global level; to determine how an increase in trading in these markets can affect the lives of people in the poorest countries; to analyse investor concern regarding the consequences that their investments may have. Throughout the project we will see how large speculators use production forecasting models to determine the shortage of a commodity in order to take a position in the futures market to profit from it. In addition we will see how an increase in trading in this market causes an increase in the price of the underlying asset in the spot market. As for investor concern, I can say it is negligible, but the idea of running pension plans or investment funds that follow some social criteria has been welcomed by those interviewed, which makes me think that different legislation is possible. This legislation will only come into existence if it is demanded by the people. A fact that now becomes complicated because without a minimum financial basis, they cannot even know how the large investment funds trade with hunger in the world. The day when most people understand how large speculators profit from famine will be the day to put pressure on governments to begin to put limits on speculation. This makes financial awareness necessary in order to achieve a curb in excessive speculation.

Markets, marketing and production issues for aquaculture in East Africa: the case of Uganda

Aquaculture is currently responsible for an insignificant proportion of total fish production in Uganda. However, given the increasing demand for fresh fish in urban and peri-urban araes, and threats to the supply of fish from natural catch fisheries, the potential exists for a strong market in aquaculture. Small-scale fish farmers located relatively close to markets or all-season roads, and who can supply consistent and high quality produce, will have the widest range of marketing opportunities, and will likely be within the area of operation of potential traders and intermediaries that deliver fish to markets. Fish farmers that are not close to roads, or produce unreliable quantities and variable quality products may face high transaction costs of marketing their product, and decreasing net returns to production. The authors found that significant on-farm labor, and access to input markets are important factors leading to positive net returns to fish production. Areas with high population density and relatively low wages will be well suited to labor intensive aquaculture. The authors concluded that aquaculture development has good potential in certain areas of Uganda and should therefore be pursued as a potential development pathway. However, policy makers should consider the importance of the price of fresh fish relative to the cost of labor, as well as other factors including the importance of smallholder credit and access to extension services, when directing investments in aquaculture technology.

New Products and Markets for Menhaden, Brevoortia spp.

Although menhaden, Brevoortia spp., represent 23.5 percent of United States commercial fishery landings, they represent only about 2.6 percent of the total landed value of fishery products. New food products and markets are needed to increase the economic value of the menhaden resource. This paper describes investigations of menhaden as a raw material for both traditional and new forms of food products. Canned menhaden is a logical food product, but the production of a menhaden surimi with good functionality has recently been demonstrated. The U.S. Food and Drug Administration has placed partially hydrogenated menhaden oil on the GRAS list of ingredients for food products, but a decision on the status of nutritionally beneficial refined menhaden oil is not yet available. Refined menhaden oil is currently the raw material for biomedical test materials being used in research approved by the National Institutes of Health to determine the health benefits of fish oils and omega-3 fatty acids. The test materials are being produced, with strict quality controls, at the NMFS Charleston Laboratory.

Research in Global Groundfish Markets: An Exercise in International Cooperation

Over roughly the last decade, most of the fishery resources of the continental shelf and nearshore areas of the world's oceans have come under the control of coastal nations. One consequence of this extension of fisheries jurisdiction (EFJ) by any individual state has been the expansion of its production possibilities. That is, with strengthened property rights in the ocean resources off its shores, a coastal nation experiences increased opportunities to produce goods and services from its newly enlarged pool of resources. Such a nation, then, would appear to be a potential gainer from EFJ.