Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Does Uncertainty Affect Investment and Labor Demand?

This paper analyzes the effect of uncertainty on investment and labor demand for Finnish firms during the time period 1987 – 2000. Utilizing a stock return based measure of uncertainty decomposed into systematic and idiosyncratic components, the results reveal that idiosyncratic uncertainty significantly reduces both investment and labor demand. Idiosyncratic uncertainty seems to influence investment in the current period, whereas the depressing effect on labor demand appears with a one-year lag. The results provide support that the depressing effect of idiosyncratic uncertainty on investment is stronger for small firms in comparison to large firms. Some evidence is reported regarding differential effects of uncertainty on labor demand conditional on firm characteristics. Most importantly, the depressing effect of lagged idiosyncratic uncertainty on labor demand tends to be stronger for diversified firms compared with focused firms.

Examining effectiveness of an immigrant labor market integration intervention. The role of acculturation, discrimination, self-efficacy, outcome expectances, action and coping planning.

The designing of effective intervention tools to improve immigrants’ labor market integration remains an important topic in contemporary Western societies. This study examines whether and how a new intervention tool, Working Life Certificate (WLC), helps unemployed immigrants to find employment and strengthen their belief of their vocational skills. The study is based on quantitative longitudinal survey data from 174 unemployed immigrants of various origins who participated in the pilot phase of WLC examinations in 2009. Surveys were administered in three waves: before the test, right after it, and three months later. Although it is often argued that the unemployment among immigrants is due either to their lack of skills and cultural differences or to discrimination in recruitment, scholars within social psychology of behavior change argue that the best way of helping people to achieve their goals (e.g. finding employment) is to build up their sense of self-efficacy, alter their outcome expectances in a more positive direction or to help them to construct more detailed action and coping plans. This study aims to shed light on the role of these concepts in immigrants’ labor market integration. The results support the theories of behavior change moderately. Having positive expectances regarding the outcomes of various job search behaviors was found to predict employment in the future. Together with action and coping planning it also predicted increase in job search behavior. The intervention, WLC, was able to affect participants’ self-efficacy, but contrary to expectations, self-efficacy was found not to be related to either job search behavior or future labor market status. Also, perceived discrimination did not explain problems in finding employment, but hints of subtle or structural discrimination were found. Adoption of Finnish work culture together with strong family culture was found to predict future employment. Hence, in this thesis I argue that awarding people diplomas should be preferred in immigrant integration training as it strengthens people’s sense of self-efficacy. Instead of teaching new information, more attention should be directed at changing people’s outcome expectances in a more positive direction and helping them to construct detailed plans on how to achieve their goals.

Reasonability and the Linguistic - Division of Labor in Institutional Work

We examine institutional work from a discursive perspective and argue that reasonability, the existence of acceptable justifying reasons for beliefs and practices, is a key part of legitimation. Drawing on philosophy of language, we maintain that institutional work takes place in the context of ‘space of reasons’ determined by widely held assumptions about what is reasonable and what is not. We argue that reasonability provides the main contextual constraint of institutional work, its major outcome, and a key trigger for actors to engage in it. We draw on Hilary Putnam’s concept ‘division of linguistic labor’ to highlight the specialized distribution of knowledge and authority in defining valid ways of reasoning. In this view, individuals use institutionalized vocabularies to reason about their choices and understand their context with limited understanding of how and why these structures have become what they are. We highlight the need to understand how professions and other actors establish and maintain the criteria of reasoning in various areas of expertise through discursive institutional work.

A modell of socially optimal hydro reservoir usage - yelding socially optimal outcomes in nordic electricity markets

An extensive electricity transmission network facilitates electricity trading between Finland, Sweden, Norway and Denmark. Currently most of the area's power generation is traded at NordPool, where the trading volumes have steadily increased since the early 1990's, when the exchange was founded. The Nordic electricity is expected to follow the current trend and further integrate with the other European electricity markets. Hydro power is the source for roughly a half of the supply in the Nordic electricity market and most of the hydro is generated in Norway. The dominating role of hydro power distinguishes the Nordic electricity market from most of the other market places. Production of hydro power varies mainly due to hydro reservoirs and demand for electricity. Hydro reservoirs are affected by water inflows that differ each year. The hydro reservoirs explain remarkably the behaviour of the Nordic electricity markets. Therefore among others, Kauppi and Liski (2008) have developed a model that analyzes the behaviour of the markets using hydro reservoirs as explanatory factors. Their model includes, for example, welfare loss due to socially suboptimal hydro reservoir usage, socially optimal electricity price, hydro reservoir storage and thermal reservoir storage; that are referred as outcomes. However, the model does not explain the real market condition but rather an ideal situation. In the model the market is controlled by one agent, i.e. one agent controls all the power generation reserves; it is referred to as a socially optimal strategy. Article by Kauppi and Liski (2008) includes an assumption where an individual agent has a certain fraction of market power, e.g. 20 % or 30 %. In order to maintain the focus of this thesis, this part of their paper is omitted. The goal of this thesis is two-fold. Firstly we expand the results from the socially optimal strategy for years 2006-08, as the earlier study finishes in 2005. The second objective is to improve on the methods from the previous study. This thesis results several outcomes (SPOT-price and welfare loss, etc.) due to socially optimal actions. Welfare loss is interesting as it describes the inefficiency of the market. SPOT-price is an important output for the market participants as it often has an effect on end users' electricity bills. Another function is to modify and try to improve the model by means of using more accurate input data, e.g. by considering pollution trade rights effect on input data. After modifications to the model, new welfare losses are calculated and compared with the same results before the modifications. The hydro reservoir has the higher explanatory significance in the model followed by thermal power. In Nordic markets, thermal power reserves are mostly nuclear power and other thermal sources (coal, natural gas, oil, peat). It can be argued that hydro and thermal reservoirs determine electricity supply. Roughly speaking, the model takes into account electricity demand and supply, and several parameters related to them (water inflow, oil price, etc.), yielding finally the socially optimal outcomes. The author of this thesis is not aware of any similar model being tested before. There have been some other studies that are close to the Kauppi and Liski (2008) model, but those have a somewhat different focus. For example, a specific feature in the model is the focus on long-run capacity usage that differs from the previous studies on short-run market power. The closest study to the model is from California's wholesale electricity markets that, however, uses different methodology. Work is constructed as follows.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Electrochemical capacitors: Technical challenges and prognosis for future markets

Electrochemical capacitors are potential devices that could help bringing about major advances in future energy storage. They are lightweight and their manufacture and disposal has no detrimental effects on the environment. A comprehensive description of fundamental science of electrochemical capacitors is presented. Similarities and differences between electrochemical capacitors and secondary batteries for electrical energy storage are highlighted and various types of electrochemical capacitors are discussed with special reference to lead-carbon hybrid ultracapacitors. Some envisaged applications of electrochemical capacitors are described along with the technical challenges and prognosis for future markets. (C) 2012 Published by Elsevier Ltd.

Dynamic mechanism design for markets with strategic resources

The assignment of tasks to multiple resources becomes an interesting game theoretic problem, when both the task owner and the resources are strategic. In the classical, nonstrategic setting, where the states of the tasks and resources are observable by the controller, this problem is that of finding an optimal policy for a Markov decision process (MDP). When the states are held by strategic agents, the problem of an efficient task allocation extends beyond that of solving an MDP and becomes that of designing a mechanism. Motivated by this fact, we propose a general mechanism which decides on an allocation rule for the tasks and resources and a payment rule to incentivize agents' participation and truthful reports. In contrast to related dynamic strategic control problems studied in recent literature, the problem studied here has interdependent values: the benefit of an allocation to the task owner is not simply a function of the characteristics of the task itself and the allocation, but also of the state of the resources. We introduce a dynamic extension of Mezzetti's two phase mechanism for interdependent valuations. In this changed setting, the proposed dynamic mechanism is efficient, within period ex-post incentive compatible, and within period ex-post individually rational.