Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Fishes of the Darkness : Water colour-regulated competitive interactions in humic lakes

Humic lakes are abundant in the temperate and cold regions of the Boreal Zone. High levels of water colour and strong thermal stratification of humic lakes limit the potential fish habitats and give a special role to the intraspecific and interspecific interactions. Water colour has different effects on species depending on species-specific life-history traits and trophic interactions. Fish species whose success in predation is based on visual cues are more susceptible to suffer in competition. The main aim of the thesis was to demonstrate the effects of water colour on European perch (Perca fluviatilis) in humic lakes. The contribution of water colour to diet, feeding, growth and competitive interactions of fish was studied both in laboratory and in small humic lakes with varying levels of water colour. The main findings of the thesis were that water colour has different effects on species, depending on species-specific life-history traits and trophic interactions. Water colour affected visually-oriented perch feeding and growth negatively, and the prolonged benthic feeding phase of perch resulting from the increased water colour could increase intraspecific competition in perch populations and may result in a partial bottleneck in growth for perch. Moreover, water colour may act as a proximate factor behind the population dependency of sexual growth dimorphism in perch.

Adsorption-nucleation/growth based model for electrochemical phase formation and Avrami ansatz for a multicomponent competitive growth process

The existing models describing electrochemical phase formation involving both adsorption and a nucleation/growth process are modified. The limiting cases leading to the existing models are discussed. The characteristic features of the potentiostatic transients are presented. A generalization of the Avrami ansatz is given for two or more competitive irreversibly growing phases.

A modell of socially optimal hydro reservoir usage - yelding socially optimal outcomes in nordic electricity markets

An extensive electricity transmission network facilitates electricity trading between Finland, Sweden, Norway and Denmark. Currently most of the area's power generation is traded at NordPool, where the trading volumes have steadily increased since the early 1990's, when the exchange was founded. The Nordic electricity is expected to follow the current trend and further integrate with the other European electricity markets. Hydro power is the source for roughly a half of the supply in the Nordic electricity market and most of the hydro is generated in Norway. The dominating role of hydro power distinguishes the Nordic electricity market from most of the other market places. Production of hydro power varies mainly due to hydro reservoirs and demand for electricity. Hydro reservoirs are affected by water inflows that differ each year. The hydro reservoirs explain remarkably the behaviour of the Nordic electricity markets. Therefore among others, Kauppi and Liski (2008) have developed a model that analyzes the behaviour of the markets using hydro reservoirs as explanatory factors. Their model includes, for example, welfare loss due to socially suboptimal hydro reservoir usage, socially optimal electricity price, hydro reservoir storage and thermal reservoir storage; that are referred as outcomes. However, the model does not explain the real market condition but rather an ideal situation. In the model the market is controlled by one agent, i.e. one agent controls all the power generation reserves; it is referred to as a socially optimal strategy. Article by Kauppi and Liski (2008) includes an assumption where an individual agent has a certain fraction of market power, e.g. 20 % or 30 %. In order to maintain the focus of this thesis, this part of their paper is omitted. The goal of this thesis is two-fold. Firstly we expand the results from the socially optimal strategy for years 2006-08, as the earlier study finishes in 2005. The second objective is to improve on the methods from the previous study. This thesis results several outcomes (SPOT-price and welfare loss, etc.) due to socially optimal actions. Welfare loss is interesting as it describes the inefficiency of the market. SPOT-price is an important output for the market participants as it often has an effect on end users' electricity bills. Another function is to modify and try to improve the model by means of using more accurate input data, e.g. by considering pollution trade rights effect on input data. After modifications to the model, new welfare losses are calculated and compared with the same results before the modifications. The hydro reservoir has the higher explanatory significance in the model followed by thermal power. In Nordic markets, thermal power reserves are mostly nuclear power and other thermal sources (coal, natural gas, oil, peat). It can be argued that hydro and thermal reservoirs determine electricity supply. Roughly speaking, the model takes into account electricity demand and supply, and several parameters related to them (water inflow, oil price, etc.), yielding finally the socially optimal outcomes. The author of this thesis is not aware of any similar model being tested before. There have been some other studies that are close to the Kauppi and Liski (2008) model, but those have a somewhat different focus. For example, a specific feature in the model is the focus on long-run capacity usage that differs from the previous studies on short-run market power. The closest study to the model is from California's wholesale electricity markets that, however, uses different methodology. Work is constructed as follows.

Factors Affecting Knowledge Sharing in Strategic Alliances: The Role of Knowledge Sharing as Strategic Control Behavior among Multinational Enterprises

International strategic alliances (ISAs) have become increasingly important for the stability, growth, and long-term viability of modern business organizations. Alliance partnerships as inter-firm cooperative ventures represent an influential mechanism for asserting corporate strategic control among autonomous multinational enterprises. These different cooperative arrangements are made of equity investments or contractually-based partnerships. Different alliance forms represent different approaches that partner firms adopt to control their mutual dependence on the alliance and on other partners. Earlier research shows that the partner characteristics could provide an explanation for alliance strategic behavior and see alliances as alternative forms to markets or hierarchies for addressing specific strategic needs linked to partners’ characteristics and their subsequent strategic motives. These characteristics of the partners’ and subsequent strategic motives are analyzed as knowledge sharing factors and how these influence inter-firm control in alliances within the context of the focal-firm STMicroelectronics and its alliance partners Nokia, Ericsson and IBM. This study underline that as contracts are incomplete, they are therefore required to maintain mutual dependence based control mechanisms in addition to a contract. For example, mutual dependence based control mechanisms could be joint financial investments and the building of an ownership structure between the parties (e.g., JVs). However, the present study clarifies that subsequent inter-firm control is also exercised through inter-firm knowledge sharing. The present study contributes by presenting a dynamic interplay between competitive and cooperative rent seeking behavior. Such coopetition behavior describes the firm's strategic orientation to achieve a dynamic balance between competitive and cooperative strategies. This balance is seen in knowledge sharing based cooperation and competition behavior. Thus this study clarifies coopetition strategies by introducing the role of inter-firm cooperation and the competitive nature of knowledge sharing. Simultaneous cooperative and competitive behavior is also seen as synergetic rent-seeking behavior. Therefore, this study extends the perspective of previous studies on competitive and cooperative seeking behavior.

Rapsista tuotetun biodieselin ja vehnästä tuotetun bioetanolin energiavirrat ja ympäristövaikutukset Euroopan unionissa

The aim of this thesis was to study the crops currently used for biofuel production from the following aspects: 1. what should be the average yield/ ha to reach an energy balance at least 0 or positive 2. what are the shares of the primary and secondary energy flows in agriculture, transport, processing and usage, and 3. overall effects of biofuel crop cultivation, transport, processing and usage. This thesis concentrated on oilseed rape biodiesel and wheat bioethanol in the European Union, comparing them with competing biofuels, such as corn and sugarcane-based ethanol, and the second generation biofuels. The study was executed by comparing Life Cycle Assessment-studies from the EU-region and by analyzing them thoroughly from the differences viewpoint. The variables were the following: energy ratio, hectare yield (l/ha), impact on greenhouse gas emissions (particularly CO2), energy consumption in crop growing and processing one hectare of a particular crop to biofuel, distribution of energy in processing and effects of the secondary energy flows, like e.g. wheat straw. Processing was found to be the most energy consuming part in the production of biofuels. So if the raw materials will remain the same, the development will happen in processing. First generation biodiesel requires esterification, which consumes approximately one third of the process energy. Around 75% of the energy consumed in manufacturing the first generation wheat-based ethanol is spent in steam and electricity generation. No breakthroughs are in sight in the agricultural sector to achieve significantly higher energy ratios. It was found out that even in ideal conditions the energy ratio of first generation wheat-based ethanol will remain slightly under 2. For oilseed rape-based biodiesel the energy ratios are better, and energy consumption per hectare is lower compared to wheat-based ethanol. But both of these are lower compared to e.g. sugarcane-based ethanol. Also the hectare yield of wheat-based ethanol is significantly lower. Biofuels are in a key position when considering the future of the world’s transport sector. Uncertainties concerning biofuels are, however, several, like the schedule of large scale introduction to consumer markets, technologies used, raw materials and their availability and - maybe the biggest - the real production capacity in relation to the fuel consumption. First generation biofuels have not been the expected answer to environmental problems. Comparisons made show that sugarcane-based ethanol is the most prominent first generation biofuel at the moment, both from energy and environment point of view. Also palmoil-based biodiesel looks promising, although it involves environmental concerns as well. From this point of view the biofuels in this study - wheat-based ethanol and oilseed rape-based biodiesel - are not very competitive options. On the other hand, crops currently used for fuel production in different countries are selected based on several factors, not only based on thier relative general superiority. It is challenging to make long-term forecasts for the biofuel sector, but it can be said that satisfying the world's current and near future traffic fuel consumption with biofuels can only be regarded impossible. This does not mean that biofuels shoud be rejected and their positive aspects ignored, but maybe this reality helps us to put them in perspective. To achieve true environmental benefits through the usage of biofuels there must first be a significant drop both in traffic volumes and overall fuel consumption. Second generation biofuels are coming, but serious questions about their availability and production capacities remain open. Therefore nothing can be taken for granted in this issue, expect the need for development.

Ebselen is a potent non-competitive inhibitor of extracellular nucleoside diphosphokinase

Nucleoside di- and triphosphates and adenosine regulate several components of the mucocilairy clearance process (MCC) that protects the lung against infections, via activation of epithelial purinergic receptors. However, assessing the contribution of individual nucleotides to MCC functions remains difficult due to the complexity of the mechanisms of nucleotide release and metabolism. Enzymatic activities involved in the metabolism of extracellular nucleotides include ecto-ATPases and secreted nucleoside diphosphokinase (NDPK) and adenyl kinase, but potent and selective inhibitors of these activities are sparse. In the present study, we discovered that ebselen markedly reduced NDPK activity while having negligible effect on ecto-ATPase and adenyl kinase activities. Addition of radiotracer gamma P-32]ATP to human bronchial epithelial (HBE) cells resulted in rapid and robust accumulation of P-32]-inorganic phosphate ((32)Pi). Inclusion of UDP in the incubation medium resulted in conversion of gamma P-32]ATP to P-32]UTP, while inclusion of AMP resulted in conversion of gamma P-32]ATP to P-32]ADP. Ebselen markedly reduced P-32]UTP formation but displayed negligible effect on (32)Pi or P-32]ADP accumulations. Incubation of HBE cells with unlabeled UTP and ADP resulted in robust ebselen-sensitive formation of ATP (IC50=6.9 +/- 2 mu M). This NDPK activity was largely recovered in HBE cell secretions and supernatants from lung epithelial A549 cells. Kinetic analysis of NDPK activity indicated that ebselen reduced the V-max of the reaction (K-i=7.6 +/- 3 mu M), having negligible effect on KM values. Our study demonstrates that ebselen is a potent noncompetitive inhibitor of extracellular NDPK.

Surface chemical studies on the competitive adsorption of poly(acrylic acid) and poly(vinyl alcohol) onto alumina

The adsorption of poly(acrylic acid) (PAA) and poly(vinyl alcohol) (PVA) onto alumina has been studied as a function of pH, both individually and in the presence of each other. The adsorption density of PAA is found to decrease with an increase of pH while that of PVA shows the opposite trend. In a binary system containing PAA and PVA, the presence of PVA does not affect the adsorption of PAA onto alumina, but the addition of PAA diminishes the adsorption of PVA in the pH range investigated. The adsorption isotherm of PAA at acidic pH exhibits high-affinity Langmuirian behavior. The isotherms for PVA appear rounded and are of the low-affinity type, Once again the adsorption isotherms of PAA remain unaltered in the presence of PVA whereas those of PVA are significantly affected resulting in a lowering of the adsorption density consequent to PAA addition. A variation in the sequence of addition of PAA and PVA does not affect the adsorption behavior of either of the polymers, The electrokinetic behavior of alumina with PAA is hardly influenced by the addition of PVA, On the other hand, the electrophoretic mobility of alumina in the presence of PVA is significantly altered in the presence of PAA and closely resembles the trend observed with PAA alone. Desorption studies reveal that over 80% of PVA could be desorbed in the pH range 3-9 whereas in the case of PAA, the percent desorption increases from 20 to about 70% as the pH is increased from about 3 to 8. Solution conductivity tests confirm interaction of aluminum species and PAA in the bulk solution. FTIR spectroscopic data provide evidence in support of hydrogen bonding and chemical interaction in the case of the PAA-alumina system and hydrogen bonding with respect to the PVA-alumina interaction. (C) 1999 Academic Press.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.