Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Groundwater Management in India:Physical, Institutional and Policy Alternatives

Groundwater constitutes a vital natural resource for sustaining India’s agricultural economy and meeting the country’s social, ecological and environmental goals. It is a unique resource, widely available, providing security against droughts and yet it is closely linked to surface-water resources and the hydrological cycle. Its availability depends on geo-hydrological conditions and characteristics of aquifers, from deep to alluvium, sediment crystalline rocks to basalt formations; and agro-climate from humid to subhumid and semi-arid to arid. Its reliable supply, uniform quality and temperature, relative turbidity, pollution-safe, minimal evaporation losses, and low cost of development are attributes making groundwater more attractive compared to other resources. It plays a key role in the provision of safe drinking water to rural populations. For example, already almost 80% of domestic water use in rural areas in India is groundwater-supplied, and much of it is being supplied to farms, villages and small towns. Inadequate control of the use of groundwater, indiscriminate application of agrochemicals and unrestrained pollution of the rural environment by other human activities make groundwater usage unsustainable, necessitating proper management in the face of the twin demand for water of good quality for domestic supply and adequate supply for irrigation, ensuring equity, efficiency and sustainability of the resource. Groundwater irrigation has overtaken surface irrigation in the early 1980s, supported by well energization. It is estimated that there are about 24 million energised wells and tube wells now and it is driven by demand rather than availability, evident through the greater occurrence of wells in districts with high population densities. Apart from aquifer characteristics, land fragmentation and landholding size are the factors that decide the density of wells. The ‘rise and fall’ of local economies dependent on groundwater can be summarized as: the green revolution of 1980s, groundwaterbased agrarian boom, early symptoms of groundwater overdraft, and decline of the groundwater socio-ecology. The social characteristics and policy interventions typical of each stage provide a fascinating insight into the human-resource dynamics. This book is a compilation of nine research papers discussing various aspects of groundwater management. It attempts to integrate knowledge about the physical system, the socio-economic system, the institutional set-up and the policy environment to come out with a more realistic analysis of the situation with regard to the nature, characteristics and intensity of resource use, the size of the economy the use generates, and the negative socioeconomic consequences. Complex variables addressed in this regard focusing on northern Gujarat are the stock of groundwater available in the region, its hydrodynamics, its net outflows against inflows, the economics of its intensive use (particularly irrigation in semi-arid and arid regions), its criticality in the regional hydroecological regime, ethical aspects and social aspects of its use. The first chapter by Dinesh Kumar and Singh, dwells on complex groundwater socio-ecology of India, while emphasizing the need for policy measures to address indiscriminate over-exploitation of dwindling resources. The chapter also explores the nature of groundwater economy and the role of electricity prices on it. The next chapter on groundwater issue in north Gujarat provides a description of groundwater resource characteristics followed by a detailed analysis of the groundwater depletion and quality deterioration problems in the region and their undesirable consequences on the economy, ecosystem health and the society. Considering water-buyers and wellowning farmers individually, a methodology for economic valuation of groundwater in regions where its primary usage is in agriculture, and as assessment of the groundwater economy based on case studies from north Gujarat is presented in the fourth chapter. The next chapter focuses on the extent of dependency of milk production on groundwater, which includes the water embedded in green and dry fodder and animal feed. The study made a realistic estimate of irrigation water productivity in terms of the physics and economics of milk production. The sixth chapter analyses the extent of reduction in water usage, increase in yield and overall increase in physical productivity of alfalfa with the use of the drip irrigation system. The chapter also provides a detailed synthesis of the costs and benefits associated with the use of drip irrigation systems. A linear programmingbased optimization model with the objective to minimize groundwater use taking into account the interaction between two distinct components – farming and dairying under the constraints of food security and income stability for different scenarios, including shift in cropping pattern, introduction of water-efficient crops, water- saving technologies in addition to the ‘business as usual’ scenario is presented in the seventh chapter. The results show that sustaining dairy production in the region with reduced groundwater draft requires crop shifts and adoption of water-saving technologies. The eighth chapter provides evidences to prove that the presence of adequate economic incentive would encourage farmers to adopt water-saving irrigation devices, based on the findings of market research with reference to the level of awareness among farmers of technologies and the factors that decide the adoption of water-saving technologies. However, now the marginal cost of using electricity for agricultural pumping is almost zero. The economic incentives are strong and visible only when the farmers are either water-buyers or have to manage irrigation with limited water from tube-well partnerships. The ninth chapter explores the socio-economic viability of increasing the power tariff and inducing groundwater rationing as a tool for managing energy and groundwater demand, considering the current estimate of the country’s annual economic loss of Rs 320 billion towards electricity subsidy in the farm sector. The tenth chapter suggests private tradable property rights and development of water markets as the institutional tool for achieving equity, efficiency and sustainability of groundwater use. It identifies the externalities for local groundwater management and emphasizes the need for managing groundwater by local user groups, supported by a thorough analysis of groundwater socio-ecology in India. An institutional framework for managing the resource based on participatory approach that is capable of internalizing the externalities, comprising implementation of institutional and technical alternatives for resource management is also presented. Major findings of the analyses and key arguments in each chapter are summarized in the concluding chapter. Case studies of the social and economic benefits of groundwater use, where that use could be described as unsustainable, are interesting. The benefits of groundwater use are outlined and described with examples of social and economic impacts of groundwater and the negative aspects of groundwater development with the compilation of environmental problems based on up-to-date research results. This publication with a well-edited compilation of case studies is informative and constitutes a useful publication for students and professionals.

Soil and Groundwater Pollution from Agricultural Activities

This book introduces the major agricultural activities in India and their impact on soil and groundwater. It lists the basic aspects of agricultural activities and introduces soil properties, classification and processes, and groundwater characteristics, movement, and recharge aspects. It further discusses soil and groundwater pollution from various sources, impacts of irrigation, drainage, fertilizer, and pesticide. Finally, the book dwells upon conservation and management of groundwater and soil.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Modeling and optimization of a natural convection-based agricultural dryer

This study presents development of a computational fluid dynamic (CFD) model to predict unsteady, two-dimensional temperature, moisture and velocity distributions inside a novel, biomass-fired, natural convection-type agricultural dryer. Results show that in initial stages of drying, when material surface is wet and moisture is easily available, moisture removal rate from surface depends upon the condition of drying air. Subsequently, material surface becomes dry and moisture removal rate is driven by diffusion of moisture from inside to the material surface. An optimum 9-tray configuration is found to be more efficient than for the same mass of material and volume of dryer. A new configuration of dryer, mainly to explore its potential to increasing uniformity in drying across all trays, is also analyzed. This configuration involves diverting a portion of hot air before it enters over the first tray and is supplied directly at an intermediate location in the dryer. Uniformity in drying across trays has increased for the kind of material simulated.

Electrochemical capacitors: Technical challenges and prognosis for future markets

Electrochemical capacitors are potential devices that could help bringing about major advances in future energy storage. They are lightweight and their manufacture and disposal has no detrimental effects on the environment. A comprehensive description of fundamental science of electrochemical capacitors is presented. Similarities and differences between electrochemical capacitors and secondary batteries for electrical energy storage are highlighted and various types of electrochemical capacitors are discussed with special reference to lead-carbon hybrid ultracapacitors. Some envisaged applications of electrochemical capacitors are described along with the technical challenges and prognosis for future markets. (C) 2012 Published by Elsevier Ltd.

Dynamic mechanism design for markets with strategic resources

The assignment of tasks to multiple resources becomes an interesting game theoretic problem, when both the task owner and the resources are strategic. In the classical, nonstrategic setting, where the states of the tasks and resources are observable by the controller, this problem is that of finding an optimal policy for a Markov decision process (MDP). When the states are held by strategic agents, the problem of an efficient task allocation extends beyond that of solving an MDP and becomes that of designing a mechanism. Motivated by this fact, we propose a general mechanism which decides on an allocation rule for the tasks and resources and a payment rule to incentivize agents' participation and truthful reports. In contrast to related dynamic strategic control problems studied in recent literature, the problem studied here has interdependent values: the benefit of an allocation to the task owner is not simply a function of the characteristics of the task itself and the allocation, but also of the state of the resources. We introduce a dynamic extension of Mezzetti's two phase mechanism for interdependent valuations. In this changed setting, the proposed dynamic mechanism is efficient, within period ex-post incentive compatible, and within period ex-post individually rational.

Improved modeling of groundwater recharge in agricultural watersheds using a combination of crop model and remote sensing

For improved water management and efficiency of use in agriculture, studies dealing with coupled crop-surface water-groundwater models are needed. Such integrated models of crop and hydrology can provide accurate quantification of spatio-temporal variations of water balance parameters such as soil moisture store, evapotranspiration and recharge in a catchment. Performance of a coupled crop-hydrology model would depend on the availability of a calibrated crop model for various irrigated/rainfed crops and also on an accurate knowledge of soil hydraulic parameters in the catchment at relevant scale. Moreover, such a coupled model should be designed so as to enable the use/assimilation of recent satellite remote sensing products (optical and microwave) in order to model the processes at catchment scales. In this study we present a framework to couple a crop model with a groundwater model for applications to irrigated groundwater agricultural systems. We discuss the calibration of the STICS crop model and present a methodology to estimate the soil hydraulic parameters by inversion of crop model using both ground and satellite based data. Using this methodology we demonstrate the feasibility of estimation of potential recharge due to spatially varying soil/crop matrix.

Integrating biodiversity and conservation with modern agricultural landscapes

To achieve food security and meet the demands of the ever-growing human populations, farming systems have assumed unsustainable practices to produce more from a finite land area. This has been cause for concern mainly due to the often-irreversible damage done to the otherwise productive agricultural landscapes. Agro-ecology is proclaimed to be deteriorating due to eroding integrity of connected ecological mosaics and vulnerability to climate change. This has contributed to declining species diversity, loss of buffer vegetation, fragmentation of habitats, and loss of natural pollinators or predators, which eventually leads to decline in ecosystem services. Currently, a hierarchy of conservation initiatives is being considered to restore ecological integrity of agricultural landscapes. However, the challenge of identifying a suitable conservation strategy is a daunting task in view of socio-ecological factors that may constrain the choice of available strategies. One way to mitigate this situation and integrate biodiversity with agricultural landscapes is to implement offset mechanisms, which are compensatory and balancing approaches to restore the ecological health and function of an ecosystem. This needs to be tailored to the history of location specific agricultural practices, and the social, ecological and environmental conditions. The offset mechanisms can complement other initiatives through which farmers are insured against landscape-level risks such as droughts, fire and floods. For countries in the developing world with significant biodiversity and extensive agriculture, we should promote a comprehensive model of sustainable agricultural landscapes and ecosystem services, replicable at landscape to regional scales. Arguably, the model can be a potential option to sustain the integrity of biodiversity mosaic in agricultural landscapes.

Off-site impacts of agricultural composting: role of terrestrially derived organic matter in structuring aquatic microbial communities and their metabolic potential

While considered as sustainable and low-cost agricultural amendments, the impacts of organic fertilizers on downstream aquatic microbial communities remain poorly documented. We investigated the quantity and quality of the dissolved organic matter leaching from agricultural soil amended with compost, vermicompost or biochar and assessed their effects on lake microbial communities, in terms of viral and bacterial abundances, community structure and metabolic potential. The addition of compost and vermicompost significantly increased the amount of dissolved organic carbon in the leachate compared with soil alone. Leachates from these additions, either with or without biochar, were highly bioavailable to aquatic microbial communities, although reducing the metabolic potential of the community and harbouring more specific communities. Although not affecting bacterial richness or taxonomic distributions, the specific addition of biochar affected the original lake bacterial communities, resulting in a strongly different community. This could be partly explained by viral burst and converging bacterial abundances throughout the samples. These results underline the necessity to include off-site impacts of agricultural amendments when considering their cascading effect on downstream aquatic ecosystems.

Using long time series of agricultural-derived nitrates for estimating catchment transit times

The estimation of water and solute transit times in catchments is crucial for predicting the response of hydrosystems to external forcings (climatic or anthropogenic). The hydrogeochemical signatures of tracers (either natural or anthropogenic) in streams have been widely used to estimate transit times in catchments as they integrate the various processes at stake. However, most of these tracers are well suited for catchments with mean transit times lower than about 4-5 years. Since the second half of the 20th century, the intensification of agriculture led to a general increase of the nitrogen load in rivers. As nitrate is mainly transported by groundwater in agricultural catchments, this signal can be used to estimate transit times greater than several years, even if nitrate is not a conservative tracer. Conceptual hydrological models can be used to estimate catchment transit times provided their consistency is demonstrated, based on their ability to simulate the stream chemical signatures at various time scales and catchment internal processes such as N storage in groundwater. The objective of this study was to assess if a conceptual lumped model was able to simulate the observed patterns of nitrogen concentration, at various time scales, from seasonal to pluriannual and thus if it was relevant to estimate the nitrogen transit times in headwater catchments. A conceptual lumped model, representing shallow groundwater flow as two parallel linear stores with double porosity, and riparian processes by a constant nitrogen removal function, was applied on two paired agricultural catchments which belong to the Research Observatory ORE AgrHys. The Global Likelihood Uncertainty Estimation (GLUE) approach was used to estimate parameter values and uncertainties. The model performance was assessed on (i) its ability to simulate the contrasted patterns of stream flow and stream nitrate concentrations at seasonal and inter-annual time scales, (ii) its ability to simulate the patterns observed in groundwater at the same temporal scales, and (iii) the consistency of long-term simulations using the calibrated model and the general pattern of the nitrate concentration increase in the region since the beginning of the intensification of agriculture in the 1960s. The simulated nitrate transit times were found more sensitive to climate variability than to parameter uncertainty, and average values were found to be consistent with results from others studies in the same region involving modeling and groundwater dating. This study shows that a simple model can be used to simulate the main dynamics of nitrogen in an intensively polluted catchment and then be used to estimate the transit times of these pollutants in the system which is crucial to guide mitigation plans design and assessment. (C) 2015 Elsevier B.V. All rights reserved.