Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

How to write a passable technical paper

In my job I see many students who have not learned to write a technical paper. When they do competent work, I want them to be able to write passable reports. This article is for them. There are well established principles for citing the relevant work of others; for not copying things without giving credit; for not stealing. If a reader feels you have copied anything, a figure, even a phrase, from elsewhere without citing its source, then you are guilty of plagiarism in the eyes of that reader. Committing plagiarism is so bad that I cannot do justice to it here. So I merely say: never do it. On, then, to writing your own honest and original material. Art requires talent. In contrast, through discipline and persistence alone, you can learn how to differentiate functions and ride bicycles. Similarly, you can write a passable technical paper. You just have to realize that your job does not end with research. Writing a passable paper involves extra work.

Composite MR image reconstruction and unaliasing for general trajectories using neural networks

In rapid parallel magnetic resonance imaging, the problem of image reconstruction is challenging. Here, a novel image reconstruction technique for data acquired along any general trajectory in neural network framework, called ``Composite Reconstruction And Unaliasing using Neural Networks'' (CRAUNN), is proposed. CRAUNN is based on the observation that the nature of aliasing remains unchanged whether the undersampled acquisition contains only low frequencies or includes high frequencies too. Here, the transformation needed to reconstruct the alias-free image from the aliased coil images is learnt, using acquisitions consisting of densely sampled low frequencies. Neural networks are made use of as machine learning tools to learn the transformation, in order to obtain the desired alias-free image for actual acquisitions containing sparsely sampled low as well as high frequencies. CRAUNN operates in the image domain and does not require explicit coil sensitivity estimation. It is also independent of the sampling trajectory used, and could be applied to arbitrary trajectories as well. As a pilot trial, the technique is first applied to Cartesian trajectory-sampled data. Experiments performed using radial and spiral trajectories on real and synthetic data, illustrate the performance of the method. The reconstruction errors depend on the acceleration factor as well as the sampling trajectory. It is found that higher acceleration factors can be obtained when radial trajectories are used. Comparisons against existing techniques are presented. CRAUNN has been found to perform on par with the state-of-the-art techniques. Acceleration factors of up to 4, 6 and 4 are achieved in Cartesian, radial and spiral cases, respectively. (C) 2010 Elsevier Inc. All rights reserved.

Noise and outlier removal from jet engine health signals using weighted FIR median hybrid filters

The removal of noise and outliers from measurement signals is a major problem in jet engine health monitoring. Topical measurement signals found in most jet engines include low rotor speed, high rotor speed. fuel flow and exhaust gas temperature. Deviations in these measurements from a baseline 'good' engine are often called measurement deltas and the health signals used for fault detection, isolation, trending and data mining. Linear filters such as the FIR moving average filter and IIR exponential average filter are used in the industry to remove noise and outliers from the jet engine measurement deltas. However, the use of linear filters can lead to loss of critical features in the signal that can contain information about maintenance and repair events that could be used by fault isolation algorithms to determine engine condition or by data mining algorithms to learn valuable patterns in the data, Non-linear filters such as the median and weighted median hybrid filters offer the opportunity to remove noise and gross outliers from signals while preserving features. In this study. a comparison of traditional linear filters popular in the jet engine industry is made with the median filter and the subfilter weighted FIR median hybrid (SWFMH) filter. Results using simulated data with implanted faults shows that the SWFMH filter results in a noise reduction of over 60 per cent compared to only 20 per cent for FIR filters and 30 per cent for IIR filters. Preprocessing jet engine health signals using the SWFMH filter would greatly improve the accuracy of diagnostic systems. (C) 2002 Published by Elsevier Science Ltd.

Multi-Armed Bandit Mechanisms for Multi-Slot Sponsored Searth Auctions

In pay-per click sponsored search auctions which are currently extensively used by search engines, the auction for a keyword involves a certain number of advertisers (say k) competing for available slots (say m) to display their ads. This auction is typically conducted for a number of rounds (say T). There are click probabilities mu_ij associated with agent-slot pairs. The search engine's goal is to maximize social welfare, for example, the sum of values of the advertisers. The search engine does not know the true value of an advertiser for a click to her ad and also does not know the click probabilities mu_ij s. A key problem for the search engine therefore is to learn these during the T rounds of the auction and also to ensure that the auction mechanism is truthful. Mechanisms for addressing such learning and incentives issues have recently been introduced and would be referred to as multi-armed-bandit (MAB) mechanisms. When m = 1,characterizations for truthful MAB mechanisms are available in the literature and it has been shown that the regret for such mechanisms will be O(T^{2/3}). In this paper, we seek to derive a characterization in the realistic but nontrivial general case when m > 1 and obtain several interesting results.

Community-based Natural Resource Management: Issues and Cases from South Asia

Community-based natural resource management (CBNRM) is the joint management of natural resources by a community based on a community strategy, through a participatory mechanism involving all legitimate stakeholders. The approach is community-based in that the communities managing the resources have the legal rights, the local institutions and the economic incentives to take substantial responsibility for sustained use of these resources. This implies that the community plays an active role in the management of natural resources, not because it asserts sole ownership over them, but because it can claim participation in their management and benefits for practical and technical reasons1–4. This approach emerged as the dominant conservation concept in the late 1970s and early 1980s, of the disillusionment with the developmental state. Governments across South and South East Asia, Africa and Latin America have adopted and implemented CBNRM in various ways, viz. through sectoral programmes such as forestry, irrigation or wildlife management, multisectoral programmes such as watershed development and efforts towards political devolution. In India, the principle of decentralization through ‘gram swaraj’ was introduced by Mahatma Gandhi. The 73rd and 74th constitution amendments in 1992 gave impetus to the decentralized planning at panchayat levels through the creation of a statutory three-level local self-government structure5,6. The strength of this book is that it includes chapters by CBNRM advocates based on six seemingly innovative initiatives being implemented by nongovernmental organizations (NGOs) in ecologically vulnerable regions of South Asia: two in the Himalayas (watershed development programme in Lingmutechhu, Bhuthan and Thalisain tehsil, Paudi Grahwal District, Uttarakhand), three in semi-arid parts of western India (watershed development in Hivre Bazar, Maharashtra and Nathugadh village, Gujarat and water-harvesting structures in Gopalapura, Rajasthan) and one in the flood-plains of the Brahmaputra–Jamuna (Char land, Galibanda and Jamalpur districts, Bangladesh). Watersheds in semi-arid regions fall in the low-rainfall region (500–700 mm) and suffer the vagaries of drought 2–3 years in every five-year cycle. In all these locations, the major occupation is agriculture, most of which is rainfed or dry. The other two cases (in Uttarakhand) fall in the Himalayan region (temperate/sub-temperate climate), which has witnessed extensive deforestation in the last century and is now considered as one of the most vulnerable locations in South Asia. Terraced agriculture is being practised in these locations for a long time. The last case (Gono Chetona) falls in the Brahmaputra–Jamuna charlands which are the most ecologically vulnerable regions in the sub-continent with constantly changing landscape. Agriculture and livestock rearing are the main occupations, and there is substantial seasonal emigration for wage labour by the adult males. River erosion and floods force the people to adopt a semi-migratory lifestyle. The book attempts to analyse the potential as well as limitations of NGOdriven CBNRM endeavours across agroclimatic regions of South Asia with emphasis on four intrinsically linked normative concerns, namely sustainability, livelihood enhancement, equity and demographic decentralization in chapters 2–7. Comparative analysis of these case studies done in chapter 8, highlights the issues that require further research while portraying the strengths and limits of NGO-driven CBNRM. In Hivre Bazar, the post-watershed intervention scenario is such that farmers often grow three crops in a year – kharif bajra, rabi jowar and summer vegetable crops. Productivity has increased in the dry lands due to improvement in soil moisture levels. The revival of johads in Gopalpura has led to the proliferation of wheat and increased productivity. In Lingmuteychhu, productivity gains have also arisen, but more due to the introduction of both local and high-yielding, new varieties as opposed to increased water availability. In the case of Gono Chetona, improvements have come due to diversification of agriculture; for example, the promotion of vegetable gardens. CBNRM interventions in most cases have also led to new avenues of employment and income generation. The synthesis shows that CBNRM efforts have made significant contributions to livelihood enhancement and only limited gains in terms of collective action for sustainable and equitable access to benefits and continuing resource use, and in terms of democratic decentralization, contrary to the objectives of the programme. Livelihood benefits include improvements in availability of livelihood support resources (fuelwood, fodder, drinking water), increased productivity (including diversification of cropping pattern) in agriculture and allied activities, and new sources of livelihood. However, NGO-driven CBNRM has not met its goal of providing ‘alternative’ forms of ‘development’ due to impediments of state policy, short-sighted vision of implementers and confrontation with the socio-ecological reality of the region, which almost always are that of fragmented communities (or communities in flux) with unequal dependence and access to land and other natural resources along with great gender imbalances. Appalling, however, is the general absence of recognition of the importance of and the will to explore practical ways to bring about equitable resource transfer or benefit-sharing and the consequent innovations in this respect that are evident in the pioneering community initiatives such as pani panchayat, etc. Pertaining to the gains on the ecological sustainability front, Hivre Bazar and Thalisain initiatives through active participation of villagers have made significant regeneration of the water table within the village, and mechanisms such as ban on number of bore wells, the regulation of cropping pattern, restrictions on felling of trees and free grazing to ensure that in the future, the groundwater is neither over-exploited nor its recharge capability impaired. Nevertheless, the longterm sustainability of the interventions in the case of Ghoga and Gopalpura initiatives as the focus has been mostly on regeneration of resources, and less on regulating the use of regenerated resources. Further, in Lingmuteychhu and Gono Chetona, the interventions are mainly household-based and the focus has been less explicit on ecological components. The studies demonstrate the livelihood benefits to all of the interventions and significant variation in achievements with reference to sustainability, equity and democratic decentralization depending on the level and extent of community participation apart from the vision of implementers, strategy (or nature of intervention shaped by the question of community formation), the centrality of community formation and also the State policy. Case studies show that the influence of State policy is multi-faceted and often contradictory in nature. This necessitates NGOs to engage with the State in a much more purposeful way than in an ‘autonomous space’. Thus the role of NGOs in CBNRM is complementary, wherein they provide innovative experiments that the State can learn. This helps in achieving the goals of CBNRM through democratic decentralization. The book addresses the vital issues related to natural resource management and interests of the community. Key topics discussed throughout the book are still at the centre of the current debate. This compilation consists of well-written chapters based on rigorous synthesis of CBNRM case studies, which will serve as good references for students, researchers and practitioners in the years to come.

Bidding Dynamics of Rational Advertisers in Sponsored Search Auctions on the Web

In this paper, we address a key problem faced by advertisers in sponsored search auctions on the web: how much to bid, given the bids of the other advertisers, so as to maximize individual payoffs? Assuming the generalized second price auction as the auction mechanism, we formulate this problem in the framework of an infinite horizon alternative-move game of advertiser bidding behavior. For a sponsored search auction involving two advertisers, we characterize all the pure strategy and mixed strategy Nash equilibria. We also prove that the bid prices will lead to a Nash equilibrium, if the advertisers follow a myopic best response bidding strategy. Following this, we investigate the bidding behavior of the advertisers if they use Q-learning. We discover empirically an interesting trend that the Q-values converge even if both the advertisers learn simultaneously.

Learning polyhedral classifiers using logistic function

In this paper we propose a new algorithm for learning polyhedral classifiers. In contrast to existing methods for learning polyhedral classifier which solve a constrained optimization problem, our method solves an unconstrained optimization problem. Our method is based on a logistic function based model for the posterior probability function. We propose an alternating optimization algorithm, namely, SPLA1 (Single Polyhedral Learning Algorithm1) which maximizes the loglikelihood of the training data to learn the parameters. We also extend our method to make it independent of any user specified parameter (e.g., number of hyperplanes required to form a polyhedral set) in SPLA2. We show the effectiveness of our approach with experiments on various synthetic and real world datasets and compare our approach with a standard decision tree method (OC1) and a constrained optimization based method for learning polyhedral sets.

Truthful multi-armed bandit mechanisms for multi-slot sponsored search auctions

In pay-per-click sponsored search auctions which are currently extensively used by search engines, the auction for a keyword involves a certain number of advertisers (say k) competing for available slots (say m) to display their advertisements (ads for short). A sponsored search auction for a keyword is typically conducted for a number of rounds (say T). There are click probabilities mu(ij) associated with each agent slot pair (agent i and slot j). The search engine would like to maximize the social welfare of the advertisers, that is, the sum of values of the advertisers for the keyword. However, the search engine does not know the true values advertisers have for a click to their respective advertisements and also does not know the click probabilities. A key problem for the search engine therefore is to learn these click probabilities during the initial rounds of the auction and also to ensure that the auction mechanism is truthful. Mechanisms for addressing such learning and incentives issues have recently been introduced. These mechanisms, due to their connection to the multi-armed bandit problem, are aptly referred to as multi-armed bandit (MAB) mechanisms. When m = 1, exact characterizations for truthful MAB mechanisms are available in the literature. Recent work has focused on the more realistic but non-trivial general case when m > 1 and a few promising results have started appearing. In this article, we consider this general case when m > 1 and prove several interesting results. Our contributions include: (1) When, mu(ij)s are unconstrained, we prove that any truthful mechanism must satisfy strong pointwise monotonicity and show that the regret will be Theta T7) for such mechanisms. (2) When the clicks on the ads follow a certain click precedence property, we show that weak pointwise monotonicity is necessary for MAB mechanisms to be truthful. (3) If the search engine has a certain coarse pre-estimate of mu(ij) values and wishes to update them during the course of the T rounds, we show that weak pointwise monotonicity and type-I separatedness are necessary while weak pointwise monotonicity and type-II separatedness are sufficient conditions for the MAB mechanisms to be truthful. (4) If the click probabilities are separable into agent-specific and slot-specific terms, we provide a characterization of MAB mechanisms that are truthful in expectation.

An adaptation framework for head-pose classification in dynamic multi-view scenarios

Multi-view head-pose estimation in low-resolution, dynamic scenes is difficult due to blurred facial appearance and perspective changes as targets move around freely in the environment. Under these conditions, acquiring sufficient training examples to learn the dynamic relationship between position, face appearance and head-pose can be very expensive. Instead, a transfer learning approach is proposed in this work. Upon learning a weighted-distance function from many examples where the target position is fixed, we adapt these weights to the scenario where target positions are varying. The adaptation framework incorporates reliability of the different face regions for pose estimation under positional variation, by transforming the target appearance to a canonical appearance corresponding to a reference scene location. Experimental results confirm effectiveness of the proposed approach, which outperforms state-of-the-art by 9.5% under relevant conditions. To aid further research on this topic, we also make DPOSE- a dynamic, multi-view head-pose dataset with ground-truth publicly available with this paper.

Robust formulations for clustering-based large-scale classification

Chebyshev-inequality-based convex relaxations of Chance-Constrained Programs (CCPs) are shown to be useful for learning classifiers on massive datasets. In particular, an algorithm that integrates efficient clustering procedures and CCP approaches for computing classifiers on large datasets is proposed. The key idea is to identify high density regions or clusters from individual class conditional densities and then use a CCP formulation to learn a classifier on the clusters. The CCP formulation ensures that most of the data points in a cluster are correctly classified by employing a Chebyshev-inequality-based convex relaxation. This relaxation is heavily dependent on the second-order statistics. However, this formulation and in general such relaxations that depend on the second-order moments are susceptible to moment estimation errors. One of the contributions of the paper is to propose several formulations that are robust to such errors. In particular a generic way of making such formulations robust to moment estimation errors is illustrated using two novel confidence sets. An important contribution is to show that when either of the confidence sets is employed, for the special case of a spherical normal distribution of clusters, the robust variant of the formulation can be posed as a second-order cone program. Empirical results show that the robust formulations achieve accuracies comparable to that with true moments, even when moment estimates are erroneous. Results also illustrate the benefits of employing the proposed methodology for robust classification of large-scale datasets.

Differentially private stochastic gradient descent algorithm for multiparty classification

We consider the problem of developing privacy-preserving machine learning algorithms in a dis-tributed multiparty setting. Here different parties own different parts of a data set, and the goal is to learn a classifier from the entire data set with-out any party revealing any information about the individual data points it owns. Pathak et al [7]recently proposed a solution to this problem in which each party learns a local classifier from its own data, and a third party then aggregates these classifiers in a privacy-preserving manner using a cryptographic scheme. The generaliza-tion performance of their algorithm is sensitive to the number of parties and the relative frac-tions of data owned by the different parties. In this paper, we describe a new differentially pri-vate algorithm for the multiparty setting that uses a stochastic gradient descent based procedure to directly optimize the overall multiparty ob-jective rather than combining classifiers learned from optimizing local objectives. The algorithm achieves a slightly weaker form of differential privacy than that of [7], but provides improved generalization guarantees that do not depend on the number of parties or the relative sizes of the individual data sets. Experimental results corrob-orate our theoretical findings.

K-plane regression

In this paper, we present a novel algorithm for piecewise linear regression which can learn continuous as well as discontinuous piecewise linear functions. The main idea is to repeatedly partition the data and learn a linear model in each partition. The proposed algorithm is similar in spirit to k-means clustering algorithm. We show that our algorithm can also be viewed as a special case of an EM algorithm for maximum likelihood estimation under a reasonable probability model. We empirically demonstrate the effectiveness of our approach by comparing its performance with that of the state of art algorithms on various datasets. (C) 2014 Elsevier Inc. All rights reserved.

Surrogate Regret Bounds for Bipartite Ranking via Strongly Proper Losses

The problem of bipartite ranking, where instances are labeled positive or negative and the goal is to learn a scoring function that minimizes the probability of mis-ranking a pair of positive and negative instances (or equivalently, that maximizes the area under the ROC curve), has been widely studied in recent years. A dominant theoretical and algorithmic framework for the problem has been to reduce bipartite ranking to pairwise classification; in particular, it is well known that the bipartite ranking regret can be formulated as a pairwise classification regret, which in turn can be upper bounded using usual regret bounds for classification problems. Recently, Kotlowski et al. (2011) showed regret bounds for bipartite ranking in terms of the regret associated with balanced versions of the standard (non-pairwise) logistic and exponential losses. In this paper, we show that such (non-pairwise) surrogate regret bounds for bipartite ranking can be obtained in terms of a broad class of proper (composite) losses that we term as strongly proper. Our proof technique is much simpler than that of Kotlowski et al. (2011), and relies on properties of proper (composite) losses as elucidated recently by Reid and Williamson (2010, 2011) and others. Our result yields explicit surrogate bounds (with no hidden balancing terms) in terms of a variety of strongly proper losses, including for example logistic, exponential, squared and squared hinge losses as special cases. An important consequence is that standard algorithms minimizing a (non-pairwise) strongly proper loss, such as logistic regression and boosting algorithms (assuming a universal function class and appropriate regularization), are in fact consistent for bipartite ranking; moreover, our results allow us to quantify the bipartite ranking regret in terms of the corresponding surrogate regret. We also obtain tighter surrogate bounds under certain low-noise conditions via a recent result of Clemencon and Robbiano (2011).

An overview of recent advances in structural bioinformatics of protein-protein interactions and a guide to their principles

Rich data bearing on the structural and evolutionary principles of protein protein interactions are paving the way to a better understanding of the regulation of function in the cell. This is particularly the case when these interactions are considered in the framework of key pathways. Knowledge of the interactions may provide insights into the mechanisms of crucial `driver' mutations in oncogenesis. They also provide the foundation toward the design of protein protein interfaces and inhibitors that can abrogate their formation or enhance them. The main features to learn from known 3-D structures of protein protein complexes and the extensive literature which analyzes them computationally and experimentally include the interaction details which permit undertaking structure-based drug discovery, the evolution of complexes and their interactions, the consequences of alterations such as post-translational modifications, ligand binding, disease causing mutations, host pathogen interactions, oligomerization, aggregation and the roles of disorder, dynamics, allostery and more to the protein and the cell. This review highlights some of the recent advances in these areas, including design, inhibition and prediction of protein protein complexes. The field is broad, and much work has been carried out in these areas, making it challenging to cover it in its entirety. Much of this is due to the fast increase in the number of molecules whose structures have been determined experimentally and the vast increase in computational power. Here we provide a concise overview. (C) 2014 Elsevier Ltd. All rights reserved.