Explorations of the Practical Issues of Learning Prediction-Control Tasks Using Temporal Difference Learning Methods

There has been recent interest in using temporal difference learning methods to attack problems of prediction and control. While these algorithms have been brought to bear on many problems, they remain poorly understood. It is the purpose of this thesis to further explore these algorithms, presenting a framework for viewing them and raising a number of practical issues and exploring those issues in the context of several case studies. This includes applying the TD(lambda) algorithm to: 1) learning to play tic-tac-toe from the outcome of self-play and of play against a perfectly-playing opponent and 2) learning simple one-dimensional segmentation tasks.

Static Pricing for a Network Service Provider

This article studies the static pricing problem of a network service provider who has a fixed capacity and faces different types of customers (classes). Each type of customers can have its own capacity constraint but it is assumed that all classes have the same resource requirement. The provider must decide a static price for each class. The customer types are characterized by their arrival process, with a price-dependant arrival rate, and the random time they remain in the system. Many real-life situations could fit in this framework, for example an Internet provider or a call center, but originally this problem was thought for a company that sells phone-cards and needs to set the price-per-minute for each destination. Our goal is to characterize the optimal static prices in order to maximize the provider's revenue. We note that the model here presented, with some slight modifications and additional assumptions can be used in those cases when the objective is to maximize social welfare.