Notes on PCA, Regularization, Sparsity and Support Vector Machines

We derive a new representation for a function as a linear combination of local correlation kernels at optimal sparse locations and discuss its relation to PCA, regularization, sparsity principles and Support Vector Machines. We first review previous results for the approximation of a function from discrete data (Girosi, 1998) in the context of Vapnik"s feature space and dual representation (Vapnik, 1995). We apply them to show 1) that a standard regularization functional with a stabilizer defined in terms of the correlation function induces a regression function in the span of the feature space of classical Principal Components and 2) that there exist a dual representations of the regression function in terms of a regularization network with a kernel equal to a generalized correlation function. We then describe the main observation of the paper: the dual representation in terms of the correlation function can be sparsified using the Support Vector Machines (Vapnik, 1982) technique and this operation is equivalent to sparsify a large dictionary of basis functions adapted to the task, using a variation of Basis Pursuit De-Noising (Chen, Donoho and Saunders, 1995; see also related work by Donahue and Geiger, 1994; Olshausen and Field, 1995; Lewicki and Sejnowski, 1998). In addition to extending the close relations between regularization, Support Vector Machines and sparsity, our work also illuminates and formalizes the LFA concept of Penev and Atick (1996). We discuss the relation between our results, which are about regression, and the different problem of pattern classification.

Static Pricing for a Network Service Provider

This article studies the static pricing problem of a network service provider who has a fixed capacity and faces different types of customers (classes). Each type of customers can have its own capacity constraint but it is assumed that all classes have the same resource requirement. The provider must decide a static price for each class. The customer types are characterized by their arrival process, with a price-dependant arrival rate, and the random time they remain in the system. Many real-life situations could fit in this framework, for example an Internet provider or a call center, but originally this problem was thought for a company that sells phone-cards and needs to set the price-per-minute for each destination. Our goal is to characterize the optimal static prices in order to maximize the provider's revenue. We note that the model here presented, with some slight modifications and additional assumptions can be used in those cases when the objective is to maximize social welfare.