Notes on PCA, Regularization, Sparsity and Support Vector Machines

We derive a new representation for a function as a linear combination of local correlation kernels at optimal sparse locations and discuss its relation to PCA, regularization, sparsity principles and Support Vector Machines. We first review previous results for the approximation of a function from discrete data (Girosi, 1998) in the context of Vapnik"s feature space and dual representation (Vapnik, 1995). We apply them to show 1) that a standard regularization functional with a stabilizer defined in terms of the correlation function induces a regression function in the span of the feature space of classical Principal Components and 2) that there exist a dual representations of the regression function in terms of a regularization network with a kernel equal to a generalized correlation function. We then describe the main observation of the paper: the dual representation in terms of the correlation function can be sparsified using the Support Vector Machines (Vapnik, 1982) technique and this operation is equivalent to sparsify a large dictionary of basis functions adapted to the task, using a variation of Basis Pursuit De-Noising (Chen, Donoho and Saunders, 1995; see also related work by Donahue and Geiger, 1994; Olshausen and Field, 1995; Lewicki and Sejnowski, 1998). In addition to extending the close relations between regularization, Support Vector Machines and sparsity, our work also illuminates and formalizes the LFA concept of Penev and Atick (1996). We discuss the relation between our results, which are about regression, and the different problem of pattern classification.

An Equivalence Between Sparse Approximation and Support Vector Machines

In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT&T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem.

Characterization of ZnO Nanorods Grown on GaN Using Aqueous Solution Method

Uniformly distributed ZnO nanorods with diameter 70-100 nm and 1-2μm long have been successfully grown at low temperatures on GaN by using the inexpensive aqueous solution method. The formation of the ZnO nanorods and the growth parameters are controlled by reactant concentration, temperature and pH. No catalyst is required. The XRD studies show that the ZnO nanorods are single crystals and that they grow along the c axis of the crystal plane. The room temperature photoluminescence measurements have shown ultraviolet peaks at 388nm with high intensity, which are comparable to those found in high quality ZnO films. The mechanism of the nanorod growth in the aqueous solution is proposed. The dependence of the ZnO nanorods on the growth parameters was also investigated. While changing the growth temperature from 60°C to 150°C, the morphology of the ZnO nanorods changed from sharp tip (needle shape) to flat tip (rod shape). These kinds of structure are useful in laser and field emission application.

Growth of ZnO Nanorods on GaN Using Aqueous Solution Method

Uniformly distributed ZnO nanorods with diameter 80-120 nm and 1-2µm long have been successfully grown at low temperatures on GaN by using the inexpensive aqueous solution method. The formation of the ZnO nanorods and the growth parameters are controlled by reactant concentration, temperature and pH. No catalyst is required. The XRD studies show that the ZnO nanorods are single crystals and that they grow along the c axis of the crystal plane. The room temperature photoluminescence measurements have shown ultraviolet peaks at 388nm with high intensity, which are comparable to those found in high quality ZnO films. The mechanism of the nanorod growth in the aqueous solution is proposed. The dependence of the ZnO nanorods on the growth parameters was also investigated. While changing the growth temperature from 60°C to 150°C, the morphology of the ZnO nanorods changed from sharp tip with high aspect ratio to flat tip with smaller aspect ratio. These kinds of structure are useful in laser and field emission application.