On the Noise Model of Support Vector Machine Regression

Support Vector Machines Regression (SVMR) is a regression technique which has been recently introduced by V. Vapnik and his collaborators (Vapnik, 1995; Vapnik, Golowich and Smola, 1996). In SVMR the goodness of fit is measured not by the usual quadratic loss function (the mean square error), but by a different loss function called Vapnik"s $epsilon$- insensitive loss function, which is similar to the "robust" loss functions introduced by Huber (Huber, 1981). The quadratic loss function is well justified under the assumption of Gaussian additive noise. However, the noise model underlying the choice of Vapnik's loss function is less clear. In this paper the use of Vapnik's loss function is shown to be equivalent to a model of additive and Gaussian noise, where the variance and mean of the Gaussian are random variables. The probability distributions for the variance and mean will be stated explicitly. While this work is presented in the framework of SVMR, it can be extended to justify non-quadratic loss functions in any Maximum Likelihood or Maximum A Posteriori approach. It applies not only to Vapnik's loss function, but to a much broader class of loss functions.

A Unified Framework for Regularization Networks and Support Vector Machines

Regularization Networks and Support Vector Machines are techniques for solving certain problems of learning from examples -- in particular the regression problem of approximating a multivariate function from sparse data. We present both formulations in a unified framework, namely in the context of Vapnik's theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics.

Evaluation of sets of oriented and non-oriented receptive fields as local descriptors

Local descriptors are increasingly used for the task of object recognition because of their perceived robustness with respect to occlusions and to global geometrical deformations. We propose a performance criterion for a local descriptor based on the tradeoff between selectivity and invariance. In this paper, we evaluate several local descriptors with respect to selectivity and invariance. The descriptors that we evaluated are Gaussian derivatives up to the third order, gray image patches, and Laplacian-based descriptors with either three scales or one scale filters. We compare selectivity and invariance to several affine changes such as rotation, scale, brightness, and viewpoint. Comparisons have been made keeping the dimensionality of the descriptors roughly constant. The overall results indicate a good performance by the descriptor based on a set of oriented Gaussian filters. It is interesting that oriented receptive fields similar to the Gaussian derivatives as well as receptive fields similar to the Laplacian are found in primate visual cortex.

An Equivalence Between Sparse Approximation and Support Vector Machines

In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT&T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem.

A Note on Support Vector Machines Degeneracy

When training Support Vector Machines (SVMs) over non-separable data sets, one sets the threshold $b$ using any dual cost coefficient that is strictly between the bounds of $0$ and $C$. We show that there exist SVM training problems with dual optimal solutions with all coefficients at bounds, but that all such problems are degenerate in the sense that the "optimal separating hyperplane" is given by ${f w} = {f 0}$, and the resulting (degenerate) SVM will classify all future points identically (to the class that supplies more training data). We also derive necessary and sufficient conditions on the input data for this to occur. Finally, we show that an SVM training problem can always be made degenerate by the addition of a single data point belonging to a certain unboundedspolyhedron, which we characterize in terms of its extreme points and rays.

MA167exam00pdf

Exam questions and solutions in PDF

MA167exam00

Exam questions and solutions in LaTex. Diagrams for the questions are all together in the support.zip file, as .eps files

Vector Calculus

Exercises and solutions about vector calculus

Vector functions and curves

Exercises and solutions about vector functions and curves.

Uffington White Horse vector art files

Vector graphic files for the Uffington White Horse in PDF, AI (Adobe Illustrator EPS) and SVG formats. I manually traced these from a photo using Xara Xtreme.

University Logo in Vector and Raster

This is a selection of University of Southampton Logos in both vector (svg) and raster (png) formats. These are suitable for use on the web or in small documents and posters. You can open the SVG files using inkscape (http://inkscape.org/download/?lang=en) and edit them directly. The University logo should not be modified and attention should be paid to the branding guidelines found here: http://www.edshare.soton.ac.uk/10481 You must always leave a space the width of an capital O in Southampton on all 4 edges of the logo. The negative space makes it appear more prominently on the page.

University Logo in Vector and Raster (Smoother Version)

These are a range of logos created in the same way as Mr Patrick McSweeny http://www.edshare.soton.ac.uk/11157. The logo has been extracted from PDF documents and is smoother and accurate to the original logo design. Many thanks to to McSweeny for publishing the logo, in SVG originally, I struggled to find it anywhere else. Files are in Inkscape SVG, PDF and PNG. From Mr Patrick McSweeney: This is a selection of University of Southampton Logos in both vector (svg) and raster (png) formats. These are suitable for use on the web or in small documents and posters. You can open the SVG files using inkscape (http://inkscape.org/download/?lang=en) and edit them directly. The University logo should not be modified and attention should be paid to the branding guidelines found here: http://www.edshare.soton.ac.uk/10481 You must always leave a space the width of an capital O in Southampton on all 4 edges of the logo. The negative space makes it appear more prominently on the page.

School of Physics and Astronomy Logo - Vector and Raster

Logo for the school of Physics and Astronomy in Inkscape SVG, PDF and high-resolution PNG format

Un metodo para conectar la proyección octogonal de un vector sobre un subespacio de R dotado de un producto escalar

Se trata de un estudio matemático sobre proyecciones octogonales. En el se analizan las diversas posibilidades y variables y se concluye con las posibles soluciones a aplicar al nuevo modelo vectorial.

Los medios de representación gráfica en la modelización en ciencias físicas : el caso del vector fuerza.

Resumen tomado de la publicación