On the Noise Model of Support Vector Machine Regression

Support Vector Machines Regression (SVMR) is a regression technique which has been recently introduced by V. Vapnik and his collaborators (Vapnik, 1995; Vapnik, Golowich and Smola, 1996). In SVMR the goodness of fit is measured not by the usual quadratic loss function (the mean square error), but by a different loss function called Vapnik"s $epsilon$- insensitive loss function, which is similar to the "robust" loss functions introduced by Huber (Huber, 1981). The quadratic loss function is well justified under the assumption of Gaussian additive noise. However, the noise model underlying the choice of Vapnik's loss function is less clear. In this paper the use of Vapnik's loss function is shown to be equivalent to a model of additive and Gaussian noise, where the variance and mean of the Gaussian are random variables. The probability distributions for the variance and mean will be stated explicitly. While this work is presented in the framework of SVMR, it can be extended to justify non-quadratic loss functions in any Maximum Likelihood or Maximum A Posteriori approach. It applies not only to Vapnik's loss function, but to a much broader class of loss functions.

A Unified Framework for Regularization Networks and Support Vector Machines

Regularization Networks and Support Vector Machines are techniques for solving certain problems of learning from examples -- in particular the regression problem of approximating a multivariate function from sparse data. We present both formulations in a unified framework, namely in the context of Vapnik's theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics.

An Equivalence Between Sparse Approximation and Support Vector Machines

In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT&T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem.

Support Vector Machines: Training and Applications

The Support Vector Machine (SVM) is a new and very promising classification technique developed by Vapnik and his group at AT&T Bell Labs. This new learning algorithm can be seen as an alternative training technique for Polynomial, Radial Basis Function and Multi-Layer Perceptron classifiers. An interesting property of this approach is that it is an approximate implementation of the Structural Risk Minimization (SRM) induction principle. The derivation of Support Vector Machines, its relationship with SRM, and its geometrical insight, are discussed in this paper. Training a SVM is equivalent to solve a quadratic programming problem with linear and box constraints in a number of variables equal to the number of data points. When the number of data points exceeds few thousands the problem is very challenging, because the quadratic form is completely dense, so the memory needed to store the problem grows with the square of the number of data points. Therefore, training problems arising in some real applications with large data sets are impossible to load into memory, and cannot be solved using standard non-linear constrained optimization algorithms. We present a decomposition algorithm that can be used to train SVM's over large data sets. The main idea behind the decomposition is the iterative solution of sub-problems and the evaluation of, and also establish the stopping criteria for the algorithm. We present previous approaches, as well as results and important details of our implementation of the algorithm using a second-order variant of the Reduced Gradient Method as the solver of the sub-problems. As an application of SVM's, we present preliminary results we obtained applying SVM to the problem of detecting frontal human faces in real images.

A Note on Support Vector Machines Degeneracy

When training Support Vector Machines (SVMs) over non-separable data sets, one sets the threshold $b$ using any dual cost coefficient that is strictly between the bounds of $0$ and $C$. We show that there exist SVM training problems with dual optimal solutions with all coefficients at bounds, but that all such problems are degenerate in the sense that the "optimal separating hyperplane" is given by ${f w} = {f 0}$, and the resulting (degenerate) SVM will classify all future points identically (to the class that supplies more training data). We also derive necessary and sufficient conditions on the input data for this to occur. Finally, we show that an SVM training problem can always be made degenerate by the addition of a single data point belonging to a certain unboundedspolyhedron, which we characterize in terms of its extreme points and rays.

Student self-assessment a feedback seeking strategy through competence evaluation

There is a body of literature that suggests that student self-assessment is a main goal in higher education (Boud et al., 1995; Tan, 2008); moreover new forms of work organization require a high level of skills and competences. The efforts to deal with competence gaps could be developed at many levels, such as employers, educational institutions, individuals and public agents. Employers could put into practice competence development programs to moderate these gaps. Educational institutions can restructure the curriculum to support students in attaining the competences that are essential in the labour market. Individuals themselves may deploy their resources (time and money) in general or specific competence training. Further, government agencies could fund competence promotion programs. Such challenges for education drive change in learning curricula and method, to properly include the competences required for developing global workers who can move beyond basic competence, to enhanced flexibility and adaptability. In performance assessment methods, there is a shift from the traditional exam-based assessments to more innovative task assessment, which considers performance in multiple different tasks carry out by students. ICTs make it technologically feasible to carry out a complete and complex selfassessment of competences, which provides immediate results to students or other recipients. In the case of students, the evaluation of competences is relevant as developing competences is part - if not all - of the objectives of education. Therefore, it is an important element of the quality of educational organizations (e.g., universities), and of their organizational success. Further, educational organizations may put special emphasis on some differentiating competences, which can be a means of positioning and differentiation from competitors. Competence assessment is an instrument to make students conscious of their strengths and weaknesses, leading to higher motivation to develop their own learning career

Vector Fields

Exercises and solutions about vector fields. Diagrams for the questions are all together in the support.zip file, as .eps files

Vector Calculus

Exercises and solutions about vector calculus

Vector functions and curves

Exercises and solutions about vector functions and curves.

Uffington White Horse vector art files

Vector graphic files for the Uffington White Horse in PDF, AI (Adobe Illustrator EPS) and SVG formats. I manually traced these from a photo using Xara Xtreme.

Opportunities for Achieving Competence in Personal Attributes for Personal & Professional Developments

This document lists descriptions of generic skills / personal attributes that are useful to consider in personal and professional development. It also describes some general competencies and offers some thoughts on how to create opportunities to achieve competence with a bias towards computer science & IT students. It is based on advice given by Career Destinations at the University of Southampton and other universities

ALPS Workshop3 p10 Conscious competence model

Animation

University Logo in Vector and Raster

This is a selection of University of Southampton Logos in both vector (svg) and raster (png) formats. These are suitable for use on the web or in small documents and posters. You can open the SVG files using inkscape (http://inkscape.org/download/?lang=en) and edit them directly. The University logo should not be modified and attention should be paid to the branding guidelines found here: http://www.edshare.soton.ac.uk/10481 You must always leave a space the width of an capital O in Southampton on all 4 edges of the logo. The negative space makes it appear more prominently on the page.

University Logo in Vector and Raster (Smoother Version)

These are a range of logos created in the same way as Mr Patrick McSweeny http://www.edshare.soton.ac.uk/11157. The logo has been extracted from PDF documents and is smoother and accurate to the original logo design. Many thanks to to McSweeny for publishing the logo, in SVG originally, I struggled to find it anywhere else. Files are in Inkscape SVG, PDF and PNG. From Mr Patrick McSweeney: This is a selection of University of Southampton Logos in both vector (svg) and raster (png) formats. These are suitable for use on the web or in small documents and posters. You can open the SVG files using inkscape (http://inkscape.org/download/?lang=en) and edit them directly. The University logo should not be modified and attention should be paid to the branding guidelines found here: http://www.edshare.soton.ac.uk/10481 You must always leave a space the width of an capital O in Southampton on all 4 edges of the logo. The negative space makes it appear more prominently on the page.