A Unified Framework for Regularization Networks and Support Vector Machines

Regularization Networks and Support Vector Machines are techniques for solving certain problems of learning from examples -- in particular the regression problem of approximating a multivariate function from sparse data. We present both formulations in a unified framework, namely in the context of Vapnik's theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics.

An Equivalence Between Sparse Approximation and Support Vector Machines

In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT&T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem.

Support Vector Machines: Training and Applications

The Support Vector Machine (SVM) is a new and very promising classification technique developed by Vapnik and his group at AT&T Bell Labs. This new learning algorithm can be seen as an alternative training technique for Polynomial, Radial Basis Function and Multi-Layer Perceptron classifiers. An interesting property of this approach is that it is an approximate implementation of the Structural Risk Minimization (SRM) induction principle. The derivation of Support Vector Machines, its relationship with SRM, and its geometrical insight, are discussed in this paper. Training a SVM is equivalent to solve a quadratic programming problem with linear and box constraints in a number of variables equal to the number of data points. When the number of data points exceeds few thousands the problem is very challenging, because the quadratic form is completely dense, so the memory needed to store the problem grows with the square of the number of data points. Therefore, training problems arising in some real applications with large data sets are impossible to load into memory, and cannot be solved using standard non-linear constrained optimization algorithms. We present a decomposition algorithm that can be used to train SVM's over large data sets. The main idea behind the decomposition is the iterative solution of sub-problems and the evaluation of, and also establish the stopping criteria for the algorithm. We present previous approaches, as well as results and important details of our implementation of the algorithm using a second-order variant of the Reduced Gradient Method as the solver of the sub-problems. As an application of SVM's, we present preliminary results we obtained applying SVM to the problem of detecting frontal human faces in real images.

A Note on Support Vector Machines Degeneracy

When training Support Vector Machines (SVMs) over non-separable data sets, one sets the threshold $b$ using any dual cost coefficient that is strictly between the bounds of $0$ and $C$. We show that there exist SVM training problems with dual optimal solutions with all coefficients at bounds, but that all such problems are degenerate in the sense that the "optimal separating hyperplane" is given by ${f w} = {f 0}$, and the resulting (degenerate) SVM will classify all future points identically (to the class that supplies more training data). We also derive necessary and sufficient conditions on the input data for this to occur. Finally, we show that an SVM training problem can always be made degenerate by the addition of a single data point belonging to a certain unboundedspolyhedron, which we characterize in terms of its extreme points and rays.

The Age of Social Machines

Many of the most successful and important systems that impact our lives combine humans, data, and algorithms at Web Scale. These social machines are amalgamations of human and machine intelligence. This seminar will provide an update on SOCIAM, a five year EPSRC Programme Grant that seeks to gain a better understanding of social machines; how they are observed and constituted, how they can be designed and their fate determined. We will review how social machines can be of value to society, organisations and individuals. We will consider the challenges they present to our various disciplines.

Hierarchical Prediction Machines and Big Data Analytics

An emerging consensus in cognitive science views the biological brain as a hierarchically-organized predictive processing system. This is a system in which higher-order regions are continuously attempting to predict the activity of lower-order regions at a variety of (increasingly abstract) spatial and temporal scales. The brain is thus revealed as a hierarchical prediction machine that is constantly engaged in the effort to predict the flow of information originating from the sensory surfaces. Such a view seems to afford a great deal of explanatory leverage when it comes to a broad swathe of seemingly disparate psychological phenomena (e.g., learning, memory, perception, action, emotion, planning, reason, imagination, and conscious experience). In the most positive case, the predictive processing story seems to provide our first glimpse at what a unified (computationally-tractable and neurobiological plausible) account of human psychology might look like. This obviously marks out one reason why such models should be the focus of current empirical and theoretical attention. Another reason, however, is rooted in the potential of such models to advance the current state-of-the-art in machine intelligence and machine learning. Interestingly, the vision of the brain as a hierarchical prediction machine is one that establishes contact with work that goes under the heading of 'deep learning'. Deep learning systems thus often attempt to make use of predictive processing schemes and (increasingly abstract) generative models as a means of supporting the analysis of large data sets. But are such computational systems sufficient (by themselves) to provide a route to general human-level analytic capabilities? I will argue that they are not and that closer attention to a broader range of forces and factors (many of which are not confined to the neural realm) may be required to understand what it is that gives human cognition its distinctive (and largely unique) flavour. The vision that emerges is one of 'homomimetic deep learning systems', systems that situate a hierarchically-organized predictive processing core within a larger nexus of developmental, behavioural, symbolic, technological and social influences. Relative to that vision, I suggest that we should see the Web as a form of 'cognitive ecology', one that is as much involved with the transformation of machine intelligence as it is with the progressive reshaping of our own cognitive capabilities.

Machines at work.

Se ofrece a los lectores más jóvenes una introducción al mundo de las ciencias, para que de forma práctica y amena aprendan y descubran por sí mismos. En este volumen, se explican a traves de hechos y actividades los distintos tipos de fuerzas en los que se basa la mecánica, como parte de la física, para la cosntrucción de máquinas para el trabajo.

Minds and machines : Britain, 1750-1900.

Explora el contexto social, político y espiritual de las personas en Gran Bretaña entre 1750-1900, para ayudar a los alumnos a comprender las mentes de las personas a lo largo de años de grandes cambios. Apoya el desarrollo de habilidades de pensamiento con un lenguaje ameno y cuidadosamente planificado. Cada sección se ha estructurado como una investigación, en torno a una cuestión fundamental de importancia histórica. Cada capítulo está diseñado para poder ampliarse por escrito. Los estudiantes podrán ordenar y clasificar la información y aprender a conectar las cuestiones clave.

Minds and machines. Teacher's book : Britain, 1750-1900.

Explora el contexto social, político y espiritual de las personas en Gran Bretaña entre 1750-1900. Para ayudar a los alumnos a comprender las mentes de las personas a lo largo de años de grandes cambios. Tiene por objeto evitar la cobertura superficial, construyendo el conocimiento y la comprensión conceptual apoyándose en tres principios de enseñanza: motivación, rigor y adecuado ritmo.Los alumnos deben ser estimulados por el material, motivación, que debe ser directo y centrado en las metas, rigor, y deben tener la ayuda que garantice su ritmo adecuado de prendizaje.

Purpose built teaching equipment for a laboratory based course in three phase machines and drives

The hazards associated with high voltage three phase inverters and the rotating shafts of large electrical machines have resulted in most of the engineering courses covering these topics to be predominantly theoretical. This paper describes a set of purpose built, low voltage and low cost teaching equipment which allows the "hands on" instruction of three phase inverters and rotating machines. By using low voltages, the student can experiment freely with the motors and inverter and can access all of the current and voltage waveforms, which until now could only be studied in text books or observed as part of laboratory demonstrations. Both the motor and the inverter designs are optimized for teaching purposes cost around $25 and can be made with minimal effort.

Three-phase machines and drives - equipment for a laboratory-based course

The hazards associated with high-voltage three-phase inverters and high-powered large electrical machines have resulted in most of the engineering courses covering three-phase machines and drives theoretically. This paper describes a set of purpose-built, low-voltage, and low-cost teaching equipment that allows the hands-on instruction of three-phase inverters and rotating machines. The motivation for moving towards a system running at low voltages is that the students can safely experiment freely with the motors and inverter. The students can also access all of the current and voltage waveforms, which until now could only be studied in textbooks or observed as part of laboratory demonstrations. Both the motor and the inverter designs are for teaching purposes and require minimal effort and cost

Study of the variation of the input impedance of induction machines with frequency

The results of a study of the variation of three-phase induction machines' input impedance with frequency are proposed. A range of motors were analysed, both two-pole and four-pole, and the magnitude and phase of the input impedance were obtained over a wide frequency range of 20 Hz-1 MHz. For test results that would be useful in the prediction of the performance of induction machines during typical use, a test procedure was developed to represent closely typical three-phase stator coil connections when the induction machine is driven by a three-phase inverter. In addition, tests were performed with the motor's cases both grounded and not grounded. The results of the study show that all induction machines of the type considered exhibit a multiresonant impedance profile, where the input impedance reaches at least one maximum as the input frequency is increased. Furthermore, the test results show that the grounding of the motor's case has a significant effect on the impedance profile. Methods to exploit the input impedance profile of an induction machine to optimise machine and inverter systems are also discussed.

The postfusion structure of baculovirus gp64 supports a unified view of viral fusion machines

Viral fusion proteins mediate the merger of host and viral membranes during cell entry for all enveloped viruses. Baculovirus glycoprotein gp64 (gp64) is unusual in promoting entry into both insect and mammalian cells and is distinct from established class I and class II fusion proteins. We report the crystal structure of its postfusion form, which explains a number of gp64's biological properties including its cellular promiscuity, identifies the fusion peptides and shows it to be the third representative of a new class (III) of fusion proteins with unexpected structural homology with vesicular stomatitis virus G and herpes simplex virus type 1 gB proteins. We show that domains of class III proteins have counterparts in both class I and II proteins, suggesting that all these viral fusion machines are structurally more related than previously thought.