On the V(subscript gamma) Dimension for Regression in Reproducing Kernel Hilbert Spaces

This paper presents a computation of the $V_gamma$ dimension for regression in bounded subspaces of Reproducing Kernel Hilbert Spaces (RKHS) for the Support Vector Machine (SVM) regression $epsilon$-insensitive loss function, and general $L_p$ loss functions. Finiteness of the RV_gamma$ dimension is shown, which also proves uniform convergence in probability for regression machines in RKHS subspaces that use the $L_epsilon$ or general $L_p$ loss functions. This paper presenta a novel proof of this result also for the case that a bias is added to the functions in the RKHS.

Reproducing kernel Hilbert spaces associated with kernels on topological spaces

We analyze reproducing kernel Hilbert spaces of positive definite kernels on a topological space X being either first countable or locally compact. The results include versions of Mercer's theorem and theorems on the embedding of these spaces into spaces of continuous and square integrable functions.

Weighted Fourier–Laplace transforms in reproducing kernel Hilbert spaces on the sphere

We study the action of a weighted Fourier–Laplace transform on the functions in the reproducing kernel Hilbert space (RKHS) associated with a positive definite kernel on the sphere. After defining a notion of smoothness implied by the transform, we show that smoothness of the kernel implies the same smoothness for the generating elements (spherical harmonics) in the Mercer expansion of the kernel. We prove a reproducing property for the weighted Fourier–Laplace transform of the functions in the RKHS and embed the RKHS into spaces of smooth functions. Some relevant properties of the embedding are considered, including compactness and boundedness. The approach taken in the paper includes two important notions of differentiability characterized by weighted Fourier–Laplace transforms: fractional derivatives and Laplace–Beltrami derivatives.

Tridiagonal Reproducing Kernels and Subnormality

We consider analytic reproducing kernel Hilbert spaces H with orthonormal bases of the form {(a(n) + b(n)z)z(n) : n >= 0}. If b(n) = 0 for all n, then H is a diagonal space and multiplication by z, M-z, is a weighted shift. Our focus is on providing extensive classes of examples for which M-z is a bounded subnormal operator on a tridiagonal space H where b(n) not equal 0. The Aronszajn sum of H and (1 - z)H where H is either the Hardy space or the Bergman space on the disk are two such examples.

Analytic Kramer kernels, Lagrange-type interpolation series and de Branges spaces

The classical Kramer sampling theorem provides a method for obtaining orthogonal sampling formulas. In particular, when the involved kernel is analytic in the sampling parameter it can be stated in an abstract setting of reproducing kernel Hilbert spaces of entire functions which includes as a particular case the classical Shannon sampling theory. This abstract setting allows us to obtain a sort of converse result and to characterize when the sampling formula associated with an analytic Kramer kernel can be expressed as a Lagrange-type interpolation series. On the other hand, the de Branges spaces of entire functions satisfy orthogonal sampling formulas which can be written as Lagrange-type interpolation series. In this work some links between all these ideas are established.

Representations of Inverse Functions by the Integral Transform with the Sign Kernel

Mathematics Subject Classification: Primary 30C40

Spectral Analysis of Symmetric and Anti-Symmetric Pairwise Kernels

This work investigates theoretical properties of symmetric and anti-symmetric kernels. First chapters give an overview of the theory of kernels used in supervised machine learning. Central focus is on the regularized least squares algorithm, which is motivated as a problem of function reconstruction through an abstract inverse problem. Brief review of reproducing kernel Hilbert spaces shows how kernels define an implicit hypothesis space with multiple equivalent characterizations and how this space may be modified by incorporating prior knowledge. Mathematical results of the abstract inverse problem, in particular spectral properties, pseudoinverse and regularization are recollected and then specialized to kernels. Symmetric and anti-symmetric kernels are applied in relation learning problems which incorporate prior knowledge that the relation is symmetric or anti-symmetric, respectively. Theoretical properties of these kernels are proved in a draft this thesis is based on and comprehensively referenced here. These proofs show that these kernels can be guaranteed to learn only symmetric or anti-symmetric relations, and they can learn any relations relative to the original kernel modified to learn only symmetric or anti-symmetric parts. Further results prove spectral properties of these kernels, central result being a simple inequality for the the trace of the estimator, also called the effective dimension. This quantity is used in learning bounds to guarantee smaller variance.

An Equivalence Between Sparse Approximation and Support Vector Machines

In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT&T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem.

Application of complex extreme learning machine to multiclass classification problems with high dimensionality: a THz spectra classification problem

We extend extreme learning machine (ELM) classifiers to complex Reproducing Kernel Hilbert Spaces (RKHS) where the input/output variables as well as the optimization variables are complex-valued. A new family of classifiers, called complex-valued ELM (CELM) suitable for complex-valued multiple-input–multiple-output processing is introduced. In the proposed method, the associated Lagrangian is computed using induced RKHS kernels, adopting a Wirtinger calculus approach formulated as a constrained optimization problem similarly to the conventional ELM classifier formulation. When training the CELM, the Karush–Khun–Tuker (KKT) theorem is used to solve the dual optimization problem that consists of satisfying simultaneously smallest training error as well as smallest norm of output weights criteria. The proposed formulation also addresses aspects of quaternary classification within a Clifford algebra context. For 2D complex-valued inputs, user-defined complex-coupled hyper-planes divide the classifier input space into four partitions. For 3D complex-valued inputs, the formulation generates three pairs of complex-coupled hyper-planes through orthogonal projections. The six hyper-planes then divide the 3D space into eight partitions. It is shown that the CELM problem formulation is equivalent to solving six real-valued ELM tasks, which are induced by projecting the chosen complex kernel across the different user-defined coordinate planes. A classification example of powdered samples on the basis of their terahertz spectral signatures is used to demonstrate the advantages of the CELM classifiers compared to their SVM counterparts. The proposed classifiers retain the advantages of their ELM counterparts, in that they can perform multiclass classification with lower computational complexity than SVM classifiers. Furthermore, because of their ability to perform classification tasks fast, the proposed formulations are of interest to real-time applications.

Aspectos matemáticos do problema de aprendizagem em inteligência artificial

O objetivo deste trabalho é apresentar a base teórica para o problema de aprendizagem através de exemplos conforme as ref. [14], [15] e [16]. Aprender através de exemplos pode ser examinado como o problema de regressão da aproximação de uma função multivaluada sobre um conjunto de dados esparsos. Tal problema não é bem posto e a maneira clássica de resolvê-lo é através da teoria de regularização. A teoria de regularização clássica, como será considerada aqui, formula este problema de regressão como o problema variacional de achar a função f que minimiza o funcional Q[f] = 1 n n Xi=1 (yi ¡ f(xi))2 + ¸kfk2 K; onde kfk2 K é a norma em um espa»co de Hilbert especial que chamaremos de Núcleo Reprodutivo (Reproducing Kernel Hilbert Spaces), ou somente RKHS, IH definido pela função positiva K, o número de pontos do exemplo n e o parâmetro de regularização ¸. Sob condições gerais a solução da equação é dada por f(x) = n Xi=1 ciK(x; xi): A teoria apresentada neste trabalho é na verdade a fundamentação para uma teoria mais geral que justfica os funcionais regularizados para a aprendizagem através de um conjunto infinito de dados e pode ser usada para estender consideravelmente a estrutura clássica a regularização, combinando efetivamente uma perspectiva de análise funcional com modernos avanços em Teoria de Probabilidade e Estatística.

A comparison of statistical methods for genomic selection in a mice population

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Accurate ab initio potential energy surfaces for the (3)A '' and (3)A ' electronic states of the O(P-3) plus HBr system

In this work, we report the construction of potential energy surfaces for the (3)A '' and (3)A' states of the system O(P-3) + HBr. These surfaces are based on extensive ab initio calculations employing the MRCI+Q/CBS+SO level of theory. The complete basis set energies were estimated from extrapolation of MRCI+Q/aug-cc-VnZ(-PP) (n = Q, 5) results and corrections due to spin-orbit effects obtained at the CASSCF/aug-cc-pVTZ(-PP) level of theory. These energies, calculated over a region of the configuration space relevant to the study of the reaction O(P-3) + HBr -> OH + Br, were used to generate functions based on the many-body expansion. The three-body potentials were interpolated using the reproducing kernel Hilbert space method. The resulting surface for the (3)A '' electronic state contains van der Waals minima on the entrance and exit channels and a transition state 6.55 kcal/mol higher than the reactants. This barrier height was then scaled to reproduce the value of 5.01 kcal/mol, which was estimated from coupled cluster benchmark calculations performed to include high-order and core-valence correlation, as well as scalar relativistic effects. The (3)A' surface was also scaled, based on the fact that in the collinear saddle point geometry these two electronic states are degenerate. The vibrationally adiabatic barrier heights are 3.44 kcal/mol for the (3)A '' and 4.16 kcal/mol for the (3)A' state. (C) 2012 American Institute of Physics. [http://dx.doi.org/10.1063/1.4705428]

ANOVA kernels and RKHS of zero mean functions for model-based sensitivity analysis

Given a reproducing kernel Hilbert space (H,〈.,.〉)(H,〈.,.〉) of real-valued functions and a suitable measure μμ over the source space D⊂RD⊂R, we decompose HH as the sum of a subspace of centered functions for μμ and its orthogonal in HH. This decomposition leads to a special case of ANOVA kernels, for which the functional ANOVA representation of the best predictor can be elegantly derived, either in an interpolation or regularization framework. The proposed kernels appear to be particularly convenient for analyzing the effect of each (group of) variable(s) and computing sensitivity indices without recursivity.

RKHS-Methods at Series Summation for Software Implementation

Reproducing Kernel Hilbert Space (RKHS) and Reproducing Transformation Methods for Series Summation that allow analytically obtaining alternative representations for series in the finite form are developed.

Solutions of Tikhonov functional equations and applications to multiplication operators on Szegö spaces

We consider a natural representation of solutions for Tikhonov functional equations. This will be done by applying the theory of reproducing kernels to the approximate solutions of general bounded linear operator equations (when defined from reproducing kernel Hilbert spaces into general Hilbert spaces), by using the Hilbert-Schmidt property and tensor product of Hilbert spaces. As a concrete case, we shall consider generalized fractional functions formed by the quotient of Bergman functions by Szegö functions considered from the multiplication operators on the Szegö spaces.