Probabilistic principal component analysis

Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively, and discuss the advantages conveyed by the definition of a probability density function for PCA.

Probabilistic principal component analysis

Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based upon a probability model. In this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively, and discuss the advantages conveyed by the definition of a probability density function for PCA.

Developments of the generative topographic mapping

The generative topographic mapping (GTM) model was introduced by Bishop et al. (1998, Neural Comput. 10(1), 215-234) as a probabilistic re- formulation of the self-organizing map (SOM). It offers a number of advantages compared with the standard SOM, and has already been used in a variety of applications. In this paper we report on several extensions of the GTM, including an incremental version of the EM algorithm for estimating the model parameters, the use of local subspace models, extensions to mixed discrete and continuous data, semi-linear models which permit the use of high-dimensional manifolds whilst avoiding computational intractability, Bayesian inference applied to hyper-parameters, and an alternative framework for the GTM based on Gaussian processes. All of these developments directly exploit the probabilistic structure of the GTM, thereby allowing the underlying modelling assumptions to be made explicit. They also highlight the advantages of adopting a consistent probabilistic framework for the formulation of pattern recognition algorithms.

A basis function approach to Bayesian inference in diffusion processes

In this paper, we present a framework for Bayesian inference in continuous-time diffusion processes. The new method is directly related to the recently proposed variational Gaussian Process approximation (VGPA) approach to Bayesian smoothing of partially observed diffusions. By adopting a basis function expansion (BF-VGPA), both the time-dependent control parameters of the approximate GP process and its moment equations are projected onto a lower-dimensional subspace. This allows us both to reduce the computational complexity and to eliminate the time discretisation used in the previous algorithm. The new algorithm is tested on an Ornstein-Uhlenbeck process. Our preliminary results show that BF-VGPA algorithm provides a reasonably accurate state estimation using a small number of basis functions.

Measurements design and phenomena discrimination

The construction of measurements suitable for discriminating signal components produced by phenomena of different types is considered. The required measurements should be capable of cancelling out those signal components which are to be ignored when focusing on a phenomenon of interest. Under the hypothesis that the subspaces hosting the signal components produced by each phenomenon are complementary, their discrimination is accomplished by measurements giving rise to the appropriate oblique projector operator. The subspace onto which the operator should project is selected by nonlinear techniques in line with adaptive pursuit strategies.

Nonlinear non-extensive approach for identification of structured information

The problem of separating structured information representing phenomena of differing natures is considered. A structure is assumed to be independent of the others if can be represented in a complementary subspace. When the concomitant subspaces are well separated the problem is readily solvable by a linear technique. Otherwise, the linear approach fails to correctly discriminate the required information. Hence, a non-extensive approach is proposed. The resulting nonlinear technique is shown to be suitable for dealing with cases that cannot be tackled by the linear one.

Binary sparse nonnegative matrix factorization

This paper presents a fast part-based subspace selection algorithm, termed the binary sparse nonnegative matrix factorization (B-SNMF). Both the training process and the testing process of B-SNMF are much faster than those of binary principal component analysis (B-PCA). Besides, B-SNMF is more robust to occlusions in images. Experimental results on face images demonstrate the effectiveness and the efficiency of the proposed B-SNMF.

Oblique matching pursuit

A method for selecting a suitable subspace for discriminating signal components through an oblique projection is proposed. The selection criterion is based on the consistency principle introduced by Unser and Aldroubi and extended by Elder. An effective implementation of this principle for the purpose of subspace selection is achieved by updating of the dual vectors yielding the corresponding oblique projector. © 2007 IEEE.

Incremental tensor biased discriminant analysis:a new color-based visual tracking method

Most existing color-based tracking algorithms utilize the statistical color information of the object as the tracking clues, without maintaining the spatial structure within a single chromatic image. Recently, the researches on the multilinear algebra provide the possibility to hold the spatial structural relationship in a representation of the image ensembles. In this paper, a third-order color tensor is constructed to represent the object to be tracked. Considering the influence of the environment changing on the tracking, the biased discriminant analysis (BDA) is extended to the tensor biased discriminant analysis (TBDA) for distinguishing the object from the background. At the same time, an incremental scheme for the TBDA is developed for the tensor biased discriminant subspace online learning, which can be used to adapt to the appearance variant of both the object and background. The experimental results show that the proposed method can track objects precisely undergoing large pose, scale and lighting changes, as well as partial occlusion. © 2009 Elsevier B.V.

Outlier-resisting graph embedding

Graph embedding is a general framework for subspace learning. However, because of the well-known outlier-sensitiveness disadvantage of the L2-norm, conventional graph embedding is not robust to outliers which occur in many practical applications. In this paper, an improved graph embedding algorithm (termed LPP-L1) is proposed by replacing L2-norm with L1-norm. In addition to its robustness property, LPP-L1 avoids small sample size problem. Experimental results on both synthetic and real-world data demonstrate these advantages. © 2009 Elsevier B.V. All rights reserved.

Asplund Functions and Projectional Resolutions of the Identity

*Supported by the Grants AV ˇCR 101-97-02, 101-90-03, GA ˇCR 201-98-1449, and by the Grant of the Faculty of Civil Engineering of the Czech Technical University No. 2003.

Caractérisation Des Espaces 1-Matriciellement Normés

Let X be a closed subspace of B(H) for some Hilbert space H. In [9], Pisier introduced Sp [X] (1 ≤ p ≤ +∞) by setting Sp [X] = (S∞ [X] , S1 [X])θ , (where θ =1/p , S∞ [X] = S∞ ⊗min X and S1 [X] = S1 ⊗∧ X) and showed that there are p−matricially normed spaces. In this paper we prove that conversely, if X is a p−matricially normed space with p = 1, then there is an operator structure on X, such that M1,n (X) = S1 [X] where Sn,1 [X] is the finite dimentional version of S1 [X]. For p = 1, we have no answer.

Isomorphism Problems for the Baire Function Spaces of Topological Spaces

Let a compact Hausdorff space X contain a non-empty perfect subset. If α < β and β is a countable ordinal, then the Banach space Bα (X) of all bounded real-valued functions of Baire class α on X is a proper subspace of the Banach space Bβ (X). In this paper it is shown that: 1. Bα (X) has a representation as C(bα X), where bα X is a compactification of the space P X – the underlying set of X in the Baire topology generated by the Gδ -sets in X. 2. If 1 ≤ α < β ≤ Ω, where Ω is the first uncountable ordinal number, then Bα (X) is uncomplemented as a closed subspace of Bβ (X). These assertions for X = [0, 1] were proved by W. G. Bade [4] and in the case when X contains an uncountable compact metrizable space – by F.K.Dashiell [9]. Our argumentation is one non-metrizable modification of both Bade’s and Dashiell’s methods.

Three Spaces Problem for Lyapunov Theorem on Vector Measure

It is proved that a Banach space X has the Lyapunov property if its subspace Y and the quotient space X/Y have it.

Uniform Eberlein Compacta and Uniformly Gâteaux Smooth Norms

* Supported by grants: AV ĈR 101-95-02, GAĈR 201-94-0069 (Czech Republic) and NSERC 7926 (Canada).