Determinantal Clustering Processes - A Nonparametric Bayesian Approach to Kernel Based Semi-Supervised Clustering

Semi-supervised clustering is the task of clustering data points into clusters where only a fraction of the points are labelled. The true number of clusters in the data is often unknown and most models require this parameter as an input. Dirichlet process mixture models are appealing as they can infer the number of clusters from the data. However, these models do not deal with high dimensional data well and can encounter difficulties in inference. We present a novel nonparameteric Bayesian kernel based method to cluster data points without the need to prespecify the number of clusters or to model complicated densities from which data points are assumed to be generated from. The key insight is to use determinants of submatrices of a kernel matrix as a measure of how close together a set of points are. We explore some theoretical properties of the model and derive a natural Gibbs based algorithm with MCMC hyperparameter learning. The model is implemented on a variety of synthetic and real world data sets.

Cognitive Tomography Reveals Complex, Task-Independent Mental Representations

Humans develop rich mental representations that guide their behavior in a variety of everyday tasks. However, it is unknown whether these representations, often formalized as priors in Bayesian inference, are specific for each task or subserve multiple tasks. Current approaches cannot distinguish between these two possibilities because they cannot extract comparable representations across different tasks [1-10]. Here, we develop a novel method, termed cognitive tomography, that can extract complex, multidimensional priors across tasks. We apply this method to human judgments in two qualitatively different tasks, "familiarity" and "odd one out," involving an ecologically relevant set of stimuli, human faces. We show that priors over faces are structurally complex and vary dramatically across subjects, but are invariant across the tasks within each subject. The priors we extract from each task allow us to predict with high precision the behavior of subjects for novel stimuli both in the same task as well as in the other task. Our results provide the first evidence for a single high-dimensional structured representation of a naturalistic stimulus set that guides behavior in multiple tasks. Moreover, the representations estimated by cognitive tomography can provide independent, behavior-based regressors for elucidating the neural correlates of complex naturalistic priors. © 2013 The Authors.

Multilinear function factorisation for time series feature extraction

This work applies a variety of multilinear function factorisation techniques to extract appropriate features or attributes from high dimensional multivariate time series for classification. Recently, a great deal of work has centred around designing time series classifiers using more and more complex feature extraction and machine learning schemes. This paper argues that complex learners and domain specific feature extraction schemes of this type are not necessarily needed for time series classification, as excellent classification results can be obtained by simply applying a number of existing matrix factorisation or linear projection techniques, which are simple and computationally inexpensive. We highlight this using a geometric separability measure and classification accuracies obtained though experiments on four different high dimensional multivariate time series datasets. © 2013 IEEE.

Kernelized log linear models for continuous speech recognition

Large margin criteria and discriminative models are two effective improvements for HMM-based speech recognition. This paper proposed a large margin trained log linear model with kernels for CSR. To avoid explicitly computing in the high dimensional feature space and to achieve the nonlinear decision boundaries, a kernel based training and decoding framework is proposed in this work. To make the system robust to noise a kernel adaptation scheme is also presented. Previous work in this area is extended in two directions. First, most kernels for CSR focus on measuring the similarity between two observation sequences. The proposed joint kernels defined a similarity between two observation-label sequence pairs on the sentence level. Second, this paper addresses how to efficiently employ kernels in large margin training and decoding with lattices. To the best of our knowledge, this is the first attempt at using large margin kernel-based log linear models for CSR. The model is evaluated on a noise corrupted continuous digit task: AURORA 2.0. © 2013 IEEE.

Anisotropy preserving DTI processing

Statistical analysis of diffusion tensor imaging (DTI) data requires a computational framework that is both numerically tractable (to account for the high dimensional nature of the data) and geometric (to account for the nonlinear nature of diffusion tensors). Building upon earlier studies exploiting a Riemannian framework to address these challenges, the present paper proposes a novel metric and an accompanying computational framework for DTI data processing. The proposed approach grounds the signal processing operations in interpolating curves. Well-chosen interpolating curves are shown to provide a computational framework that is at the same time tractable and information relevant for DTI processing. In addition, and in contrast to earlier methods, it provides an interpolation method which preserves anisotropy, a central information carried by diffusion tensor data. © 2013 Springer Science+Business Media New York.

Linear dimensionality reduction: Survey, insights, and generalizations

© 2015 John P. Cunningham and Zoubin Ghahramani. Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, input-output relationships, and margin between data classes. Methods have been developed with a variety of names and motivations in many fields, and perhaps as a result the connections between all these methods have not been highlighted. Here we survey methods from this disparate literature as optimization programs over matrix manifolds. We discuss principal component analysis, factor analysis, linear multidimensional scaling, Fisher's linear discriminant analysis, canonical correlations analysis, maximum autocorrelation factors, slow feature analysis, sufficient dimensionality reduction, undercomplete independent component analysis, linear regression, distance metric learning, and more. This optimization framework gives insight to some rarely discussed shortcomings of well-known methods, such as the suboptimality of certain eigenvector solutions. Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal low-dimensional projection of the data. This simple optimization framework further allows straightforward generalizations and novel variants of classical methods, which we demonstrate here by creating an orthogonal-projection canonical correlations analysis. More broadly, this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox, objective-agnostic numerical technology.

Critical current anisotropy in high temperature superconductors

After nearly 15 years of research effort, High Temperature Superconductors (HTS) are finding a wide range of practical applications. A clear understanding of the factors controlling the current carrying capacity of these materials is a prerequisite to their successful technological development. The critical current density (Jc) in HTS is directly dependent on the structure and pinning of the Flux Line Lattice (FLL) in these materials. This thesis presents an investigation of the Jc anisotropy in HTS. The use of thin films grown on off c-axis (vicinal) substrates allowed the effect of current directions outside the cuprate planes to be studied. With this experimental geometry Berghuis, et al. (Phys. Rev. Lett. 79, 12, pg. 2332) observed a striking flux channelling effect in vicinal YBa2Cu3O7-δ (YBCO) films. By confirming, and extending, this observation, it is demonstrated that this is an intrinsic effect. The results obtained, appear to fit well with the predictions of a field angle dependent cross-over from a three dimensional rectilinear FLL to a kinked lattice of strings and pancakes. The pinning force density for movement of strings inside the cuprate planes is considerably less than that on vortex pancake elements. When the FLL is entirely string-like this reduced pinning leads to the observed channelling minima. It is observed that anti-phase boundaries enhance the Jc in vicinal YBCO films by strongly pinning vortex strings. The effect on the FLL structure cross-over of increasing anisotropy has been elucidated using de-oxygenated vicinal YBCO films. Intriguingly, the counter intuitive prediction that the range of applied field angle for which the kinked lattice is fully developed reduces with increasing anisotropy, appears to be confirmed. Although vortex channelling cannot be observed in c-axis YBCO films, the pinning force density for vortex string channelling has been extracted by observing string dragging. By studying the effect of rotating the applied field at a constant angle to the cuprate planes, it is possible to observe the cross-over into the string pancake regime in c-axis films. In the 3D region, the observed behaviour is well explained by the anisotropic Ginzburg-Landau model. Measurements were also made on thin films of the much more anisotropic Bi 2Sr2CaCu2O8+x material, grown on vicinal substrates. The absence of any flux channelling effect and clear adherence to the expected Kes-Law behaviour in the observed Jc characteristics does not provide evidence for the existence of the predicted ‘crossing lattice’ in Bi 2Sr2CaCu2O8+x .