On the relation of slow feature analysis and Laplacian eigenmaps

The past decade has seen a rise of interest in Laplacian eigenmaps (LEMs) for nonlinear dimensionality reduction. LEMs have been used in spectral clustering, in semisupervised learning, and for providing efficient state representations for reinforcement learning. Here, we show that LEMs are closely related to slow feature analysis (SFA), a biologically inspired, unsupervised learning algorithm originally designed for learning invariant visual representations. We show that SFA can be interpreted as a function approximation of LEMs, where the topological neighborhoods required for LEMs are implicitly defined by the temporal structure of the data. Based on this relation, we propose a generalization of SFA to arbitrary neighborhood relations and demonstrate its applicability for spectral clustering. Finally, we review previous work with the goal of providing a unifying view on SFA and LEMs. © 2011 Massachusetts Institute of Technology.

Predictive coding and the slowness principle: an information-theoretic approach.

Understanding the guiding principles of sensory coding strategies is a main goal in computational neuroscience. Among others, the principles of predictive coding and slowness appear to capture aspects of sensory processing. Predictive coding postulates that sensory systems are adapted to the structure of their input signals such that information about future inputs is encoded. Slow feature analysis (SFA) is a method for extracting slowly varying components from quickly varying input signals, thereby learning temporally invariant features. Here, we use the information bottleneck method to state an information-theoretic objective function for temporally local predictive coding. We then show that the linear case of SFA can be interpreted as a variant of predictive coding that maximizes the mutual information between the current output of the system and the input signal in the next time step. This demonstrates that the slowness principle and predictive coding are intimately related.

Slowness: an objective for spike-timing-dependent plasticity?

Our nervous system can efficiently recognize objects in spite of changes in contextual variables such as perspective or lighting conditions. Several lines of research have proposed that this ability for invariant recognition is learned by exploiting the fact that object identities typically vary more slowly in time than contextual variables or noise. Here, we study the question of how this "temporal stability" or "slowness" approach can be implemented within the limits of biologically realistic spike-based learning rules. We first show that slow feature analysis, an algorithm that is based on slowness, can be implemented in linear continuous model neurons by means of a modified Hebbian learning rule. This approach provides a link to the trace rule, which is another implementation of slowness learning. Then, we show analytically that for linear Poisson neurons, slowness learning can be implemented by spike-timing-dependent plasticity (STDP) with a specific learning window. By studying the learning dynamics of STDP, we show that for functional interpretations of STDP, it is not the learning window alone that is relevant but rather the convolution of the learning window with the postsynaptic potential. We then derive STDP learning windows that implement slow feature analysis and the "trace rule." The resulting learning windows are compatible with physiological data both in shape and timescale. Moreover, our analysis shows that the learning window can be split into two functionally different components that are sensitive to reversible and irreversible aspects of the input statistics, respectively. The theory indicates that irreversible input statistics are not in favor of stable weight distributions but may generate oscillatory weight dynamics. Our analysis offers a novel interpretation for the functional role of STDP in physiological neurons.

Riemannian metric and geometric mean for positive semidefinite matrices of fixed rank

This paper introduces a new metric and mean on the set of positive semidefinite matrices of fixed-rank. The proposed metric is derived from a well-chosen Riemannian quotient geometry that generalizes the reductive geometry of the positive cone and the associated natural metric. The resulting Riemannian space has strong geometrical properties: it is geodesically complete, and the metric is invariant with respect to all transformations that preserve angles (orthogonal transformations, scalings, and pseudoinversion). A meaningful approximation of the associated Riemannian distance is proposed, that can be efficiently numerically computed via a simple algorithm based on SVD. The induced mean preserves the rank, possesses the most desirable characteristics of a geometric mean, and is easy to compute. © 2009 Society for Industrial and Applied Mathematics.

Synchronization in networks of identical linear systems

The paper investigates the synchronization of a network of identical linear time-invariant state-space models under a possibly time-varying and directed interconnection structure. The main result is the construction of a dynamic output feedback coupling that achieves synchronization if the decoupled systems have no exponentially unstable mode and if the communication graph is uniformly connected. Stronger conditions are shown to be sufficient - but to some extent, also necessary - to ensure synchronization with the diffusive static output coupling often considered in the literature. © 2008 IEEE.

Stabilization of planar collective motion with limited communication

This paper proposes a design methodology to stabilize relative equilibria in a model of identical, steered particles moving in the plane at unit speed. Relative equilibria either correspond to parallel motion of all particles with fixed relative spacing or to circular motion of all particles around the same circle. Particles exchange relative information according to a communication graph that can be undirected or directed and time-invariant or time-varying. The emphasis of this paper is to show how previous results assuming all-to-all communication can be extended to a general communication framework. © 2008 IEEE.

Collective motion of self-propelled particles: Stabilizing symmetric formations on closed curves

We provide feedback control laws to stabilize formations of multiple, unit speed particles on smooth, convex, and closed curves with definite curvature. As in previous work we exploit an analogy with coupled phase oscillators to provide controls which isolate symmetric particle formations that are invariant to rigid translation of all the particles. In this work, we do not require all particles to be able to communicate; rather we assume that inter-particle communication is limited and can be modeled by a fixed, connected, and undirected graph. Because of their unique spectral properties, the Laplacian matrices of circulant graphs play a key role. The methodology is demonstrated using a superellipse, which is a type of curve that includes circles, ellipses, and rounded rectangles. These results can be used in applications involving multiple autonomous vehicles that travel at constant speed around fixed beacons. ©2006 IEEE.

Riemannian geometry of Grassmann manifolds with a view on algorithmic computation

We give simple formulas for the canonical metric, gradient, Lie derivative, Riemannian connection, parallel translation, geodesics and distance on the Grassmann manifold of p-planes in ℝn. In these formulas, p-planes are represented as the column space of n × p matrices. The Newton method on abstract Riemannian manifolds proposed by Smith is made explicit on the Grassmann manifold. Two applications - computing an invariant subspace of a matrix and the mean of subspaces - are worked out.

Local equilibrium of the Gibbs interface in two-phase systems

We analyze the local equilibrium assumption for interfaces from the perspective of gauge transformations, which are the small displacements of Gibbs' dividing surface. The gauge invariance of thermodynamic properties turns out to be equivalent to conditions for jumps of bulk densities across the interface. This insight strengthens the foundations of the local equilibrium assumption for interfaces and can be used to characterize nonequilibrium interfaces in a compact and consistent way, with a clear focus on gauge-invariant properties. Using the principle of gauge invariance, we show that the validity of Clapeyron equations can be extended to nonequilibrium interfaces, and an additional jump condition for the momentum density is recognized to be of the Clapeyron type. © 2012 Europhysics Letters Association.

Probability-Matching Predictors for Extreme Extremes

A location- and scale-invariant predictor is constructed which exhibits good probability matching for extreme predictions outside the span of data drawn from a variety of (stationary) general distributions. It is constructed via the three-parameter {\mu, \sigma, \xi} Generalized Pareto Distribution (GPD). The predictor is designed to provide matching probability exactly for the GPD in both the extreme heavy-tailed limit and the extreme bounded-tail limit, whilst giving a good approximation to probability matching at all intermediate values of the tail parameter \xi. The predictor is valid even for small sample sizes N, even as small as N = 3. The main purpose of this paper is to present the somewhat lengthy derivations which draw heavily on the theory of hypergeometric functions, particularly the Lauricella functions. Whilst the construction is inspired by the Bayesian approach to the prediction problem, it considers the case of vague prior information about both parameters and model, and all derivations are undertaken using sampling theory.

Cognitive Tomography Reveals Complex, Task-Independent Mental Representations

Humans develop rich mental representations that guide their behavior in a variety of everyday tasks. However, it is unknown whether these representations, often formalized as priors in Bayesian inference, are specific for each task or subserve multiple tasks. Current approaches cannot distinguish between these two possibilities because they cannot extract comparable representations across different tasks [1-10]. Here, we develop a novel method, termed cognitive tomography, that can extract complex, multidimensional priors across tasks. We apply this method to human judgments in two qualitatively different tasks, "familiarity" and "odd one out," involving an ecologically relevant set of stimuli, human faces. We show that priors over faces are structurally complex and vary dramatically across subjects, but are invariant across the tasks within each subject. The priors we extract from each task allow us to predict with high precision the behavior of subjects for novel stimuli both in the same task as well as in the other task. Our results provide the first evidence for a single high-dimensional structured representation of a naturalistic stimulus set that guides behavior in multiple tasks. Moreover, the representations estimated by cognitive tomography can provide independent, behavior-based regressors for elucidating the neural correlates of complex naturalistic priors. © 2013 The Authors.

Designing output-feedback predictive controllers by reverse-engineering existing LTI controllers

An approach to designing a constrained output-feedback predictive controller that has the same small-signal properties as a pre-existing output-feedback linear time invariant controller is proposed. Systematic guidelines are proposed to select an appropriate (non-unique) realization of the resulting state observer. A method is proposed to transform a class of offset-free reference tracking controllers into the combination of an observer, steady-state target calculator and predictive controller. The procedure is demonstrated with a numerical example. © 2013 IEEE.

Network reconstruction using knock-out and over-expression data

This paper outlines necessary and sufficient conditions for network reconstruction of linear, time-invariant systems using data from either knock-out or over-expression experiments. These structural system perturbations, which are common in biological experiments, can be formulated as unknown system inputs, allowing the network topology and dynamics to be found. We assume that only partial state measurements are available and propose an algorithm that can reconstruct the network at the level of the measured states using either time-series or steady-state data. A simulated example illustrates how the algorithm successfully reconstructs a network from data. © 2013 EUCA.

Control limitations from distributed sensing: Theory and Extremely Large Telescope application

We investigate performance bounds for feedback control of distributed plants where the controller can be centralized (i.e. it has access to measurements from the whole plant), but sensors only measure differences between neighboring subsystem outputs. Such "distributed sensing" can be a technological necessity in applications where system size exceeds accuracy requirements by many orders of magnitude. We formulate how distributed sensing generally limits feedback performance robust to measurement noise and to model uncertainty, without assuming any controller restrictions (among others, no "distributed control" restriction). A major practical consequence is the necessity to cut down integral action on some modes. We particularize the results to spatially invariant systems and finally illustrate implications of our developments for stabilizing the segmented primary mirror of the European Extremely Large Telescope. © 2013 Elsevier Ltd. All rights reserved.

First principles interatomic potential for tungsten based on Gaussian process regression

An accurate description of atomic interactions, such as that provided by first principles quantum mechanics, is fundamental to realistic prediction of the properties that govern plasticity, fracture or crack propagation in metals. However, the computational complexity associated with modern schemes explicitly based on quantum mechanics limits their applications to systems of a few hundreds of atoms at most. This thesis investigates the application of the Gaussian Approximation Potential (GAP) scheme to atomistic modelling of tungsten - a bcc transition metal which exhibits a brittle-to-ductile transition and whose plasticity behaviour is controlled by the properties of $\frac{1}{2} \langle 111 \rangle$ screw dislocations. We apply Gaussian process regression to interpolate the quantum-mechanical (QM) potential energy surface from a set of points in atomic configuration space. Our training data is based on QM information that is computed directly using density functional theory (DFT). To perform the fitting, we represent atomic environments using a set of rotationally, permutationally and reflection invariant parameters which act as the independent variables in our equations of non-parametric, non-linear regression. We develop a protocol for generating GAP models capable of describing lattice defects in metals by building a series of interatomic potentials for tungsten. We then demonstrate that a GAP potential based on a Smooth Overlap of Atomic Positions (SOAP) covariance function provides a description of the $\frac{1}{2} \langle 111 \rangle$ screw dislocation that is in agreement with the DFT model. We use this potential to simulate the mobility of $\frac{1}{2} \langle 111 \rangle$ screw dislocations by computing the Peierls barrier and model dislocation-vacancy interactions to QM accuracy in a system containing more than 100,000 atoms.