A Maximum-Likelihood Interpretation for Slow Feature Analysis

The brain extracts useful features from a maelstrom of sensory information, and a fundamental goal of theoretical neuroscience is to work out how it does so. One proposed feature extraction strategy is motivated by the observation that the meaning of sensory data, such as the identity of a moving visual object, is often more persistent than the activation of any single sensory receptor. This notion is embodied in the slow feature analysis (SFA) algorithm, which uses “slowness” as an heuristic by which to extract semantic information from multi-dimensional time-series. Here, we develop a probabilistic interpretation of this algorithm showing that inference and learning in the limiting case of a suitable probabilistic model yield exactly the results of SFA. Similar equivalences have proved useful in interpreting and extending comparable algorithms such as independent component analysis. For SFA, we use the equivalent probabilistic model as a conceptual spring-board, with which to motivate several novel extensions to the algorithm.

Spin-cluster effect and lattice-deformation-induced Kondo effect, spin-glass freezing, and strong phonon scattering in La 0.7 Ca 0.3Mn 1-xCr xO 3

Besides the Kondo effect observed in dilute magnetic alloys, the Cr-doped perovskite manganate compounds La0.7 Ca0.3 Mn1-x Crx O3 also exhibit Kondo effect and spin-glass freezing in a certain composition range. An extensive investigation for the La0.7 Ca0.3 Mn1-x Crx O3 (x=0.01, 0.05, 0.10, 0.3, 0.6, and 1.0) system on the magnetization and ac susceptibility, the resistivity and magnetoresistance, as well as the thermal conductivity is done at low temperature. The spin-glass behavior has been confirmed for these compounds with x=0.05, 0.1, and 0.3. For temperatures above Tf (the spin-glass freezing temperature) a Curie-Weiss law is obeyed. The paramagnetic Curie temperature θ is dependent on Cr doping. Below Tf there exists a Kondo minimum in the resistivity. Colossal magnetoresistance has been observed in this system with Cr concentration up to x=0.6. We suppose that the substitution of Mn with Cr dilutes Mn ions and changes the long-range ferromagnetic order of La0.7 Ca0.3 MnO3. These behaviors demonstrate that short-range ferromagnetic correlation and fluctuation exist among Mn spins far above Tf. Furthermore, these interactions are a precursor of the cooperative freezing at Tf. The "double bumps" feature in the resistivity-temperature curve is observed in compounds with x=0.05 and 0.1. The phonon scattering is enhanced at low temperatures, where the second peak of double bumps comes out. The results indicate that the spin-cluster effect and lattice deformation induce Kondo effect, spin-glass freezing, and strong phonon scattering in mixed perovskite La0.7 Ca0.3 Mn1-x Crx O3. © 2005 American Institute of Physics.

On the relation of slow feature analysis and Laplacian eigenmaps

The past decade has seen a rise of interest in Laplacian eigenmaps (LEMs) for nonlinear dimensionality reduction. LEMs have been used in spectral clustering, in semisupervised learning, and for providing efficient state representations for reinforcement learning. Here, we show that LEMs are closely related to slow feature analysis (SFA), a biologically inspired, unsupervised learning algorithm originally designed for learning invariant visual representations. We show that SFA can be interpreted as a function approximation of LEMs, where the topological neighborhoods required for LEMs are implicitly defined by the temporal structure of the data. Based on this relation, we propose a generalization of SFA to arbitrary neighborhood relations and demonstrate its applicability for spectral clustering. Finally, we review previous work with the goal of providing a unifying view on SFA and LEMs. © 2011 Massachusetts Institute of Technology.

A theory of slow feature analysis for transformation-based input signals with an application to complex cells

We develop a group-theoretical analysis of slow feature analysis for the case where the input data are generated by applying a set of continuous transformations to static templates. As an application of the theory, we analytically derive nonlinear visual receptive fields and show that their optimal stimuli, as well as the orientation and frequency tuning, are in good agreement with previous simulations of complex cells in primary visual cortex (Berkes and Wiskott, 2005). The theory suggests that side and end stopping can be interpreted as a weak breaking of translation invariance. Direction selectivity is also discussed. © 2011 Massachusetts Institute of Technology.

Slow peaking and low-gain designs for global stabilization of nonlinear systems

This paper presents an analysis of the slow-peaking phenomenon, a pitfall of low-gain designs that imposes basic limitations to large regions of attraction in nonlinear control systems. The phenomenon is best understood on a chain of integrators perturbed by a vector field up(x, u) that satisfies p(x, 0) = 0. Because small controls (or low-gain designs) are sufficient to stabilize the unperturbed chain of integrators, it may seem that smaller controls, which attenuate the perturbation up(x, u) in a large compact set, can be employed to achieve larger regions of attraction. This intuition is false, however, and peaking may cause a loss of global controllability unless severe growth restrictions are imposed on p(x, u). These growth restrictions are expressed as a higher order condition with respect to a particular weighted dilation related to the peaking exponents of the nominal system. When this higher order condition is satisfied, an explicit control law is derived that achieves global asymptotic stability of x = 0. This stabilization result is extended to more general cascade nonlinear systems in which the perturbation p(x, v) v, v = (ξ, u) T, contains the state ξ and the control u of a stabilizable subsystem ξ = a(ξ, u). As an illustration, a control law is derived that achieves global stabilization of the frictionless ball-and-beam model.