Slowness: an objective for spike-timing-dependent plasticity?

Our nervous system can efficiently recognize objects in spite of changes in contextual variables such as perspective or lighting conditions. Several lines of research have proposed that this ability for invariant recognition is learned by exploiting the fact that object identities typically vary more slowly in time than contextual variables or noise. Here, we study the question of how this "temporal stability" or "slowness" approach can be implemented within the limits of biologically realistic spike-based learning rules. We first show that slow feature analysis, an algorithm that is based on slowness, can be implemented in linear continuous model neurons by means of a modified Hebbian learning rule. This approach provides a link to the trace rule, which is another implementation of slowness learning. Then, we show analytically that for linear Poisson neurons, slowness learning can be implemented by spike-timing-dependent plasticity (STDP) with a specific learning window. By studying the learning dynamics of STDP, we show that for functional interpretations of STDP, it is not the learning window alone that is relevant but rather the convolution of the learning window with the postsynaptic potential. We then derive STDP learning windows that implement slow feature analysis and the "trace rule." The resulting learning windows are compatible with physiological data both in shape and timescale. Moreover, our analysis shows that the learning window can be split into two functionally different components that are sensitive to reversible and irreversible aspects of the input statistics, respectively. The theory indicates that irreversible input statistics are not in favor of stable weight distributions but may generate oscillatory weight dynamics. Our analysis offers a novel interpretation for the functional role of STDP in physiological neurons.

Regression on fixed-rank positive semidefinite matrices: A Riemannian approach

The paper addresses the problem of learning a regression model parameterized by a fixed-rank positive semidefinite matrix. The focus is on the nonlinear nature of the search space and on scalability to high-dimensional problems. The mathematical developments rely on the theory of gradient descent algorithms adapted to the Riemannian geometry that underlies the set of fixedrank positive semidefinite matrices. In contrast with previous contributions in the literature, no restrictions are imposed on the range space of the learned matrix. The resulting algorithms maintain a linear complexity in the problem size and enjoy important invariance properties. We apply the proposed algorithms to the problem of learning a distance function parameterized by a positive semidefinite matrix. Good performance is observed on classical benchmarks. © 2011 Gilles Meyer, Silvere Bonnabel and Rodolphe Sepulchre.

Paraphrastic language models

In natural languages multiple word sequences can represent the same underlying meaning. Only modelling the observed surface word sequence can result in poor context coverage, for example, when using n-gram language models (LM). To handle this issue, this paper presents a novel form of language model, the paraphrastic LM. A phrase level transduction model that is statistically learned from standard text data is used to generate paraphrase variants. LM probabilities are then estimated by maximizing their marginal probability. Significant error rate reductions of 0.5%-0.6% absolute were obtained on a state-ofthe-art conversational telephone speech recognition task using a paraphrastic multi-level LM modelling both word and phrase sequences.

Values and actions in aversion

Model-based and model-free controllers can, in principle, learn arbitrary actions to optimize their behavior, at least those actions that can be expressed and explored. Indeed, these are often referred to as instrumental controllers because their choices are learned to be instrumental for the delivery of desired outcomes. Although this flexibility is very powerful, it comes with an attendant cost of learning. Evolution appears to have endowed everything from the simplest organisms to us with powerful, pre-specified, but inflexible alternatives. These responses are termed Pavlovian, after the famous Russian physiologist and psychologist Pavlov. The responses of the Pavlovian controller are determined by evolutionary (phylogenetic) considerations rather than (ontogenetic) aspects of the contingent development or learning of an individual. These responses directly interact with instrumental choices arising from goal-directed and habitual controllers. This interaction has been studied in a wealth of animal paradigms, and can be helpful, neutral, or harmful, according to circumstance. Although there has been less careful or analytical study of it in humans, it can be interpreted as underpinning a wealth of behavioral aberrations. © 2009 Elsevier Inc. All rights reserved.

On-line policy optimisation of Bayesian spoken dialogue systems via human interaction

A partially observable Markov decision process has been proposed as a dialogue model that enables robustness to speech recognition errors and automatic policy optimisation using reinforcement learning (RL). However, conventional RL algorithms require a very large number of dialogues, necessitating a user simulator. Recently, Gaussian processes have been shown to substantially speed up the optimisation, making it possible to learn directly from interaction with human users. However, early studies have been limited to very low dimensional spaces and the learning has exhibited convergence problems. Here we investigate learning from human interaction using the Bayesian Update of Dialogue State system. This dynamic Bayesian network based system has an optimisation space covering more than one hundred features, allowing a wide range of behaviours to be learned. Using an improved policy model and a more robust reward function, we show that stable learning can be achieved that significantly outperforms a simulator trained policy. © 2013 IEEE.

Gaussian process volatility model

The prediction of time-changing variances is an important task in the modeling of financial data. Standard econometric models are often limited as they assume rigid functional relationships for the evolution of the variance. Moreover, functional parameters are usually learned by maximum likelihood, which can lead to over-fitting. To address these problems we introduce GP-Vol, a novel non-parametric model for time-changing variances based on Gaussian Processes. This new model can capture highly flexible functional relationships for the variances. Furthermore, we introduce a new online algorithm for fast inference in GP-Vol. This method is much faster than current offline inference procedures and it avoids overfitting problems by following a fully Bayesian approach. Experiments with financial data show that GP-Vol performs significantly better than current standard alternatives.

Soft robotics: The next generation of intelligent machines

There has been an increasing interest in applying biological principles to the design and control of robots. Unlike industrial robots that are programmed to execute a rather limited number of tasks, the new generation of bio-inspired robots is expected to display a wide range of behaviours in unpredictable environments, as well as to interact safely and smoothly with human co-workers. In this article, we put forward some of the properties that will characterize these new robots: soft materials, flexible and stretchable sensors, modular and efficient actuators, self-organization and distributed control. We introduce a number of design principles; in particular, we try to comprehend the novel design space that now includes soft materials and requires a completely different way of thinking about control. We also introduce a recent case study of developing a complex humanoid robot, discuss the lessons learned and speculate about future challenges and perspectives.

Finding resonance: Adaptive frequency oscillators for dynamic legged locomotion

There is much to gain from providing walking machines with passive dynamics, e.g. by including compliant elements in the structure. These elements can offer interesting properties such as self-stabilization, energy efficiency and simplified control. However, there is still no general design strategy for such robots and their controllers. In particular, the calibration of control parameters is often complicated because of the highly nonlinear behavior of the interactions between passive components and the environment. In this article, we propose an approach in which the calibration of a key parameter of a walking controller, namely its intrinsic frequency, is done automatically. The approach uses adaptive frequency oscillators to automatically tune the intrinsic frequency of the oscillators to the resonant frequency of a compliant quadruped robot The tuning goes beyond simple synchronization and the learned frequency stays in the controller when the robot is put to halt. The controller is model free, robust and simple. Results are presented illustrating how the controller can robustly tune itself to the robot, as well as readapt when the mass of the robot is changed. We also provide an analysis of the convergence of the frequency adaptation for a linearized plant, and show how that analysis is useful for determining which type of sensory feedback must be used for stable convergence. This approach is expected to explain some aspects of developmental processes in biological and artificial adaptive systems that "develop" through the embodied system-environment interactions. © 2006 IEEE.

Analysis of structural health monitoring data from hammersmith flyover

There has recently been considerable research published on the applicability of monitoring systems for improving civil infrastructure management decisions. Less research has been published on the challenges in interpreting the collected data to provide useful information for engineering decision makers. This paper describes some installed monitoring systems on the Hammersmith Flyover, a major bridge located in central London (United Kingdom). The original goals of the deployments were to evaluate the performance of systems for monitoring prestressing tendon wire breaks and to assess the performance of the bearings supporting the bridge piers because visual inspections had indicated evidence of deterioration in both. This paper aims to show that value can be derived from detailed analysis of measurements from a number of different sensors, including acoustic emission monitors, strain, temperature and displacement gauges. Two structural monitoring systems are described, a wired system installed by a commercial contractor on behalf of the client and a research wireless deployment installed by the University of Cambridge. Careful interpretation of the displacement and temperature gauge data enabled bearings that were not functioning as designed to be identified. The acoustic emission monitoring indicated locations at which rapid deterioration was likely to be occurring; however, it was not possible to verify these results using any of the other sensors installed and hence the only method for confirming these results was by visual inspection. Recommendations for future bridge monitoring projects are made in light of the lessons learned from this monitoring case study. © 2014 This work is made available under the terms of the Creative Commons Attribution 4.0 International license,.