Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.

Theories of instrumental learning are centred on understanding how success and failure are used to improve future decisions. These theories highlight a central role for reward prediction errors in updating the values associated with available actions. In animals, substantial evidence indicates that the neurotransmitter dopamine might have a key function in this type of learning, through its ability to modulate cortico-striatal synaptic efficacy. However, no direct evidence links dopamine, striatal activity and behavioural choice in humans. Here we show that, during instrumental learning, the magnitude of reward prediction error expressed in the striatum is modulated by the administration of drugs enhancing (3,4-dihydroxy-L-phenylalanine; L-DOPA) or reducing (haloperidol) dopaminergic function. Accordingly, subjects treated with L-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with haloperidol. Furthermore, incorporating the magnitude of the prediction errors into a standard action-value learning algorithm accurately reproduced subjects' behavioural choices under the different drug conditions. We conclude that dopamine-dependent modulation of striatal activity can account for how the human brain uses reward prediction errors to improve future decisions.

Decentralised minimum-time consensus

We consider the discrete-time dynamics of a network of agents that exchange information according to a nearest-neighbour protocol under which all agents are guaranteed to reach consensus asymptotically. We present a fully decentralised algorithm that allows any agent to compute the final consensus value of the whole network in finite time using the minimum number of successive values of its own state history. We show that the minimum number of steps is related to a Jordan block decomposition of the network dynamics, and present an algorithm to compute the final consensus value in the minimum number of steps by checking a rank condition of a Hankel matrix of local observations. Furthermore, we prove that the minimum number of steps is related to graph theoretical notions that can be directly computed from the Laplacian matrix of the graph and from the minimum external equitable partition. © 2013 Elsevier Ltd. All rights reserved.

Consensus in non-commutative spaces

Convergence analysis of consensus algorithms is revisited in the light of the Hilbert distance. The Lyapunov function used in the early analysis by Tsitsiklis is shown to be the Hilbert distance to consensus in log coordinates. Birkhoff theorem, which proves contraction of the Hilbert metric for any positive homogeneous monotone map, provides an early yet general convergence result for consensus algorithms. Because Birkhoff theorem holds in arbitrary cones, we extend consensus algorithms to the cone of positive definite matrices. The proposed generalization finds applications in the convergence analysis of quantum stochastic maps, which are a generalization of stochastic maps to non-commutative probability spaces. ©2010 IEEE.

Phase synchronization through entrainment by a consensus input

The paper proposes a synchronization mechanism in a set of nonlinear oscillators interconnected through a communication network. In contrast to many existing results, we do not employ strong, diffusive couplings between the individual oscillators. Instead, each individual oscillator is weakly forced by a linear resonator system. The resonator systems are synchronized using results from consensus theory. The synchronized resonator systems force the frequencies of the nonlinear oscillators to a constant frequency and thereby yield synchronization of the oscillators. We prove this result using the theory of small forcings of stable oscillators. This synchronization scheme allows for synchronization of nonlinear oscillators over uniformly connected communication graphs. ©2010 IEEE.

Consensus on homogeneous manifolds

The present paper considers distributed consensus algorithms for agents evolving on a connected compact homogeneous (CCH) manifold. The agents track no external reference and communicate their relative state according to an interconnection graph. The paper first formalizes the consensus problem for synchronization (i.e. maximizing the consensus) and balancing (i.e. minimizing the consensus); it thereby introduces the induced arithmetic mean, an easily computable mean position on CCH manifolds. Then it proposes and analyzes various consensus algorithms on manifolds: natural gradient algorithms which reach local consensus equilibria; an adaptation using auxiliary variables for almost-global synchronization or balancing; and a stochastic gossip setting for global synchronization. It closes by investigating the dependence of synchronization properties on the attraction function between interacting agents on the circle. The theory is also illustrated on SO(n) and on the Grassmann manifolds. ©2009 IEEE.

Consensus optimization on manifolds

The present paper considers distributed consensus algorithms that involve N agents evolving on a connected compact homogeneous manifold. The agents track no external reference and communicate their relative state according to a communication graph. The consensus problem is formulated in terms of the extrema of a cost function. This leads to efficient gradient algorithms to synchronize (i.e., maximizing the consensus) or balance (i.e., minimizing the consensus) the agents; a convenient adaptation of the gradient algorithms is used when the communication graph is directed and time-varying. The cost function is linked to a specific centroid definition on manifolds, introduced here as the induced arithmetic mean, that is easily computable in closed form and may be of independent interest for a number of manifolds. The special orthogonal group SO (n) and the Grassmann manifold Grass (p, n) are treated as original examples. A link is also drawn with the many existing results on the circle. © 2009 Society for Industrial and Applied Mathematics.

Cooperative attitude synchronization in satellite swarms: A consensus approach

The present paper considers the problem of autonomous synchronization of attitudes in a swarm of spacecraft. Building upon our recent results on consensus on manifolds, we model the spacecraft as particles on SO(3) and drive these particles to a common point in SO(3). Unlike the Euler angle or quaternion descriptions, this model suffers no singularities nor double-points. Our approach is fully cooperative and autonomous: we use no leader nor external reference. We present two types of control laws, in terms of applied control torques, that globally drive the swarm towards attitude synchronization: one that requires tree-like or all-to-all inter-satellite communication (most efficient) and one that works with nearly arbitrary communication (most robust).

Consensus under general convexity

A method is proposed to characterize contraction of a set through orthogonal projections. For discrete-time multi-agent systems, quantitative estimates of convergence (to a consensus) rate are provided by means of contracting convex sets. Required convexity for the sets that should include the values that the transition maps of agents take is considered in a more general sense than that of Euclidean geometry. © 2007 IEEE.

Stabilization of collective motion in three dimensions: A consensus approach

This paper proposes a methodology to stabilize relative equilibria in a model of identical, steered particles moving in three-dimensional Euclidean space. Exploiting the Lie group structure of the resulting dynamical system, the stabilization problem is reduced to a consensus problem. We first derive the stabilizing control laws in the presence of all-to-all communication. Providing each agent with a consensus estimator, we then extend the results to a general setting that allows for unidirectional and time-varying communication topologies. © 2007 IEEE.

Change prediction using interface data

Managing change can be challenging due to the high levels of interdependency in concurrent engineering processes. A key activity in engineering change management is propagation analysis, which can be supported using the change prediction method. In common with most other change prediction approaches, the change prediction method has three important limitations: L1: it depends on highly subjective input data; L2: it is capable of modelling 'generalised cases' only and cannot be; customised to assess specific changes; and L3: the input data are static, and thus, guidance does not reflect changes in the design. This article contributes to resolving these limitations by incorporating interface information into the change prediction method. The enhanced method is illustrated using an example based on a flight simulator. © The Author(s) 2013.

Bayesian Structured Prediction Using Gaussian Processes

We introduce a conceptually novel structured prediction model, GPstruct, which is kernelized, non-parametric and Bayesian, by design. We motivate the model with respect to existing approaches, among others, conditional random fields (CRFs), maximum margin Markov networks (M3N), and structured support vector machines (SVMstruct), which embody only a subset of its properties. We present an inference procedure based on Markov Chain Monte Carlo. The framework can be instantiated for a wide range of structured objects such as linear chains, trees, grids, and other general graphs. As a proof of concept, the model is benchmarked on several natural language processing tasks and a video gesture segmentation task involving a linear chain structure. We show prediction accuracies for GPstruct which are comparable to or exceeding those of CRFs and SVMstruct.

Vibration from railways: Can we achieve better than +/-10dB prediction accuracy?

This paper examines the sources of uncertainly in models used to predict vibration from underground railways. It will become clear from this presentation that by varying parameters by a small amount, consistent with uncertainties in measured data, the predicted vibration levels vary significantly, often by more than 10dB. This error cannot be forecast. Small changes made to soil parameters (Compressive and Shear Wave velocities and density), to slab bending stiffness and mass and to the measurement position give rise to changes in vibration levels of more than lOdB. So if 10dB prediction error results from small uncertainties in soil parameters and measurement position it cannot be sensible to rely on prediction models for accuracy better than 10dB. The presentation will demonstrate in real time the use of the new - and freely-available - PiP software for calculating vibration from railway tunnels in real time.