Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

Single-sensor maximum power point tracking algorithms

Single-sensor maximum power point tracking algorithms for photovoltaic systems are presented. The algorithms have the features, characteristics and advantages of the widely used incremental conductance (INC) algorithm. However; unlike the INC algorithm which requires two sensors (the voltage sensor and the current sensor), the single-sensor algorithms are more desirable because they require only one sensor: the voltage sensor. The algorithms operate by maximising power at the DC-DC converter output, instead of the input. © 2013 The Institution of Engineering and Technology.

Generalized power method for sparse principal component analysis

In this paper we develop a new approach to sparse principal component analysis (sparse PCA). We propose two single-unit and two block optimization formulations of the sparse PCA problem, aimed at extracting a single sparse dominant principal component of a data matrix, or more components at once, respectively. While the initial formulations involve nonconvex functions, and are therefore computationally intractable, we rewrite them into the form of an optimization program involving maximization of a convex function on a compact set. The dimension of the search space is decreased enormously if the data matrix has many more columns (variables) than rows. We then propose and analyze a simple gradient method suited for the task. It appears that our algorithm has best convergence properties in the case when either the objective function or the feasible set are strongly convex, which is the case with our single-unit formulations and can be enforced in the block case. Finally, we demonstrate numerically on a set of random and gene expression test problems that our approach outperforms existing algorithms both in quality of the obtained solution and in computational speed. © 2010 Michel Journée, Yurii Nesterov, Peter Richtárik and Rodolphe Sepulchre.

A PDE viewpoint on basic properties of coordination algorithms with symmetries

Several recent control applications consider the coordination of subsystems through local interaction. Often the interaction has a symmetry in state space, e.g. invariance with respect to a uniform translation of all subsystem values. The present paper shows that in presence of such symmetry, fundamental properties can be highlighted by viewing the distributed system as the discrete approximation of a partial differential equation. An important fact is that the symmetry on the state space differs from the popular spatial invariance property, which is not necessary for the present results. The relevance of the viewpoint is illustrated on two examples: (i) ill-conditioning of interaction matrices in coordination/consensus problems and (ii) the string instability issue. ©2009 IEEE.

Consensus on homogeneous manifolds

The present paper considers distributed consensus algorithms for agents evolving on a connected compact homogeneous (CCH) manifold. The agents track no external reference and communicate their relative state according to an interconnection graph. The paper first formalizes the consensus problem for synchronization (i.e. maximizing the consensus) and balancing (i.e. minimizing the consensus); it thereby introduces the induced arithmetic mean, an easily computable mean position on CCH manifolds. Then it proposes and analyzes various consensus algorithms on manifolds: natural gradient algorithms which reach local consensus equilibria; an adaptation using auxiliary variables for almost-global synchronization or balancing; and a stochastic gossip setting for global synchronization. It closes by investigating the dependence of synchronization properties on the attraction function between interacting agents on the circle. The theory is also illustrated on SO(n) and on the Grassmann manifolds. ©2009 IEEE.

Consensus optimization on manifolds

The present paper considers distributed consensus algorithms that involve N agents evolving on a connected compact homogeneous manifold. The agents track no external reference and communicate their relative state according to a communication graph. The consensus problem is formulated in terms of the extrema of a cost function. This leads to efficient gradient algorithms to synchronize (i.e., maximizing the consensus) or balance (i.e., minimizing the consensus) the agents; a convenient adaptation of the gradient algorithms is used when the communication graph is directed and time-varying. The cost function is linked to a specific centroid definition on manifolds, introduced here as the induced arithmetic mean, that is easily computable in closed form and may be of independent interest for a number of manifolds. The special orthogonal group SO (n) and the Grassmann manifold Grass (p, n) are treated as original examples. A link is also drawn with the many existing results on the circle. © 2009 Society for Industrial and Applied Mathematics.

Optimization Algorithms on Matrix Manifolds

This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms.

Grain boundary interface mechanics in strain gradient crystal plasticity

Interactions between dislocations and grain boundaries play an important role in the plastic deformation of polycrystalline metals. Capturing accurately the behaviour of these internal interfaces is particularly important for applications where the relative grain boundary fraction is significant, such as ultra fine-grained metals, thin films and microdevices. Incorporating these micro-scale interactions (which are sensitive to a number of dislocation, interface and crystallographic parameters) within a macro-scale crystal plasticity model poses a challenge. The innovative features in the present paper include (i) the formulation of a thermodynamically consistent grain boundary interface model within a microstructurally motivated strain gradient crystal plasticity framework, (ii) the presence of intra-grain slip system coupling through a microstructurally derived internal stress, (iii) the incorporation of inter-grain slip system coupling via an interface energy accounting for both the magnitude and direction of contributions to the residual defect from all slip systems in the two neighbouring grains, and (iv) the numerical implementation of the grain boundary model to directly investigate the influence of the interface constitutive parameters on plastic deformation. The model problem of a bicrystal deforming in plane strain is analysed. The influence of dissipative and energetic interface hardening, grain misorientation, asymmetry in the grain orientations and the grain size are systematically investigated. In each case, the crystal response is compared with reference calculations with grain boundaries that are either 'microhard' (impenetrable to dislocations) or 'microfree' (an infinite dislocation sink). © 2013 Elsevier Ltd. All rights reserved.

Finite Element studies on indentation size effect using a higher order strain gradient theory

In this work, a Finite Element implementation of a higher order strain gradient theory (due to Fleck and Hutchinson, 2001) has been used within the framework of large deformation elasto-viscoplasticity to study the indentation of metals with indenters of various geometries. Of particular interest is the indentation size effect (ISE) commonly observed in experiments where the hardness of a range of materials is found to be significantly higher at small depths of indentation but reduce to a lower, constant value at larger depths. That the ISE can be explained by strain gradient plasticity is well known but this work aims to qualitatively compare a gamut of experimental observations on this effect with predictions from a higher order strain gradient theory. Results indicate that many of the experimental observations are qualitatively borne out by our simulations. However, areas exist where conflicting experimental results make assessment of numerical predictions difficult. © 2012 Elsevier Ltd. All rights reserved.

Optimization on manifolds: Methods and applications

This paper provides an introduction to the topic of optimization on manifolds. The approach taken uses the language of differential geometry, however,we choose to emphasise the intuition of the concepts and the structures that are important in generating practical numerical algorithms rather than the technical details of the formulation. There are a number of algorithms that can be applied to solve such problems and we discuss the steepest descent and Newton's method in some detail as well as referencing the more important of the other approaches.There are a wide range of potential applications that we are aware of, and we briefly discuss these applications, as well as explaining one or two in more detail. © 2010 Springer -Verlag Berlin Heidelberg.