Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

Symmetry-extended counting rules for periodic frameworks.

A symmetry-adapted version of the Maxwell rule appropriate to periodic bar-and-joint frameworks is obtained, and is further extended to body-and-joint systems. The treatment deals with bodies and forces that are replicated in every unit cell, and uses the point group isomorphic to the factor group of the space group of the framework. Explicit expressions are found for the numbers and symmetries of detectable mechanisms and states of self-stress in terms of the numbers and symmetries of framework components. This approach allows detection and characterization of mechanisms and states of self-stress in microscopic and macroscopic materials and meta-materials. Illustrative examples are described. The notion of local isostaticity of periodic frameworks is extended to include point-group symmetry.