Code-specific learning rules improve action selection by populations of spiking neurons

Population coding is widely regarded as a key mechanism for achieving reliable behavioral decisions. We previously introduced reinforcement learning for population-based decision making by spiking neurons. Here we generalize population reinforcement learning to spike-based plasticity rules that take account of the postsynaptic neural code. We consider spike/no-spike, spike count and spike latency codes. The multi-valued and continuous-valued features in the postsynaptic code allow for a generalization of binary decision making to multi-valued decision making and continuous-valued action selection. We show that code-specific learning rules speed up learning both for the discrete classification and the continuous regression tasks. The suggested learning rules also speed up with increasing population size as opposed to standard reinforcement learning rules. Continuous action selection is further shown to explain realistic learning speeds in the Morris water maze. Finally, we introduce the concept of action perturbation as opposed to the classical weight- or node-perturbation as an exploration mechanism underlying reinforcement learning. Exploration in the action space greatly increases the speed of learning as compared to exploration in the neuron or weight space.

Spike timing dependent plasticity: a consequence of more fundamental learning rules.

Spike timing dependent plasticity (STDP) is a phenomenon in which the precise timing of spikes affects the sign and magnitude of changes in synaptic strength. STDP is often interpreted as the comprehensive learning rule for a synapse - the "first law" of synaptic plasticity. This interpretation is made explicit in theoretical models in which the total plasticity produced by complex spike patterns results from a superposition of the effects of all spike pairs. Although such models are appealing for their simplicity, they can fail dramatically. For example, the measured single-spike learning rule between hippocampal CA3 and CA1 pyramidal neurons does not predict the existence of long-term potentiation one of the best-known forms of synaptic plasticity. Layers of complexity have been added to the basic STDP model to repair predictive failures, but they have been outstripped by experimental data. We propose an alternate first law: neural activity triggers changes in key biochemical intermediates, which act as a more direct trigger of plasticity mechanisms. One particularly successful model uses intracellular calcium as the intermediate and can account for many observed properties of bidirectional plasticity. In this formulation, STDP is not itself the basis for explaining other forms of plasticity, but is instead a consequence of changes in the biochemical intermediate, calcium. Eventually a mechanism-based framework for learning rules should include other messengers, discrete change at individual synapses, spread of plasticity among neighboring synapses, and priming of hidden processes that change a synapse's susceptibility to future change. Mechanism-based models provide a rich framework for the computational representation of synaptic plasticity.

Should There Be New Multilateral Rules for Digital Trade?

The Internet revolution and the digital environment have spurred a significant amount of innovative activity that has had spillover effects on many sectors of the economy. For a growing group of countries – both developed and developing – digital goods and services have become an important engine of economic growth and a clear priority in their future-oriented economic strategies. Neither the rapid technological developments associated with digitization, nor their increased societal significance have so far been reflected in international economic law in a comprehensive manner. The law of the World Trade Organization (WTO) in particular, has not reacted in any proactive manner. A pertinent question that arises is whether the WTO rules are still useful and able to accommodate the new digital economy or whether they have been rendered outdated and incapable of dealing with this important development? The present think-piece seeks answers to these questions and maps the key issues and challenges which the WTO faces. In appraisal of the current state of affairs, developments in venues other than the WTO, and proposals tabled by stakeholders, some recommendations for the ways forward are made.

The WTO and Food Security: What’s Wrong with the Rules?

The WTO Agreement on Agriculture (AoA) is the predominant multilateral legal framework governing agricultural trade. The objective of the AoA is to liberalise trade in agriculture through reductions in tariffs, domestic support and export subsidies. The AoA has not, however, ‘levelled the playing field’ and has not resulted in the equitable distribution of food, particularly for the poorer developing countries. On the other hand, support for small farmers does not ensure food security for the poor. While food security has no simple solutions such as “free trade is good for you”, reform proposals for trade rules which only address agricultural policy instruments fail to account for consumer and other interests: neither tariff reductions and subsidy disciplines, nor safeguards and other measures of producer protection can automatically increase food security. Rather, what is needed is the full and proper implementation of a number of commitments which the international community has already entered into in various human rights treaties, but which even the envisaged results of the now failed Doha Round negotiations could not ensure without revisiting relevant multilateral trade and investment rules.

Do WTO rules improve or impair the right to food?

The global food crisis of 2007–08 seems to be forgotten. Media attention at the time focused on food riots in Haiti and Mozambique, while world leaders and more than a dozen international organizations gathered for several food summits, calling for immediate relief measures. But not a single government seems to remember its obligations under the Right to Food (R2F) which the United Nations (UN) had enshrined back in 1948. Today we have to acknowledge that the R2F still lacks an adequate response under the present multilateral rules and disciplines applying to food production and trade. This chapter examines the present rules and disciplines under the AoA and of those contemplated in the Doha Development Round. Here we find that despite claims to the contrary they contribute precious little to the R2F. Some of the present rules, or the lack thereof, can even act as disincentives for global and national food security. Various forms of production and export subsidies, food aid abuse and export restrictions, are still WTO-legal, with few remedies available to food insecure developing countries. This amounts to a violation of their R2F obligations by many WTO Members.