Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Rapid assessment of internodal myelin integrity in central nervous system tissue

Monitoring pathology/regeneration in experimental models of de-/remyelination requires an accurate measure not only of functional changes but also of the amount of myelin. We tested whether X-ray diffraction (XRD), which measures periodicity in unfixed myelin, can assess the structural integrity of myelin in fixed tissue. From laboratories involved in spinal cord injury research and in studying the aging primate brain, we solicited "blind" samples and used an electronic detector to record rapidly the diffraction patterns (30 min each pattern) from them. We assessed myelin integrity by measuring its periodicity and relative amount. Fixation of tissue itself introduced +/-10% variation in periodicity and +/-40% variation in relative amount of myelin. For samples having the most native-like periods, the relative amounts of myelin detected allowed distinctions to be made between normal and demyelinating segments, between motor and sensory tracts within the spinal cord, and between aged and young primate CNS. Different periodicities also allowed distinctions to be made between samples from spinal cord and nerve roots and between well-fixed and poorly fixed samples. Our findings suggest that, in addition to evaluating the effectiveness of different fixatives, XRD could also be used as a robust and rapid technique for quantitating the relative amount of myelin among spinal cords and other CNS tissue samples from experimental models of de- and remyelination.

ZeroG: overground gait and balance training system

A new overground body-weight support system called ZeroG has been developed that allows patients with severe gait impairments to practice gait and balance activities in a safe, controlled manner. The unloading system is capable of providing up to 300 lb of static support and 150 lb of dynamic (or constant force) support using a custom-series elastic actuator. The unloading system is mounted to a driven trolley, which rides along an overhead rail. We evaluated the performance of ZeroG's unloading system, as well as the trolley tracking system, using benchtop and human-subject testing. Average root-mean-square and peak errors in unloading were 2.2 and 7.2 percent, respectively, over the range of forces tested while trolley tracking errors were less than 3 degrees, indicating the system was able to maintain its position above the subject. We believe training with ZeroG will allow patients to practice activities that are critical to achieving functional independence at home and in the community.

Atypical presentation of Behçet's disease with central nervous system involvement successfully treated with infliximab

Central nervous system involvement is a rare and serious complication of Behçet's disease (BD). Herein, we describe a patient with an atypical central lesion, who experienced progressive hypesthesia of the right arm and sensory loss of the trigeminal nerve together with intense headache. A repeated biopsy was necessary to conclusively establish the diagnosis of BD. Therapy with infusions of infliximab led to a remarkable full remission. TNFα-blocking therapy was successfully replaced by azathioprine. The present well-illustrated case demonstrates the difficulty of establishing the diagnosis of BD with central nervous system involvement, the dramatic benefit of short given TNF-α-blocking agent, and the long-term remission with azathioprin.

Farming Systems and Resilience Building in a Changing Climate On-Farm Comparisons of Organic and Conventional Cocoa Cultivation in Alto Beni, Bolivia

Resilience research has been applied to socioeconomic as well as for agroecological studies in the last 20 years. It provides a conceptual and methodological approach for a better understanding of interrelations between the performance of ecological and social systems. In the research area Alto Beni, Bolivia, the production of cocoa (Theobroma cacao L.), is one of the main sources of income. Farmers in the region have formed producers’ associations to enhance organic cocoa cultivation and obtain fair prices since the 1980s. In cooperation with the long-term system comparisons by the Research Institute of Organic Agriculture (FiBL) in Alto Beni, aspects of the field trial are applied for the use in on-farm research: a comparison of soil fertility, biomass and crop diversity is combined with qualitative interviews and participatory observation methods. Fieldwork is carried out together with Bolivian students through the Swiss KFPE-programme Echanges Universitaires. For the system comparisons, four different land-use types were classified according to their ecological complexity during a preliminary study in 2009: successional agroforestry systems, simple agroforestry systems (both organically managed and certified), traditional systems and conventional monocultures. The study focuses on interrelations between different ways of cocoa cultivation, livelihoods and the related socio-cultural rationales behind them. In particular this second aspect is innovative as it allows to broaden the biophysical perspective to a more comprehensive evaluation with socio-ecological aspects thereby increasing the relevance of the agronomic field studies for development policy and practice. Moreover, such a socio-ecological baseline allows to assess the potential of organic agriculture regarding resilience-building face to socio-environmental stress factors. Among others, the results of the pre-study illustrate local farmers’ perceptions of climate change and the consequences for the different crop-systems: all interviewees mentioned rising temperatures and/or an extended dry season as negative impacts more with regard to their own working conditions than to their crops. This was the case in particular for conventional monocultures and in plots where slash-and-burn cultivation was practised whereas for organic agroforestry systems the advantage of working in the shade was stressed indicating that their relevance rises in the context of climate change.

The C-Port xV® vascular anastomosis system: results from an animal trial

In this study, facilitated anastomosis using an anastomotic device was compared to conventional hand-sewn (HS) vascular anastomosis in an animal model.

Current patch test results with the European baseline series and extensions to it from the 'European Surveillance System on Contact Allergy' network, 2007-2008

The pattern of contact sensitization to the supposedly most important allergens assembled in the baseline series differs between countries, presumably at least partly because of exposure differences. Objectives. To describe the prevalence of contact sensitization to allergens tested in consecutive patients in the years 2007 and 2008, and to discuss possible differences.