Spike-based decision learning of nash equilibria in two-player games

Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.

Climate change: an additional risk factor for agriculture and food security in the South

Climate change is expected to have far-reaching negative effects on agricultural production and food security in developing and transition countries. What do we know about these expected impacts, what are the factors that might affect production, and what are the implications for agricultural extension systems?

Proposal of Bisgaardia hudsonensis gen. nov., sp. nov. and an additional genomospecies, isolated from seals, as new members of the family Pasteurellaceae

Phenotypic and phylogenetic studies were performed on eight Gram-negative-staining, rod-shaped bacteria isolated from seals. Biochemical and physiological studies showed identical profiles for all of the isolates and indicated that they were related to the family Pasteurellaceae. 16S rRNA gene sequencing demonstrated that the organism represented a distinct cluster with two sublines within the family Pasteurellaceae with <96% sequence similarity to any recognized species. Multilocus sequence analysis (MLSA) including rpoB, infB and recN genes further confirmed these findings with the eight isolates forming a genus-like cluster with two branches. Genome relatedness as deduced from recN gene sequences suggested that the isolates represented a new genus with two species. On the basis of the results of the phylogenetic analysis and phenotypic criteria, it is proposed that these bacteria from seals are classified as Bisgaardia hudsonensis gen. nov., sp. nov. (the type species) and Bisgaardia genomospecies 1. The G+C content of the DNA was 39.5 mol%. The type strain of Bisgaardia hudsonensis gen. nov., sp. nov. is M327/99/2(T) (=CCUG 43067(T)=NCTC 13475(T)=98-D-690B(T)) and the reference strain of Bisgaardia genomospecies 1 is M1765/96/5 (=CCUG 59551=NCTC 13474).

TA-RE: An Exchange Language for Mining Software Repositories

Software repositories have been getting a lot of attention from researchers in recent years. In order to analyze software repositories, it is necessary to first extract raw data from the version control and problem tracking systems. This poses two challenges: (1) extraction requires a non-trivial effort, and (2) the results depend on the heuristics used during extraction. These challenges burden researchers that are new to the community and make it difficult to benchmark software repository mining since it is almost impossible to reproduce experiments done by another team. In this paper we present the TA-RE corpus. TA-RE collects extracted data from software repositories in order to build a collection of projects that will simplify extraction process. Additionally the collection can be used for benchmarking. As the first step we propose an exchange language capable of making sharing and reusing data as simple as possible.

Treatment of Retained Fetal Membranes: Comparison of the Postpartum Period after Routine Treatment or Routine Treatment Including an Additional Phytotherapeutic Substance in Dairy Cattle in Switzerland

Background: The therapy of retained fetal membranes (RFM) is a controversial subject. In Switzerland, intrauterine antibiotics are routinely administered although their effect on fertility parameters is questionable. The objective of this study was to compare the post-partal period after a routine treatment of RFM in 2 groups: one group received a placebo additionally (A), whereas the other group received a phytotherapeutic substance (lime bark) (B) additionally. The routine treatment of RFM included an attempt to manually remove the fetal membranes (for a maximum of 5 min), intramuscular administration of oxytetracycline and intrauterine treatment with tetracycline. In case of an elevated rectal temperature (>39.0°C), an additional non-steroidal inflam-matory drug was allowed. Methods: Cows undergoing caesarean section, suffering from prolapse of the uterus, deep cervical or vaginal injuries, hypocalcaemia, and illnesses during the last 14 days before calving were excluded. Cows had to be more than 265 days pregnant. Only cows that were artificially inseminated after RFM were included. Group stratification was done according to the last number on the ear tag (even/uneven) with (n = 50) cows in group A and (n = 55) cows in group B. Results: The number of treatments after the initial treatment of RFM was not significantly different between groups. The median interval from calving to the first insemination was 77 days in group A compared to 82 days in group B (p = 0.72). The number of AI’s until conception was not significantly different between groups. The median number of days open was 89 days in group A compared to 96 days in group B (p = 0.57). The culling rate was not significantly different between groups. Conclusion: There was neither a difference between the groups concerning therapies within the first 50 days after RFM nor concerning the subsequent fertility variables.

Plasticity in the adult language system: a longitudinal electrophysiological study on second language learning

Event-related potentials (ERPs) were used to trace changes in brain activity related to progress in second language learning. Twelve English-speaking exchange students learning German in Switzerland were recruited. ERPs to visually presented single words from the subjects' native language (English), second language (German) and an unknown language (Romansh) were measured before (day 1) and after (day 2) 5 months of intense German language learning. When comparing ERPs to German words from day 1 and day 2, we found topographic differences between 396 and 540 ms. These differences could be interpreted as a latency shift indicating faster processing of German words on day 2. Source analysis indicated that the topographic differences were accounted for by shorter activation of left inferior frontal gyrus (IFG) on day 2. In ERPs to English words, we found Global Field Power differences between 472 and 644 ms. This may due to memory traces related to English words being less easily activated on day 2. Alternatively, it might reflect the fact that--with German words becoming familiar on day 2--English words loose their oddball character and thus produce a weaker P300-like effect on day 2. In ERPs to Romansh words, no differences were observed. Our results reflect plasticity in the neuronal networks underlying second language acquisition. They indicate that with a higher level of second language proficiency, second language word processing is faster and requires shorter frontal activation. Thus, our results suggest that the reduced IFG activation found in previous fMRI studies might not reflect a generally lower activation but rather a shorter duration of activity.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).