Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).

Spindle burst like spike discharge oscillations in organotypic cultures of neonatal rat cortex

Early network oscillations and spindle bursts are typical patterns of spontaneous rhythmic activity in cortical networks of neonatal rodents in vivo and in vitro. The latter can also be triggered in vivo by stimulation of afferent inputs. The mechanisms underlying such oscillations undergo profound developmental changes in the first postnatal weeks. Their possible role in cortical development is postulated but not known in detail. We have studied spontaneous and evoked patterns of activity in organotypic cultures of slices from neonatal rat cortex grown on multielectrode arrays (MEAs) for extracellular single- and multi-unit recording. Episodes of spontaneous spike discharge oscillations at 7 - 25 Hz lasting for 0.6 - 3 seconds appeared in about half of these cultures spontaneously and could be triggered by electrical stimulation of few distinct electrodes. These oscillations usually covered only restricted areas of the slices. Besides oscillations, single population bursts that spread in a wavelike manner over the whole slice also appeared spontaneously and were triggered by electrical stimulation. In most but not all cultures, population bursts preceded the oscillations. Both population bursts and spike discharge oscillations required intact glutamatergic synaptic transmission since they were suppressed by the AMPA/kainate glutamate receptor antagonist CNQX. The NMDA antagonist d-APV suppressed the oscillations but not the population bursts, suggesting an involvement of NMDA receptors in the oscillations. These findings show that spindle burst like cortical rhythms are reproduced in organotypic cultures of neonatal cortex. The culture model thus allows investigating the role of such rhythms in cortical circuit formation. Supported by SNF grant No. 3100A0-107641/1.

Coreleased orexin and glutamate evoke nonredundant spike outputs and computations in histamine neurons.

Stable wakefulness requires orexin/hypocretin neurons (OHNs) and OHR2 receptors. OHNs sense diverse environmental cues and control arousal accordingly. For unknown reasons, OHNs contain multiple excitatory transmitters, including OH peptides and glutamate. To analyze their cotransmission within computational frameworks for control, we optogenetically stimulated OHNs and examined resulting outputs (spike patterns) in a downstream arousal regulator, the histamine neurons (HANs). OHR2s were essential for sustained HAN outputs. OHR2-dependent HAN output increased linearly during constant OHN input, suggesting that the OHN→HAN(OHR2) module may function as an integral controller. OHN stimulation evoked OHR2-dependent slow postsynaptic currents, similar to midnanomolar OH concentrations. Conversely, glutamate-dependent output transiently communicated OHN input onset, peaking rapidly then decaying alongside OHN→HAN glutamate currents. Blocking glutamate-driven spiking did not affect OH-driven spiking and vice versa, suggesting isolation (low cross-modulation) of outputs. Therefore, in arousal regulators, cotransmitters may translate distinct features of OHN activity into parallel, nonredundant control signals for downstream effectors.

CD36 and macrophage scavenger receptor a modulate foam cell formation via inhibition of lipid-laden platelet phagocytosis

CD34 (+) progenitor cells are a promising source of regeneration in atherosclerosis or ischemic heart disease. However, as recently published, CD34(+) progenitor cells have the potential to differentiate not only into endothelial cells but also into foam cells upon interaction with platelets. The mechanism of platelet-induced differentiation of progenitor cells into foam cells is as yet unclear. In the present study we investigated the role of scavenger receptor (SR)-A and CD36 in platelet-induced foam cell formation. Human CD34(+) progenitor cells were freshly derived from human umbilical veins and were co-incubated with platelets (2 x 10(8)/mL) up to 14 days resulting in large lipid-laden foam cells. Developing macrophages expressed SR-A, CD36, and Lox-1 as measured by fluorescent-activated cell sorting analysis. The presence of a blocking anti-CD36 or anti-SR-A antibody nearly abrogated foam cell formation, whereas anti-Lox-1 did not affect foam cell formation. Consistently blocking either anti-CD36 or anti-SR-A antibody significantly reduced the phagocytosis of lipid-laden platelets by macrophages. We conclude that CD36 and SR-A play an important role in platelet-induced foam cell formation from CD34(+) progenitor cells and thus represent a promising target to inhibit platelet-induced foam cell formation.

High aldehyde dehydrogenase activity identifies tumor-initiating and metastasis-initiating cells in human prostate cancer

Metastatic progression of advanced prostate cancer is a major clinical problem. Identifying the cell(s) of origin in prostate cancer and its distant metastases may permit the development of more effective treatment and preventive therapies. In this study, aldehyde dehydrogenase (ALDH) activity was used as a basis to isolate and compare subpopulations of primary human prostate cancer cells and cell lines. ALDH-high prostate cancer cells displayed strongly elevated clonogenicity and migratory behavior in vitro. More strikingly, ALDH-high cells readily formed distant metastases with strongly enhanced tumor progression at both orthotopic and metastatic sites in preclinical models. Several ALDH isoforms were expressed in human prostate cancer cells and clinical specimens of primary prostate tumors with matched bone metastases. Our findings suggest that ALDH-based viable cell sorting can be used to identify and characterize tumor-initiating and, more importantly perhaps, metastasis-initiating cells in human prostate cancer.

The risk of musculoskeletal disorders for workers due to repetitive movements during tomato harvesting

Tomatoes are the most common crop in Italy. The production cycle requires operations in the field and factory that can cause musculoskeletal disorders due to the repetitive movements of the upper limbs of the workers employed in the sorting phase. This research aims to evaluate these risks using the OCRA (occupational repetitive actions) index method This method is based firstly on the calculation of a maximum number of recommended actions, related to the way the operation is performed, and secondly on a comparison of the number of actions effectively carried out by the upper limb with the recommended calculated value. The results of the risk evaluation for workers who manually sort tomatoes during harvest showed a risk for the workers, with an exposure index greater than 20; the OCRA index defines an index higher than 3.5 as unacceptable. The present trend of replacing manual sorting onboard a vehicle with optical sorters seems to be appropriate to reduce the risk of work-related musculoskeletal disorders (WMSDs) and is supported from both a financial point of view and as a quality control measure.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.