Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

New ways of looking at synapses

Current concepts of synaptic fine-structure are derived from electron microscopic studies of tissue fixed by chemical fixation using aldehydes. However, chemical fixation with glutaraldehyde and paraformaldehyde and subsequent dehydration in ethanol result in uncontrolled tissue shrinkage. While electron microscopy allows for the unequivocal identification of synaptic contacts, it cannot be used for real-time analysis of structural changes at synapses. For the latter purpose advanced fluorescence microscopy techniques are to be applied which, however, do not allow for the identification of synaptic contacts. Here, two approaches are described that may overcome, at least in part, some of these drawbacks in the study of synapses. By focusing on a characteristic, easily identifiable synapse, the mossy fiber synapse in the hippocampus, we first describe high-pressure freezing of fresh tissue as a method that may be applied to study subtle changes in synaptic ultrastructure associated with functional synaptic plasticity. Next, we propose to label presynaptic mossy fiber terminals and postsynaptic complex spines on CA3 pyramidal neurons by different fluorescent dyes to allow for the real-time monitoring of these synapses in living tissue over extended periods of time. We expect these approaches to lead to new insights into the structure and function of central synapses.

Fine structure of synapses on dendritic spines

Camillo Golgi's "Reazione Nera" led to the discovery of dendritic spines, small appendages originating from dendritic shafts. With the advent of electron microscopy (EM) they were identified as sites of synaptic contact. Later it was found that changes in synaptic strength were associated with changes in the shape of dendritic spines. While live-cell imaging was advantageous in monitoring the time course of such changes in spine structure, EM is still the best method for the simultaneous visualization of all cellular components, including actual synaptic contacts, at high resolution. Immunogold labeling for EM reveals the precise localization of molecules in relation to synaptic structures. Previous EM studies of spines and synapses were performed in tissue subjected to aldehyde fixation and dehydration in ethanol, which is associated with protein denaturation and tissue shrinkage. It has remained an issue to what extent fine structural details are preserved when subjecting the tissue to these procedures. In the present review, we report recent studies on the fine structure of spines and synapses using high-pressure freezing (HPF), which avoids protein denaturation by aldehydes and results in an excellent preservation of ultrastructural detail. In these studies, HPF was used to monitor subtle fine-structural changes in spine shape associated with chemically induced long-term potentiation (cLTP) at identified hippocampal mossy fiber synapses. Changes in spine shape result from reorganization of the actin cytoskeleton. We report that cLTP was associated with decreased immunogold labeling for phosphorylated cofilin (p-cofilin), an actin-depolymerizing protein. Phosphorylation of cofilin renders it unable to depolymerize F-actin, which stabilizes the actin cytoskeleton. Decreased levels of p-cofilin, in turn, suggest increased actin turnover, possibly underlying the changes in spine shape associated with cLTP. The findings reviewed here establish HPF as an appropriate method for studying the fine structure and molecular composition of synapses on dendritic spines.

Synaptic regulation of spike firing in interneurons of the striatum in vivo

The striatum, the major input nucleus of the basal ganglia, is numerically dominated by a single class of principal neurons, the GABAergic spiny projection neuron (SPN) that has been extensively studied both in vitro and in vivo. Much less is known about the sparsely distributed interneurons, principally the cholinergic interneuron (CIN) and the GABAergic fast-spiking interneuron (FSI). Here, we summarize results from two recent studies on these interneurons where we used in vivo intracellular recording techniques in urethane-anaesthetized rats (Schulz et al., J Neurosci 31[31], 2011; J Physiol, in press). Interneurons were identified by their characteristic responses to intracellular current steps and spike waveforms. Spontaneous spiking contained a high proportion (~45%) of short inter-spike intervals (ISI) of <30 ms in FSIs, but virtually none in CINs. Spiking patterns in CINs covered a broad spectrum ranging from regular tonic spiking to phasic activity despite very similar unimodal membrane potential distributions across neurons. In general, phasic spiking activity occurred in phase with the slow ECoG waves, whereas CINs exhibiting tonic regular spiking were little affected by afferent network activity. In contrast, FSIs exhibited transitions between Down and Up states very similar to SPNs. Compared to SPNs, the FSI Up state membrane potential was noisier and power spectra exhibited significantly larger power at frequencies in the gamma range (55-95 Hz). Cortical-evoked inputs had faster dynamics in FSIs than SPNs and the membrane potential preceding spontaneous spike discharge exhibited short and steep trajectories, suggesting that fast input components controlled spike output in FSIs. Intrinsic resonance mechanisms may have further enhanced the sensitivity of FSIs to fast oscillatory inputs. Induction of an activated ECoG state by local ejection of bicuculline into the superior colliculus, resulted in increased spike frequency in both interneuron classes without changing the overall distribution of ISIs. This manipulation also made CINs responsive to a light flashed into the contralateral eye. Typically, the response consisted of an excitation at short latency followed by a pause in spike firing, via an underlying depolarization-hyperpolarization membrane sequence. These results highlight the differential sensitivity of striatal interneurons to afferent synaptic signals and support a model where CINs modulate the striatal network in response to salient sensory bottom-up signals, while FSIs serve gating of top-down signals from the cortex during action selection and reward-related learning.

Impact of interocclusal contacts on infrared laser fluorescence in pits of sound first permanent molars in children

A device based on infrared laser fluorescence (IRLF) has become available as an adjunct for the diagnosis of dental caries.

Eluding oblivion with smart stochastic selection of synaptic updates

The variables involved in the equations that describe realistic synaptic dynamics always vary in a limited range. Their boundedness makes the synapses forgetful, not for the mere passage of time, but because new experiences overwrite old memories. The forgetting rate depends on how many synapses are modified by each new experience: many changes means fast learning and fast forgetting, whereas few changes means slow learning and long memory retention. Reducing the average number of modified synapses can extend the memory span at the price of a reduced amount of information stored when a new experience is memorized. Every trick which allows to slow down the learning process in a smart way can improve the memory performance. We review some of the tricks that allow to elude fast forgetting (oblivion). They are based on the stochastic selection of the synapses whose modifications are actually consolidated following each new experience. In practice only a randomly selected, small fraction of the synapses eligible for an update are actually modified. This allows to acquire the amount of information necessary to retrieve the memory without compromising the retention of old experiences. The fraction of modified synapses can be further reduced in a smart way by changing synapses only when it is really necessary, i.e. when the post-synaptic neuron does not respond as desired. Finally we show that such a stochastic selection emerges naturally from spike driven synaptic dynamics which read noisy pre and post-synaptic neural activities. These activities can actually be generated by a chaotic system.

VE-PTP and VE-cadherin ectodomains interact to facilitate regulation of phosphorylation and cell contacts

VE-cadherin is the essential adhesion molecule in endothelial adherens junctions, and the regulation of protein tyrosine phosphorylation is thought to be important for the control of adherens junction integrity. We show here that VE-PTP (vascular endothelial protein tyrosine phosphatase), an endothelial receptor-type phosphatase, co-precipitates with VE-cadherin, but not with beta-catenin, from cell lysates of transfected COS-7 cells and of endothelial cells. Co-precipitation of VE-cadherin and VE-PTP required the most membrane-proximal extracellular domains of each protein. Expression of VE-PTP in triple-transfected COS-7 cells and in CHO cells reversed the tyrosine phosphorylation of VE-cadherin elicited by vascular endothelial growth factor receptor 2 (VEGFR-2). Expression of VE-PTP under an inducible promotor in CHO cells transfected with VE-cadherin and VEGFR-2 increased the VE-cadherin-mediated barrier integrity of a cellular monolayer. Surprisingly, a catalytically inactive mutant form of VE-PTP had the same effect on VE-cadherin phosphorylation and cell layer permeability. Thus, VE-PTP is a transmembrane binding partner of VE-cadherin that associates through an extracellular domain and reduces the tyrosine phosphorylation of VE-cadherin and cell layer permeability independently of its enzymatic activity.

Learning real-world stimuli in a neural network with spike-driven synaptic dynamics

We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).

Angiopoietins assemble distinct Tie2 signalling complexes in endothelial cell-cell and cell-matrix contacts

The receptor tyrosine kinase Tie2, and its activating ligand Angiopoietin-1 (Ang1), are required for vascular remodelling and vessel integrity, whereas Ang2 may counteract these functions. However, it is not known how Tie2 transduces these different signals. Here, we show that Ang1 induces unique Tie2 complexes in mobile and confluent endothelial cells. Matrix-bound Ang1 induced cell adhesion, motility and Tie2 activation in cell-matrix contacts that became translocated to the trailing edge in migrating endothelial cells. In contrast, in contacting cells Ang1 induced Tie2 translocation to cell-cell contacts and the formation of homotypic Tie2-Tie2 trans-associated complexes that included the vascular endothelial phosphotyrosine phosphatase, leading to inhibition of paracellular permeability. Distinct signalling proteins were preferentially activated by Tie2 in the cell-matrix and cell-cell contacts, where Ang2 inhibited Ang1-induced Tie2 activation. This novel type of cellular microenvironment-dependent receptor tyrosine kinase activation may explain some of the effects of angiopoietins in angiogenesis and vessel stabilization.