Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Dual-energy multidetector CT: how does it work, what can it tell us, and when can we use it in abdominopelvic imaging?

Dual-energy CT provides information about how substances behave at different energies, the ability to generate virtual unenhanced datasets, and improved detection of iodine-containing substances on low-energy images. Knowing how a substance behaves at two different energies can provide information about tissue composition beyond that obtainable with single-energy techniques. The term K edge refers to the spike in attenuation that occurs at energy levels just greater than that of the K-shell binding because of the increased photoelectric absorption at these energy levels. K-edge values vary for each element, and they increase as the atomic number increases. The energy dependence of the photoelectric effect and the variability of K edges form the basis of dual-energy techniques, which may be used to detect substances such as iodine, calcium, and uric acid crystals. The closer the energy level used in imaging is to the K edge of a substance such as iodine, the more the substance attenuates. In the abdomen and pelvis, dual-energy CT may be used in the liver to increase conspicuity of hypervascular lesions; in the kidneys, to distinguish hyperattenuating cysts from enhancing renal masses and to characterize renal stone composition; in the adrenal glands, to characterize adrenal nodules; and in the pancreas, to differentiate between normal and abnormal parenchyma.

The time window for generation of dendritic spikes by coincidence of action potentials and EPSPs is layer specific in somatosensory cortex

The precise timing of events in the brain has consequences for intracellular processes, synaptic plasticity, integration and network behaviour. Pyramidal neurons, the most widespread excitatory neuron of the neocortex have multiple spike initiation zones, which interact via dendritic and somatic spikes actively propagating in all directions within the dendritic tree. For these neurons, therefore, both the location and timing of synaptic inputs are critical. The time window for which the backpropagating action potential can influence dendritic spike generation has been extensively studied in layer 5 neocortical pyramidal neurons of rat somatosensory cortex. Here, we re-examine this coincidence detection window for pyramidal cell types across the rat somatosensory cortex in layers 2/3, 5 and 6. We find that the time-window for optimal interaction is widest and shifted in layer 5 pyramidal neurons relative to cells in layers 6 and 2/3. Inputs arriving at the same time and locations will therefore differentially affect spike-timing dependent processes in the different classes of pyramidal neurons.

Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.

The Carrington event not observed in most ice core nitrate records

The Carrington Event of 1859 is considered to be among the largest space weather events of the last 150 years. We show that only one out of 14 well-resolved ice core records from Greenland and Antarctica has a nitrate spike dated to 1859. No sharp spikes are observed in the Antarctic cores studied here. In Greenland numerous spikes are observed in the 40 years surrounding 1859, but where other chemistry was measured, all large spikes have the unequivocal signal, including co-located spikes in ammonium, formate, black carbon and vanillic acid, of biomass burning plumes. It seems certain that most spikes in an earlier core, including that claimed for 1859, are also due to biomass burning plumes, and not to solar energetic particle (SEP) events. We conclude that an event as large as the Carrington Event did not leave an observable, widespread imprint in nitrate in polar ice. Nitrate spikes cannot be used to derive the statistics of SEPs.

The GABAB1b isoform mediates long-lasting inhibition of dendritic Ca2+ spikes in layer 5 somatosensory pyramidal neurons

The apical tuft of layer 5 pyramidal neurons is innervated by a large number of inhibitory inputs with unknown functions. Here, we studied the functional consequences and underlying molecular mechanisms of apical inhibition on dendritic spike activity. Extracellular stimulation of layer 1, during blockade of glutamatergic transmission, inhibited the dendritic Ca2+ spike for up to 400 ms. Activation of metabotropic GABAB receptors was responsible for a gradual and long-lasting inhibitory effect, whereas GABAA receptors mediated a short-lasting (approximately 150 ms) inhibition. Our results suggest that the mechanism underlying the GABAB inhibition of Ca2+ spikes involves direct blockade of dendritic Ca2+ channels. By using knockout mice for the two predominant GABAB1 isoforms, GABAB1a and GABAB1b, we showed that postsynaptic inhibition of Ca2+ spikes is mediated by GABAB1b, whereas presynaptic inhibition of GABA release is mediated by GABAB1a. We conclude that the molecular subtypes of GABAB receptors play strategically different physiological roles in neocortical neurons.

Multiple time scales of temporal response in pyramidal and fast spiking cortical neurons

Neural dynamic processes correlated over several time scales are found in vivo, in stimulus-evoked as well as spontaneous activity, and are thought to affect the way sensory stimulation is processed. Despite their potential computational consequences, a systematic description of the presence of multiple time scales in single cortical neurons is lacking. In this study, we injected fast spiking and pyramidal (PYR) neurons in vitro with long-lasting episodes of step-like and noisy, in-vivo-like current. Several processes shaped the time course of the instantaneous spike frequency, which could be reduced to a small number (1-4) of phenomenological mechanisms, either reducing (adapting) or increasing (facilitating) the neuron's firing rate over time. The different adaptation/facilitation processes cover a wide range of time scales, ranging from initial adaptation (<10 ms, PYR neurons only), to fast adaptation (<300 ms), early facilitation (0.5-1 s, PYR only), and slow (or late) adaptation (order of seconds). These processes are characterized by broad distributions of their magnitudes and time constants across cells, showing that multiple time scales are at play in cortical neurons, even in response to stationary stimuli and in the presence of input fluctuations. These processes might be part of a cascade of processes responsible for the power-law behavior of adaptation observed in several preparations, and may have far-reaching computational consequences that have been recently described.

Eluding oblivion with smart stochastic selection of synaptic updates

The variables involved in the equations that describe realistic synaptic dynamics always vary in a limited range. Their boundedness makes the synapses forgetful, not for the mere passage of time, but because new experiences overwrite old memories. The forgetting rate depends on how many synapses are modified by each new experience: many changes means fast learning and fast forgetting, whereas few changes means slow learning and long memory retention. Reducing the average number of modified synapses can extend the memory span at the price of a reduced amount of information stored when a new experience is memorized. Every trick which allows to slow down the learning process in a smart way can improve the memory performance. We review some of the tricks that allow to elude fast forgetting (oblivion). They are based on the stochastic selection of the synapses whose modifications are actually consolidated following each new experience. In practice only a randomly selected, small fraction of the synapses eligible for an update are actually modified. This allows to acquire the amount of information necessary to retrieve the memory without compromising the retention of old experiences. The fraction of modified synapses can be further reduced in a smart way by changing synapses only when it is really necessary, i.e. when the post-synaptic neuron does not respond as desired. Finally we show that such a stochastic selection emerges naturally from spike driven synaptic dynamics which read noisy pre and post-synaptic neural activities. These activities can actually be generated by a chaotic system.

A multivariate approach for processing magnetization effects in triggered event-related functional magnetic resonance imaging time series

Triggered event-related functional magnetic resonance imaging requires sparse intervals of temporally resolved functional data acquisitions, whose initiation corresponds to the occurrence of an event, typically an epileptic spike in the electroencephalographic trace. However, conventional fMRI time series are greatly affected by non-steady-state magnetization effects, which obscure initial blood oxygen level-dependent (BOLD) signals. Here, conventional echo-planar imaging and a post-processing solution based on principal component analysis were employed to remove the dominant eigenimages of the time series, to filter out the global signal changes induced by magnetization decay and to recover BOLD signals starting with the first functional volume. This approach was compared with a physical solution using radiofrequency preparation, which nullifies magnetization effects. As an application of the method, the detectability of the initial transient BOLD response in the auditory cortex, which is elicited by the onset of acoustic scanner noise, was used to demonstrate that post-processing-based removal of magnetization effects allows to detect brain activity patterns identical with those obtained using the radiofrequency preparation. Using the auditory responses as an ideal experimental model of triggered brain activity, our results suggest that reducing the initial magnetization effects by removing a few principal components from fMRI data may be potentially useful in the analysis of triggered event-related echo-planar time series. The implications of this study are discussed with special caution to remaining technical limitations and the additional neurophysiological issues of the triggered acquisition.

Riluzole-induced oscillations in spinal networks

We previously showed in dissociated cultures of fetal rat spinal cord that disinhibition-induced bursting is based on intrinsic spiking, network recruitment, and a network refractory period after the bursts. A persistent sodium current (I(NaP)) underlies intrinsic spiking, which, by recurrent excitation, generates the bursting activity. Although full blockade of I(NaP) with riluzole disrupts such bursting, the present study shows that partial blockade of I(NaP) with low doses of riluzole maintains bursting activity with unchanged burst rate and burst duration. More important, low doses of riluzole turned bursts composed of persistent activity into bursts composed of oscillatory activity at around 5 Hz. In a search for the mechanisms underlying the generation of such intraburst oscillations, we found that activity-dependent synaptic depression was not changed with low doses of riluzole. On the other hand, low doses of riluzole strongly increased spike-frequency adaptation and led to early depolarization block when bursts were simulated by injecting long current pulses into single neurons in the absence of fast synaptic transmission. Phenytoin is another I(NaP) blocker. When applied in doses that reduced intrinsic activity by 80-90%, as did low doses of riluzole, it had no effect either on spike-frequency adaptation or on depolarization block. Nor did phenytoin induce intraburst oscillations after disinhibition. A theoretical model incorporating a depolarization block mechanism could reproduce the generation of intraburst oscillations at the network level. From these findings we conclude that riluzole-induced intraburst oscillations are a network-driven phenomenon whose major accommodation mechanism is depolarization block arising from strong sodium channel inactivation.

BOLD correlates of continuously fluctuating epileptic activity isolated by independent component analysis

Combined EEG/fMRI recordings offer a promising opportunity to detect brain areas with altered BOLD signal during interictal epileptic discharges (IEDs). These areas are likely to represent the irritative zone, which is itself a reflection of the epileptogenic zone. This paper reports on the imaging findings using independent component analysis (ICA) to continuously quantify epileptiform activity in simultaneously acquired EEG and fMRI. Using ICA derived factors coding for the epileptic activity takes into account that epileptic activity is continuously fluctuating with each spike differing in amplitude, duration and maybe topography, including subthreshold epileptic activity besides clear IEDs and may thus increase the sensitivity and statistical power of combined EEG/fMRI in epilepsy. Twenty patients with different types of focal and generalized epilepsy syndromes were investigated. ICA separated epileptiform activity from normal physiological brain activity and artifacts. In 16/20 patients, BOLD correlates of epileptic activity matched the EEG sources, the clinical semiology, and, if present, the structural lesions. In clinically equivocal cases, the BOLD correlates aided to attribute proper diagnosis of the underlying epilepsy syndrome. Furthermore, in one patient with temporal lobe epilepsy, BOLD correlates of rhythmic delta activity could be employed to delineate the affected hippocampus. Compared to BOLD correlates of manually identified IEDs, the sensitivity was improved from 50% (10/20) to 80%. The ICA EEG/fMRI approach is a safe, non-invasive and easily applicable technique, which can be used to identify regions with altered hemodynamic effects related to IEDs as well as intermittent rhythmic discharges in different types of epilepsy.

Steam pops during irrigated radiofrequency ablation: feasibility of impedance monitoring for prevention

BACKGROUND: Steam pops are a risk of irrigated radiofrequency catheter ablation (RFA) and may cause cardiac perforation. Data to guide radiofrequency (RF) energy titration to avoid steam pops are limited. OBJECTIVE: This study sought to assess the frequency and consequence of audible pops and to determine the feasibility of using the magnitude of impedance change to predict pops. METHODS: We reviewed consecutive endocardial open-irrigated RFA for ventricular tachycardia (VT) with continuously recorded ablation data in 142 patients with structural heart disease. Steam pops were defined as an audible pop associated with a sudden spike in impedance. Ablation lesions before or after pops served as controls. RESULTS: From a total of 4,107 ablation lesions, 62 (1.5%) steam pops occurred in 42 procedures in 38 patients. Perforation with tamponade occurred with 1 of 62 (2%) pops. Applications with pops had a greater impedance decrease (22 +/- 7 Omega vs. 18 +/- 8 Omega, P = .001) and a higher maximum power (45 +/- 5 W vs. 43 +/- 6 W, P = .011), but did not differ in maximum catheter tip temperature (40 degrees C +/- 4 degrees C vs. 40 degrees C +/- 4 degrees C, P = .180) from applications without pops. Eighty percent of pops occurred after impedance decreased by at least 18 Omega. CONCLUSION: During VT ablation with open irrigation, audible pops are infrequent and do not usually cause perforation. Limiting RF power to achieve an impedance decrease of <18 Omega is a feasible method of reducing the likelihood of a pop when perforation risk is of concern.

Concurrent monitoring of cerebral electrophysiology and metabolism after traumatic brain injury: an experimental and clinical study

Multiparameter cerebral monitoring has been widely applied in traumatic brain injury to study posttraumatic pathophysiology and to manage head-injured patients (e.g., combining O(2) and pH sensors with cerebral microdialysis). Because a comprehensive approach towards understanding injury processes will also require functional measures, we have added electrophysiology to these monitoring modalities by attaching a recording electrode to the microdialysis probe. These dual-function (microdialysis/electrophysiology) probes were placed in rats following experimental fluid percussion brain injuries, and in a series of severely head-injured human patients. Electrical activity (cell firing, EEG) was monitored concurrently with microdialysis sampling of extracellular glutamate, glucose and lactate. Electrophysiological parameters (firing rate, serial correlation, field potential occurrences) were analyzed offline and compared to dialysate concentrations. In rats, these probes demonstrated an injury-induced suppression of neuronal firing (from a control level of 2.87 to 0.41 spikes/sec postinjury), which was associated with increases in extracellular glutamate and lactate, and decreases in glucose levels. When placed in human patients, the probes detected sparse and slowly firing cells (mean = 0.21 spike/sec), with most units (70%) exhibiting a lack of serial correlation in the spike train. In some patients, spontaneous field potentials were observed, suggesting synchronously firing neuronal populations. In both the experimental and clinical application, the addition of the recording electrode did not appreciably affect the performance of the microdialysis probe. The results suggest that this technique provides a functional monitoring capability which cannot be obtained when electrophysiology is measured with surface or epidural EEG alone.

Cortical local and long-range synchronization interplay in human absence seizure initiation

Brain activity relies on transient, fluctuating interactions between segregated neuronal populations. Synchronization within a single and between distributed neuronal clusters reflects the dynamics of these cooperative patterns. Thus absence epilepsy can be used as a model for integrated, large-scale investigation of the emergence of pathological collective dynamics in the brain. Indeed, spike-wave discharges (SWD) of an absence seizure are thought to reflect abnormal cortical hypersynchronization. In this paper, we address two questions: how and where do SWD arise in the human brain? Therefore, we explored the spatio-temporal dynamics of interactions within and between widely distributed cortical sites using magneto-encephalographic recordings of spontaneous absence seizures. We then extracted, from their time-frequency analysis, local synchronization of cortical sources and long-range synchronization linking distant sites. Our analyses revealed a reproducible sequence of 1) long-range desynchronization, 2) increased local synchronization and 3) increased long-range synchronization. Although both local and long-range synchronization displayed different spatio-temporal profiles, their cortical projection within an initiation time window overlap and reveal a multifocal fronto-central network. These observations contradict the classical view of sudden generalized synchronous activities in absence epilepsy. Furthermore, they suggest that brain states transition may rely on multi-scale processes involving both local and distant interactions.