Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Block of the hERG channel by bupivacaine: Electrophysiological and modeling insights towards stereochemical optimization

The hERG voltage-gated potassium channel mediates the cardiac I(Kr) current, which is crucial for the duration of the cardiac action potential. Undesired block of the channel by certain drugs may prolong the QT interval and increase the risk of malignant ventricular arrhythmias. Although the molecular determinants of hERG block have been intensively studied, not much is known about its stereoselectivity. Levo-(S)-bupivacaine was the first drug reported to have a higher affinity to block hERG than its enantiomer. This study strives to understand the principles underlying the stereoselectivity of bupivacaine block with the help of mutagenesis analyses and molecular modeling simulations. Electrophysiological measurements of mutated hERG channels allowed for the identification of residues involved in bupivacaine binding and stereoselectivity. Docking and molecular mechanics simulations for both enantiomers of bupivacaine and terfenadine (a non-stereoselective blocker) were performed inside an open-state model of the hERG channel. The predicted binding modes enabled a clear depiction of ligand-protein interactions. Estimated binding affinities for both enantiomers were consistent with electrophysiological measurements. A similar computational procedure was applied to bupivacaine enantiomers towards two mutated hERG channels (Tyr652Ala and Phe656Ala). This study confirmed, at the molecular level, that bupivacaine stereoselectively binds the hERG channel. These results help to lay the foundation for structural guidelines to optimize the cardiotoxic profile of drug candidates in silico.

Nedd4-2-dependent ubiquitylation and regulation of the cardiac potassium channel hERG1

The voltage-gated cardiac potassium channel hERG1 (human ether-à-gogo-related gene 1) plays a key role in the repolarization phase of the cardiac action potential (AP). Mutations in its gene, KCNH2, can lead to defects in the biosynthesis and maturation of the channel, resulting in congenital long QT syndrome (LQTS). To identify the molecular mechanisms regulating the density of hERG1 channels at the plasma membrane, we investigated channel ubiquitylation by ubiquitin ligase Nedd4-2, a post-translational regulatory mechanism previously linked to other ion channels. We found that whole-cell hERG1 currents recorded in HEK293 cells were decreased upon neural precursor cell expressed developmentally down-regulated 4-2 (Nedd4-2) co-expression. The amount of hERG1 channels in total HEK293 lysates and at the cell surface, as assessed by Western blot and biotinylation assays, respectively, were concomitantly decreased. Nedd4-2 and hERG1 interact via a PY motif located in the C-terminus of hERG1. Finally, we determined that Nedd4-2 mediates ubiquitylation of hERG1 and that deletion of this motif affects Nedd4-2-dependent regulation. These results suggest that ubiquitylation of the hERG1 protein by Nedd4-2, and its subsequent down-regulation, could represent an important mechanism for modulation of the duration of the human cardiac action potential.

Cardiac Sodium Channel Nav1.5 Mechanosensitivity is Inhibited by Ranolazine

The cardiac action potential (AP) is initiated by the depolarizing inward sodium current (I(Na)). The pore-forming subunit of the cardiac sodium channel, Na(v)1.5, is the main ion channel that conducts I(Na) in cardiac cells. Despite the large number of studies investigating Na(v)1.5, year after year, we are still learning new aspects regarding its roles in normal cardiac function and in diseased states. The clinical relevance of this channel cannot be understated. The cardiac I(Na) is the target of the class 1 anti-arrhythmic drugs(1), which are nowadays less frequently prescribed because of their well-documented pro-arrhythmic properties(2). In addition, since the first description in 1995 by Keating's group(3) of mutations in patients suffering from congenital long QT syndrome (LQTS) type 3, several hundred genetic variants in SCN5A, the gene coding for Na(v)1.5, have been reported and investigated(4). Interestingly, many of these genetic variants have been found in patients with diverse cardiac manifestations(5) such as congenital LQTS type 3, Brugada syndrome, conduction disorders, and more recently, atrial fibrillation and dilated cardiomyopathy. This impressive list underlines the importance of Na(v)1.5 in cardiac pathologies and raises the question about possible unknown roles and regulatory mechanisms of this channel in cardiac cells. Recent studies have provided experimental evidence that the function of Na(v)1.5, among many other described regulatory mechanisms(6), is also modulated by the mechanical stretch of the membrane in which it is embedded(7), thus suggesting that Na(v)1.5, like other ion channels, is "mechanosensitive". What does this mean? (SELECT FULL TEXT TO CONTINUE).

Stereoselective Inhibition of the hERG1 Potassium Channel

A growing number of drugs have been shown to prolong cardiac repolarization, predisposing individuals to life-threatening ventricular arrhythmias known as Torsades de Pointes. Most of these drugs are known to interfere with the human ether à-gogo related gene 1 (hERG1) channel, whose current is one of the main determinants of action potential duration. Prolonged repolarization is reflected by lengthening of the QT interval of the electrocardiogram, as seen in the suitably named drug-induced long QT syndrome. Chirality (presence of an asymmetric atom) is a common feature of marketed drugs, which can therefore exist in at least two enantiomers with distinct three-dimensional structures and possibly distinct biological fates. Both the pharmacokinetic and pharmacodynamic properties can differ between enantiomers, as well as also between individuals who take the drug due to metabolic polymorphisms. Despite the large number of reports about drugs reducing the hERG1 current, potential stereoselective contributions have only been scarcely investigated. In this review, we present a non-exhaustive list of clinically important molecules which display chiral toxicity that may be related to hERG1-blocking properties. We particularly focus on methadone cardiotoxicity, which illustrates the importance of the stereoselective effect of drug chirality as well as individual variations resulting from pharmacogenetics. Furthermore, it seems likely that, during drug development, consideration of chirality in lead optimization and systematic assessment of the hERG1 current block with all enantiomers could contribute to the reduction of the risk of drug-induced LQTS.

Uncovering the dynamics of cardiac systems using stochastic pacing and frequency domain analyses

Alternans of cardiac action potential duration (APD) is a well-known arrhythmogenic mechanism which results from dynamical instabilities. The propensity to alternans is classically investigated by examining APD restitution and by deriving APD restitution slopes as predictive markers. However, experiments have shown that such markers are not always accurate for the prediction of alternans. Using a mathematical ventricular cell model known to exhibit unstable dynamics of both membrane potential and Ca2+ cycling, we demonstrate that an accurate marker can be obtained by pacing at cycle lengths (CLs) varying randomly around a basic CL (BCL) and by evaluating the transfer function between the time series of CLs and APDs using an autoregressive-moving-average (ARMA) model. The first pole of this transfer function corresponds to the eigenvalue (λalt) of the dominant eigenmode of the cardiac system, which predicts that alternans occurs when λalt≤−1. For different BCLs, control values of λalt were obtained using eigenmode analysis and compared to the first pole of the transfer function estimated using ARMA model fitting in simulations of random pacing protocols. In all versions of the cell model, this pole provided an accurate estimation of λalt. Furthermore, during slow ramp decreases of BCL or simulated drug application, this approach predicted the onset of alternans by extrapolating the time course of the estimated λalt. In conclusion, stochastic pacing and ARMA model identification represents a novel approach to predict alternans without making any assumptions about its ionic mechanisms. It should therefore be applicable experimentally for any type of myocardial cell.

Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.

Cardiac sodium channel NaV1.5 distribution in myocytes via interacting proteins: the multiple pool model

The cardiac sodium current (INa) is responsible for the rapid depolarization of cardiac cells, thus allowing for their contraction. It is also involved in regulating the duration of the cardiac action potential (AP) and propagation of the impulse throughout the myocardium. Cardiac INa is generated by the voltage-gated Na(+) channel, NaV1.5, a 2016-residue protein which forms the pore of the channel. Over the past years, hundreds of mutations in SCN5A, the human gene coding for NaV1.5, have been linked to many cardiac electrical disorders, including the congenital and acquired long QT syndrome, Brugada syndrome, conduction slowing, sick sinus syndrome, atrial fibrillation, and dilated cardiomyopathy. Similar to many membrane proteins, NaV1.5 has been found to be regulated by several interacting proteins. In some cases, these different proteins, which reside in distinct membrane compartments (i.e. lateral membrane vs. intercalated disks), have been shown to interact with the same regulatory domain of NaV1.5, thus suggesting that several pools of NaV1.5 channels may co-exist in cardiac cells. The aim of this review article is to summarize the recent works that demonstrate its interaction with regulatory proteins and illustrate the model that the sodium channel NaV1.5 resides in distinct and different pools in cardiac cells. This article is part of a Special Issue entitled: Cardiomyocyte Biology: Cardiac Pathways of Differentiation, Metabolism and Contraction.

Dynamic of ion channel expression at the plasma membrane of cardiomyocytes

Cardiac myocytes are characterized by distinct structural and functional entities involved in the generation and transmission of the action potential and the excitation-contraction coupling process. Key to their function is the specific organization of ion channels and transporters to and within distinct membrane domains, which supports the anisotropic propagation of the depolarization wave. This review addresses the current knowledge on the molecular actors regulating the distinct trafficking and targeting mechanisms of ion channels in the highly polarized cardiac myocyte. In addition to ubiquitous mechanisms shared by other excitable cells, cardiac myocytes show unique specialization, illustrated by the molecular organization of myocyte-myocyte contacts, e.g., the intercalated disc and the gap junction. Many factors contribute to the specialization of the cardiac sarcolemma and the functional expression of cardiac ion channels, including various anchoring proteins, motors, small GTPases, membrane lipids, and cholesterol. The discovery of genetic defects in some of these actors, leading to complex cardiac disorders, emphasizes the importance of trafficking and targeting of ion channels to cardiac function. A major challenge in the field is to understand how these and other actors work together in intact myocytes to fine-tune ion channel expression and control cardiac excitability.

Cellular mechanisms of burst firing-mediated long-term depression in rat neocortical pyramidal cells

During wakefulness and sleep, neurons in the neocortex emit action potentials tonically or in rhythmic bursts, respectively. However, the role of synchronized discharge patterns is largely unknown. We have recently shown that pairings of excitatory postsynaptic potentials (EPSPs) and action potential bursts or single spikes lead to long-term depression (burst-LTD) or long-term potentiation, respectively. In this study, we elucidate the cellular mechanisms of burst-LTD and characterize its functional properties. Whole-cell patch-clamp recordings were obtained from layer V pyramidal cells in somatosensory cortex of juvenile rats in vitro and composite EPSPs and EPSCs were evoked extracellularly in layers II/III. Repetitive burst-pairings led to a long-lasting depression of EPSPs and EPSCs that was blocked by inhibitors of metabotropic glutamate group 1 receptors, phospholipase C, protein kinase C (PKC) and calcium release from the endoplasmic reticulum, and that required an intact machinery for endocytosis. Thus, burst-LTD is induced via a Ca2+- and phosphatidylinositol-dependent activation of PKC and expressed through phosphorylation-triggered endocytosis of AMPA receptors. Functionally, burst-LTD is inversely related to EPSP size and bursts dominate single spikes in determining the sign of synaptic plasticity. Thus burst-firing constitutes a signal by which coincident synaptic inputs are proportionally downsized. Overall, our data thus suggest a mechanism by which synaptic weights can be reconfigured during non-rapid eye movement sleep.

Combined voltage and calcium epifluorescence imaging in vitro and in vivo reveals subthreshold and suprathreshold dynamics of mouse barrel cortex

Cortical dynamics can be imaged at high spatiotemporal resolution with voltage-sensitive dyes (VSDs) and calcium-sensitive dyes (CaSDs). We combined these two imaging techniques using epifluorescence optics together with whole cell recordings to measure the spatiotemporal dynamics of activity in the mouse somatosensory barrel cortex in vitro and in the supragranular layers in vivo. The two optical signals reported distinct aspects of cortical function. VSD fluorescence varied linearly with membrane potential and was dominated by subthreshold postsynaptic potentials, whereas the CaSD signal predominantly reflected local action potential firing. Combining VSDs and CaSDs allowed us to monitor the synaptic drive and the spiking activity of a given area at the same time in the same preparation. The spatial extent of the two dye signals was different, with VSD signals spreading further than CaSD signals, reflecting broad subthreshold and narrow suprathreshold receptive fields. Importantly, the signals from the dyes were differentially affected by pharmacological manipulations, stimulation strength, and depth of isoflurane anesthesia. Combined VSD and CaSD measurements can therefore be used to specify the temporal and spatial relationships between subthreshold and suprathreshold activity of the neocortex.

Properties of basal dendrites of layer 5 pyramidal neurons: a direct patch-clamp recording study

Basal dendrites receive the majority of synapses that contact neocortical pyramidal neurons, yet our knowledge of synaptic processing in these dendrites has been hampered by their inaccessibility for electrical recordings. A new approach to patch-clamp recordings enabled us to characterize the integrative properties of these cells. Despite the short physical length of rat basal dendrites, synaptic inputs were electrotonically remote from the soma (>30-fold excitatory postsynaptic potential (EPSP) attenuation) and back-propagating action potentials were significantly attenuated. Unitary EPSPs were location dependent, reaching large amplitudes distally (>8 mV), yet their somatic contribution was relatively location independent. Basal dendrites support sodium and NMDA spikes, but not calcium spikes, for 75% of their length. This suggests that basal dendrites, despite their proximity to the site of action potential initiation, do not form a single basal-somatic region but rather should be considered as a separate integrative compartment favoring two integration modes: subthreshold, location-independent summation versus local amplification of incoming spatiotemporally clustered information.

Dynamic changes of cardiac conduction during rapid pacing

Slow conduction and unidirectional conduction block (UCB) are key mechanisms of reentry. Following abrupt changes in heart rate, dynamic changes of conduction velocity (CV) and structurally determined UCB may critically influence arrhythmogenesis. Using patterned cultures of neonatal rat ventricular myocytes grown on microelectrode arrays, we investigated the dynamics of CV in linear strands and the behavior of UCB in tissue expansions following an abrupt decrease in pacing cycle length (CL). Ionic mechanisms underlying rate-dependent conduction changes were investigated using the Pandit-Clark-Giles-Demir model. In linear strands, CV gradually decreased upon a reduction of CL from 500 ms to 230-300 ms. In contrast, at very short CLs (110-220 ms), CV first decreased before increasing again. The simulations suggested that the initial conduction slowing resulted from gradually increasing action potential duration (APD), decreasing diastolic intervals, and increasing postrepolarization refractoriness, which impaired Na(+) current (I(Na)) recovery. Only at very short CLs did APD subsequently shorten again due to increasing Na(+)/K(+) pump current secondary to intracellular Na(+) accumulation, which caused recovery of CV. Across tissue expansions, the degree of UCB gradually increased at CLs of 250-390 ms, whereas at CLs of 180-240 ms, it first increased and subsequently decreased. In the simulations, reduction of inward currents caused by increasing intracellular Na(+) and Ca(2+) concentrations contributed to UCB progression, which was reversed by increasing Na(+)/K(+) pump activity. In conclusion, CV and UCB follow intricate dynamics upon an abrupt decrease in CL that are determined by the interplay among I(Na) recovery, postrepolarization refractoriness, APD changes, ion accumulation, and Na(+)/K(+) pump function.

Adenoviral expression of IKs contributes to wavebreak and fibrillatory conduction in neonatal rat ventricular cardiomyocyte monolayers

Previous studies have shown that the gating kinetics of the slow component of the delayed rectifier K(+) current (I(Ks)) contribute to postrepolarization refractoriness in isolated cardiomyocytes. However, the impact of such kinetics on arrhythmogenesis remains unknown. We surmised that expression of I(Ks) in rat cardiomyocyte monolayers contributes to wavebreak formation and facilitates fibrillatory conduction by promoting postrepolarization refractoriness. Optical mapping was performed in 44 rat ventricular myocyte monolayers infected with an adenovirus carrying the genomic sequences of KvLQT1 and minK (molecular correlates of I(Ks)) and 41 littermate controls infected with a GFP adenovirus. Repetitive bipolar stimulation was applied at increasing frequencies, starting at 1 Hz until loss of 1:1 capture or initiation of reentry. Action potential duration (APD) was significantly shorter in I(Ks)-infected monolayers than in controls at 1 to 3 Hz (P<0.05), whereas differences at higher pacing frequencies did not reach statistical significance. Stable rotors occurred in both groups, with significantly higher rotation frequencies, lower conduction velocities, and shorter action potentials in the I(Ks) group. Wavelengths in the latter were significantly shorter than in controls at all rotation frequencies. Wavebreaks leading to fibrillatory conduction occurred in 45% of the I(Ks) reentry episodes but in none of the controls. Moreover, the density of wavebreaks increased with time as long as a stable source sustained the fibrillatory activity. These results provide the first demonstration that I(Ks)-mediated postrepolarization refractoriness can promote wavebreak formation and fibrillatory conduction during pacing and sustained reentry and may have important implications in tachyarrhythmias.