Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Spatio-temporal credit assignment in population learning

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

Heavy metal cations permeate the TRPV6 epithelial cation channel

TRPV6 belongs to the vanilloid family of the transient receptor potential channel (TRP) superfamily. This calcium-selective channel is highly expressed in the duodenum and the placenta, being responsible for calcium absorption in the body and fetus. Previous observations have suggested that TRPV6 is not only permeable to calcium but also to other divalent cations in epithelial tissues. In this study, we tested whether TRPV6 is indeed also permeable to cations such as zinc and cadmium. We found that the basal intracellular calcium concentration was higher in HEK293 cells transfected with hTRPV6 than in non-transfected cells, and that this difference almost disappeared in nominally calcium-free solution. Live cell imaging experiments with Fura-2 and NewPort Green DCF showed that overexpression of human TRPV6 increased the permeability for Ca(2+), Ba(2+), Sr(2+), Mn(2+), Zn(2+), Cd(2+), and interestingly also for La(3+) and Gd(3+). These results were confirmed using the patch clamp technique. (45)Ca uptake experiments showed that cadmium, lanthanum and gadolinium were also highly efficient inhibitors of TRPV6-mediated calcium influx at higher micromolar concentrations. Our results suggest that TRPV6 is not only involved in calcium transport but also in the transport of other divalent cations, including heavy metal ions, which may have toxicological implications.

Chemical inhibitors of the calcium entry channel TRPV6

Calcium entry channels in the plasma membrane are thought to play a major role in maintaining cellular Ca(2+) levels, crucial for growth and survival of normal and cancer cells. The calcium-selective channel TRPV6 is expressed in prostate, breast, and other cancer cells. Its expression coincides with cancer progression, suggesting that it drives cancer cell growth. However, no specific inhibitors for TRPV6 have been identified thus far.

Nerve excitability studies characterize Kv1.1 fast potassium channel dysfunction in patients with episodic ataxia type 1

Episodic ataxia type 1 is a neuronal channelopathy caused by mutations in the KCNA1 gene encoding the fast K(+) channel subunit K(v)1.1. Episodic ataxia type 1 presents with brief episodes of cerebellar dysfunction and persistent neuromyotonia and is associated with an increased incidence of epilepsy. In myelinated peripheral nerve, K(v)1.1 is highly expressed in the juxtaparanodal axon, where potassium channels limit the depolarizing afterpotential and the effects of depolarizing currents. Axonal excitability studies were performed on patients with genetically confirmed episodic ataxia type 1 to characterize the effects of K(v)1.1 dysfunction on motor axons in vivo. The median nerve was stimulated at the wrist and compound muscle action potentials were recorded from abductor pollicis brevis. Threshold tracking techniques were used to record strength-duration time constant, threshold electrotonus, current/threshold relationship and the recovery cycle. Recordings from 20 patients from eight kindreds with different KCNA1 point mutations were compared with those from 30 normal controls. All 20 patients had a history of episodic ataxia and 19 had neuromyotonia. All patients had similar, distinctive abnormalities: superexcitability was on average 100% higher in the patients than in controls (P < 0.00001) and, in threshold electrotonus, the increase in excitability due to a depolarizing current (20% of threshold) was 31% higher (P < 0.00001). Using these two parameters, the patients with episodic ataxia type 1 and controls could be clearly separated into two non-overlapping groups. Differences between the different KCNA1 mutations were not statistically significant. Studies of nerve excitability can identify K(v)1.1 dysfunction in patients with episodic ataxia type 1. The simple 15 min test may be useful in diagnosis, since it can differentiate patients with episodic ataxia type 1 from normal controls with high sensitivity and specificity.

In vivo assessment of HCN channel current (I(h)) in human motor axons

The "Trond" protocol of nerve excitability tests has been used widely to assess axonal function in peripheral nerve. In this study, the routine Trond protocol was expanded to refine assessment of cAMP-dependent, hyperpolarization-activated current (I(h)) activity. I(h) activity is generated by hyperpolarization-activated, cyclic nucleotide-modulated (HCN) channels in response to hyperpolarization. It limits activity-dependent hyperpolarization, contributes to neuronal automaticity, and is implicated in chronic pain states. Published data regarding I(h) activity in motor nerve are scant. We used additional strong, prolonged hyperpolarizing conditioning stimuli in the threshold electrotonus component of the Trond protocol to demonstrate the time-course of activation of I(h) in motor axons. Fifteen healthy volunteers were tested on four occasions during 1 week. I(h) action was revealed in the threshold electrotonus by the limiting and often reversal, after about 100 ms, of the threshold increase caused by strong hyperpolarizing currents. Statistical analysis by repeated-measures analysis of variance enabled confidence limits to be established for variation between subjects and within subjects. The results demonstrate that, of all the excitability parameters, those dependent on I(h) were the most characteristic of an individual, because variance between subjects was more than four times the variance within subjects. This study demonstrates a reliable method for in vivo assessment of I(h,) and also serves to document the normal variability in nerve excitability properties within subjects.

Dissonance as a resource for probing the qubit depolarizing channel

Quantum channel identification, a standard problem in quantum metrology, is the task of estimating parameter(s) of a quantum channel. We investigate dissonance (quantum discord in the absence of entanglement) as an aid to quantum channel identification and find evidence for dissonance as a resource for quantum information processing. We consider the specific case of dissonant Bell-diagonal probes of the qubit depolarizing channel, using quantum Fisher information as a measure of statistical information extracted by the probe. In this setting dissonant quantum probes yield more statistical information about the depolarizing probability than do corresponding probes without dissonance and greater dissonance yields greater information. This effect only operates consistently when we control for classical correlation between the probe and its ancilla and the joint and marginal purities of the ancilla and probe.

Block of the hERG channel by bupivacaine: Electrophysiological and modeling insights towards stereochemical optimization

The hERG voltage-gated potassium channel mediates the cardiac I(Kr) current, which is crucial for the duration of the cardiac action potential. Undesired block of the channel by certain drugs may prolong the QT interval and increase the risk of malignant ventricular arrhythmias. Although the molecular determinants of hERG block have been intensively studied, not much is known about its stereoselectivity. Levo-(S)-bupivacaine was the first drug reported to have a higher affinity to block hERG than its enantiomer. This study strives to understand the principles underlying the stereoselectivity of bupivacaine block with the help of mutagenesis analyses and molecular modeling simulations. Electrophysiological measurements of mutated hERG channels allowed for the identification of residues involved in bupivacaine binding and stereoselectivity. Docking and molecular mechanics simulations for both enantiomers of bupivacaine and terfenadine (a non-stereoselective blocker) were performed inside an open-state model of the hERG channel. The predicted binding modes enabled a clear depiction of ligand-protein interactions. Estimated binding affinities for both enantiomers were consistent with electrophysiological measurements. A similar computational procedure was applied to bupivacaine enantiomers towards two mutated hERG channels (Tyr652Ala and Phe656Ala). This study confirmed, at the molecular level, that bupivacaine stereoselectively binds the hERG channel. These results help to lay the foundation for structural guidelines to optimize the cardiotoxic profile of drug candidates in silico.

Patient with syncope and LQTS carrying a mutation in the PAS domain of the hERG1 channel

We report the case of a woman with syncope and persistently prolonged QTc interval. Screening of congenital long QT syndrome (LQTS) genes revealed that she was a heterozygous carrier of a novel KCNH2 mutation, c.G238C. Electrophysiological and biochemical characterizations unveiled the pathogenicity of this new mutation, displaying a 2-fold reduction in protein expression and current density due to a maturation/trafficking-deficient mechanism. The patient's phenotype can be fully explained by this observation. This study illustrates the importance of performing genetic analyses and mutation characterization when there is a suspicion of congenital LQTS. Identifying mutations in the PAS domain or other domains of the hERG1 channel and understanding their effect may provide more focused and mutation-specific risk assessment in this population.

Nedd4-2-dependent ubiquitylation and regulation of the cardiac potassium channel hERG1

The voltage-gated cardiac potassium channel hERG1 (human ether-à-gogo-related gene 1) plays a key role in the repolarization phase of the cardiac action potential (AP). Mutations in its gene, KCNH2, can lead to defects in the biosynthesis and maturation of the channel, resulting in congenital long QT syndrome (LQTS). To identify the molecular mechanisms regulating the density of hERG1 channels at the plasma membrane, we investigated channel ubiquitylation by ubiquitin ligase Nedd4-2, a post-translational regulatory mechanism previously linked to other ion channels. We found that whole-cell hERG1 currents recorded in HEK293 cells were decreased upon neural precursor cell expressed developmentally down-regulated 4-2 (Nedd4-2) co-expression. The amount of hERG1 channels in total HEK293 lysates and at the cell surface, as assessed by Western blot and biotinylation assays, respectively, were concomitantly decreased. Nedd4-2 and hERG1 interact via a PY motif located in the C-terminus of hERG1. Finally, we determined that Nedd4-2 mediates ubiquitylation of hERG1 and that deletion of this motif affects Nedd4-2-dependent regulation. These results suggest that ubiquitylation of the hERG1 protein by Nedd4-2, and its subsequent down-regulation, could represent an important mechanism for modulation of the duration of the human cardiac action potential.

Identification of a novel loss-of-function calcium channel gene mutation in short QT syndrome (SQTS6)

Short QT syndrome (SQTS) is a genetically determined ion-channel disorder, which may cause malignant tachyarrhythmias and sudden cardiac death. Thus far, mutations in five different genes encoding potassium and calcium channel subunits have been reported. We present, for the first time, a novel loss-of-function mutation coding for an L-type calcium channel subunit.

Regulation of the cardiac sodium channel Nav1.5 by utrophin in dystrophin-deficient mice

Duchenne muscular dystrophy (DMD) is a severe striated muscle disease due to the absence of dystrophin. Dystrophin deficiency results in dysfunctional sodium channels and conduction abnormalities in hearts of mdx mice. Disease progression in the mdx mouse only modestly reflects that of DMD patients, possibly due to utrophin up-regulation. Here, we investigated mice deficient in both dystrophin and utrophin [double knockout (DKO)] to assess the role of utrophin in the regulation of the cardiac sodium channel (Na(v)1.5) in mdx mice.

Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.