Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

The increase of methicillin-resistant Staphylococcus aureus (MRSA) and the presence of an unusual sequence type ST49 in slaughter pigs in Switzerland

Background: In years past, methicillin-resistant S. aureus (MRSA) has been frequently detected in pigs in Europe, North America and Asia. Recent, yet sporadic studies have revealed a low occurrence of MRSA in Switzerland. In 2009, a monitoring survey of the prevalence and genetic diversity of methicillin-resistant S. aureus (MRSA) in slaughter pigs in Switzerland was conducted using methods recommended by the EU guidelines, and using a sampling strategy evenly distributed throughout the year and representative of the Swiss slaughter pig population. Monitoring should determine if the overall prevalence of MRSA in the entire country is increasing over the years and if specific multi-resistant MRSA clones are spreading over the country.;Results: In 2009, the nasal cavities of eight out of 405 randomly selected pigs were positive for MRSA, representing a prevalence of 2.0% (95% CI 0.9-3.9). The following year, 23 out of 392 pigs were positive for MRSA [5.9% prevalence (95% CI 3.8-8.7)]. Three multilocus sequence types (ST), four spa types and two types of staphylococcal cassette chromosome mec (SCCmec) elements were detected. The most frequent genotypes were ST398 (MLST)-(spa)t034-V (SCCmec) (n = 18) and ST49-t208-V (n = 7), followed by ST398-t011-V (n = 4), ST398-t1451-V (n = 1), and ST1-t2279-IVc (n = 1). The isolates displayed resistance to beta-lactams [mecA, (31/31); blaZ, (19/31)]; tetracycline [tet(M), (31/31); tet(K), (30/31)] (n = 31); macrolides and lincosamides [erm(C) (4/31) or erm(A) (18/31)] (n = 22); tiamulin [vga(A)v (9/31) or unknown mechanism (18/31)] (n = 27); trimethoprim [dfr(G) (18/31); spectinomycin [ant(9)-Ia (19/31) or unknown mechanism (3/31)] (n = 22); streptomycin [str(19/31)]; sulphamethoxazole (7/31) and ciprofloxacin (n = 1) (mechanisms not determined).;Conclusions: This study is the first to describe the presence of MRSA ST49 in slaughter pigs, and to demonstrate a significant and nearly three-fold increase of MRSA prevalence in pigs within two years. The presence of a specific clonal lineage of MRSA from Switzerland suggests that it has been selected in Swiss pig husbandry. Effective hygiene measures should be enhanced within the entire pig production chain to suppress the spread of these pathogens into the community.

Effects of exposure to physical factors on homeopathic preparations as determined by ultraviolet light spectroscopy

Clinical trials have reported statistically significant and clinically relevant effects of homeopathic preparations. We applied ultraviolet (UV) spectroscopy to investigate the physical properties of homeopathic preparations and to contribute to an understanding of the not-yet-identified mode of action. In previous investigations, homeopathic preparations had significantly lower UV light transmissions than controls. The aim of this study was to explore the possible effects of external factors (UV light and temperature) on the homeopathic preparations. Homeopathic centesimal (c) dilutions, 1c to 30c, of copper sulfate (CuSO(4)), decimal dilutions of sulfur (S(8)), 1x to 30x, and controls (succussed potentization medium) were prepared, randomized, and blinded. UV transmission was measured at six different time points after preparation (from 4 to 256 days). In addition, one series of samples was exposed to UV light of a sterilization lamp for 12 h, one was incubated at 37 degrees Celsius for 24 h, and one was heated to 90 degrees Celsius for 15 min. UV light transmission values from 190 or 220 nm to 340 nm were measured several times and averaged. After each exposure, UV transmission of the homeopathic preparations of CuSO(4) was significantly reduced compared to the controls, particularly after heating to 37 degrees Celsius. Overall, the nonexposed CuSO(4) preparations did not show significantly lower UV transmission compared to controls; however, the pooled subgroup of measurements at days 26, 33, and 110 yielded significant differences. UV light transmission for S(8) preparations did not show any differences compared to controls. Our conclusion is that exposure to external factors, incubation at 37 degrees Celsius in particular, increases the difference in light transmission of homeopathic CuSO(4) preparations compared to controls.

Effect of Acute Psychological Stress on Prefrontal GABA Concentration Determined by Proton Magnetic Resonance Spectroscopy

Objective Impaired function of the central gamma-aminobutyric acid (GABA) system, which provides the brain’s major inhibitory pathways, is thought to play an important role in the pathophysiology of anxiety disorders. The effect of acute psychological stress on the human GABA-ergic system is still unknown, however. The purpose of this study was to determine the effect of acute stress on prefrontal GABA levels. Method A recently developed noninvasive magnetic resonance spectroscopy method was used to measure changes in the GABA concentration of the prefrontal cortex in 10 healthy human subjects during a threat-of-shock condition and during a safe condition (two sessions on different days). The main outcome measure was the mean GABA concentration within a 3×3×2-cm3 voxel selected from the medial prefrontal cortex. Results Prefrontal GABA decreased by approximately 18% in the threat-of-shock condition relative to the safe condition. This reduction was specific to GABA, since the concentrations of N-acetyl-aspartate, choline-containing compounds, and glutamate/glutamine levels obtained in the same spectra did not change significantly. Conclusions This result appeared compatible with evidence from preclinical studies in rodents, which showed rapid presynaptic down-regulation of GABA-ergic neurotransmission in response to acute psychological stress. The molecular mechanism and functional significance of this reduced inhibitory effect of acute psychological stress in relation to impaired GABA-ergic function in anxiety disorders merit further investigation.

Changes in cell proliferation, but not in vascularisation are characteristic for human endometrium in different reproductive failures--a pilot study

Reproductive failure, determined as recurrent spontaneous abortions (RSA) or recurrent implantation failure (RIF) in women is not well understood. Several factors, including embryo quality, and cellular and molecular changes in endometrium may contribute to the insufficient feto-maternal interaction resulting in reproductive failure. Prior clinical studies suggest an inadequate endometrial growth and development of the endometrium, leading to a lesser endometrial thickness.

Intestinal glucose absorption but not endogenous glucose production differs between colostrum- and formula-fed neonatal calves

Glucose supply markedly changes during the transition to extrauterine life. In this study, we investigated diet effects on glucose metabolism in neonatal calves. Calves were fed colostrum (C; n = 7) or milk-based formula (F; n = 7) with similar nutrient content up to d 4 of life. Blood plasma samples were taken daily before feeding and 2 h after feeding on d 4 to measure glucose, lactate, nonesterified fatty acids, protein, urea, insulin, glucagon, and cortisol concentrations. On d 2, additional blood samples were taken to measure glucose first-pass uptake (FPU) and turnover by oral [U-(13)C]-glucose and i.v. [6,6-(2)H(2)]-glucose infusion. On d 3, endogenous glucose production and gluconeogenesis were determined by i.v. [U-(13)C]-glucose and oral deuterated water administration after overnight feed deprivation. Liver tissue was obtained 2 h after feeding on d 4 and glycogen concentration and activities and mRNA abundance of gluconeogenic enzymes were measured. Plasma glucose and protein concentrations and hepatic glycogen concentration were higher (P < 0.05), whereas plasma urea, glucagon, and cortisol (d 2) concentrations as well as hepatic pyruvate carboxylase mRNA level and activity were lower (P < 0.05) in group C than in group F. Orally administered [U-(13)C]-glucose in blood was higher (P < 0.05) but FPU tended to be lower (P < 0.1) in group C than in group F. The improved glucose status in group C resulted from enhanced oral glucose absorption. Metabolic and endocrine changes pointed to elevated amino acid degradation in group F, presumably to provide substrates to meet energy requirements and to compensate for impaired oral glucose uptake.

Genetic variations in bile acid homeostasis are not overrepresented in alcoholic cirrhosis compared to patients with heavy alcohol abuse and absent liver disease

Increased serum bile salt levels have been associated to a single-nucleotide polymorphism in the bile salt export pump (BSEP; ABCB11) in several acquired cholestatic liver diseases but there is little evidence in alcoholic liver disease (ALD). Furthermore, a crosstalk between vitamin D and bile acid synthesis has recently been discovered. Whether this crosstalk has an influence on the course of ALD is unclear to date. Our aim was to analyse the role of genetic polymorphisms in BSEP and the vitamin D receptor gene (NR1I1) on the emergence of cirrhosis in patients with ALD. Therefore, 511 alcoholic patients (131 with cirrhosis and 380 without cirrhosis) underwent ABCB11 genotyping (rs2287622). Of these, 321 (131 with cirrhosis and 190 without cirrhosis) were also tested for NR1I1 polymorphisms (bat-haplotype: BsmI rs1544410, ApaI rs7975232 and TaqI rs731236). Frequencies of ABCB11 and NR1I1 genotypes and haplotypes were compared between alcoholic patients with and without cirrhosis and correlated to serum bile salt, bilirubin and aspartate aminotransferase levels in those with cirrhosis. Frequencies of ABCB11 and NR1I1 genotypes and haplotypes did not differ between the two subgroups and no significant association between genotypes/haplotypes and liver function tests could be determined for neither polymorphism. We conclude that ABCB11 and NR1I1 polymorphisms are obviously not associated with development of cirrhosis in patients with ALD.

Marked global reduction in mGluR5 receptor binding in smokers and ex-smokers determined by [11C]ABP688 positron emission tomography

Nicotine addiction is a major public health problem, resulting in primary glutamatergic dysfunction. We measured the glutamate receptor binding in the human brain and provided direct evidence for the abnormal glutamate system in smokers. Because antagonism of the metabotropic glutamate receptor 5 (mGluR5) reduced nicotine self-administration in rats and mice, mGluR5 is suggested to be involved in nicotine addiction. mGluR5 receptor binding specifically to an allosteric site was observed by using positron emission tomography with [(11)C]ABP688. We found a marked global reduction (20.6%; P < 0.0001) in the mGluR5 distribution volume ratio (DVR) in the gray matter of 14 smokers. The most prominent reductions were found in the bilateral medial orbitofrontal cortex. Compared with 14 nonsmokers, 14 ex-smokers had global reductions in the average gray matter mGluR5 DVR (11.5%; P < 0.005), and there was a significant difference in average gray matter mGluR5 DVR between smokers and ex-smokers (9.2%; P < 0.01). Clinical variables reflecting current nicotine consumption, dependence and abstinence were not correlated with mGluR5 DVR. This decrease in mGluR5 receptor binding may be an adaptation to chronic increases in glutamate induced by chronic nicotine administration, and the decreased down-regulation seen in the ex-smokers could be due to incomplete recovery of the receptors, especially because the ex-smokers were abstinent for only 25 wk on average. These results encourage the development and testing of drugs against addiction that directly target the glutamatergic system.

Oral imatinib treatment reduces early fibrogenesis but does not prevent progression in the long term

BACKGROUND/AIMS: Transactivated hepatic stellate cells (HSCs) represent the key source of extra cellular matrix (ECM) in fibrotic liver. Imatinib, a potent inhibitor of the PDGF receptor tyrosine kinase, reduces HSC proliferation and fibrogenesis when treatment is initiated before fibrosis has developed. We tested the antifibrotic potential of imatinib in ongoing liver injury and in established fibrosis. METHODS: BDL-rats were gavage fed with 20 mg/kg/d imatinib either early (days 0-21) or late (days 22-35) after BDL. Untreated BDL-rats served as controls. ECM and activated HSCs were quantified by morphometry. Tissue activity of MMP-2 was determined by gelatin zymography. mRNA expression of TIMP-1 and procollagen alpha1(I) were measured by RT-PCR. Liver tissue concentration of imatinib was measured by tandem mass spectrometry. RESULTS: Early imatinib reduced ECM formation by 30% (P=0.0455) but left numbers of activated HSCs and procollagen I expression unchanged. MMP-2 activity and TIMP-1 expression were reduced by 50%. Late imatinib treatment did not alter histological or molecular markers of fibrogenesis despite high imatinib tissue levels. CONCLUSIONS: The antifibrotic effectiveness of imatinib is limited to the early phase of fibrogenesis. In ongoing liver injury other mediators most likely compensate for the inhibited PDGF effect.

Stimulation induced variability of pulse plethysmography does not discriminate responsiveness to intubation

BACKGROUND: Hypnotic depth but not haemodynamic response to painful stimulation can be measured with various EEG-based anaesthesia monitors. We evaluated the variation of pulse plethysmography amplitude induced by an electrical tetanic stimulus (PPG variation) as a potential measure for analgesia and predictor of haemodynamic responsiveness during general anaesthesia. METHODS: Ninety-five patients, ASA I or II, were randomly assigned to five groups [Group 1: bispectral index (BIS) (range) 40-50, effect site remifentanil concentration 1 ng ml(-1);Group 2: BIS 40-50, remifentanil 2 ng ml(-1); Group 3: BIS 40-50, remifentanil 4 ng ml(-1); Group 4: BIS 25-35, remifentanil 2 ng ml(-1); Group 5: BIS 55-65, remifentanil 2 ng ml(-1)]. A 60 mA tetanic stimulus was applied for 5 s on the ulnar nerve. From the digitized pulse oximeter wave recorded on a laptop computer, linear and non-linear parameters of PPG variation during the 60 s period after stimulation were computed. The haemodynamic response to subsequent orotracheal intubation was recorded. The PPG variation was compared between groups and between responders and non-responders to intubation (anova). Variables independently predicting the response were determined by logistic regression. RESULTS: The probability of a response to tracheal intubation was 0.77, 0.47, 0.05, 0.18 and 0.52 in Groups 1-5, respectively (P<0.03). The PPG variability was significantly higher in responders than in non-responders but it did not improve the prediction of the response to tracheal intubation based on BIS level and effect site remifentanil concentration. CONCLUSION: Tetanic stimulation induced PPG variation does not reflect the analgesic state in a wide clinical range of surgical anaesthesia.

Prostate specific antigen expression does not necessarily correlate with prostate cancer cell growth

PURPOSE: The antiproliferative effects of pharmacological agents used for androgen ablative therapy in prostate cancer, including goserelin, bicalutamide and cyproterone acetate (Fluka Chemie, Buchs, Switzerland), were tested in vitro. It was determined whether they affected prostate specific antigen mRNA and protein expression independent of growth inhibition. MATERIALS AND METHODS: Goserelin, bicalutamide (AstraZeneca, Zug, Switzerland) and cyproterone acetate were added to prostate specific antigen expressing, androgen dependent LNCaP and androgen independent C4-2 cell line (Urocor, Oklahoma City, Oklahoma) cultures. Proliferation was determined by the 3-[4,5-dimethylthiazol-2-yl]-2,5-diphenyltetrazoliumbromide assay (Roche, Mannheim, Germany). Prostate specific antigen mRNA expression was assessed by quantitative real-time polymerase chain reaction. Secreted prostate specific antigen protein levels were quantified by microparticle enzyme-immunoassay. RESULTS: Goserelin inhibited cell growth and prostate specific antigen protein secretion in LNCaP and C4-2 cells. Prostate specific antigen mRNA expression was not decreased. Bicalutamide did not affect cell growth or prostate specific antigen mRNA expression in LNCaP or C4-2 cells, although it significantly decreased prostate specific antigen protein secretion in LNCaP and to a lesser extent in C4-2 cells. Cyproterone acetate decreased the growth of C4-2 but not of LNCaP cells. It did not affect prostate specific antigen mRNA or protein expression in either cell line. CONCLUSIONS: Prostate specific antigen expression does not necessarily correlate with cell growth. Without a substantial effect on cell growth bicalutamide lowers prostate specific antigen synthesis, whereas cyproterone acetate decreases cell growth with no effect on prostate specific antigen secretion. Prostate specific antigen expression may be influenced by growth inhibition but also by altered mRNA and protein levels depending on the agent, its concentration and the cell line evaluated. For interpreting clinical trials prostate specific antigen is not necessarily a surrogate end point marker for a treatment effect on prostate cancer cell growth.

Ondansetron does not reduce the shivering threshold in healthy volunteers

BACKGROUND: Ondansetron, a serotonin-3 receptor antagonist, reduces postoperative shivering. Drugs that reduce shivering usually impair central thermoregulatory control, and may thus be useful for preventing shivering during induction of therapeutic hypothermia. We determined, therefore, whether ondansetron reduces the major autonomic thermoregulatory response thresholds (triggering core temperatures) in humans. METHODS: Control (placebo) and ondansetron infusions at the target plasma concentration of 250 ng ml(-1) were studied in healthy volunteers on two different days. Each day, skin and core temperatures were increased to provoke sweating; then reduced to elicit peripheral vasoconstriction and shivering. We determined the core-temperature sweating, vasoconstriction and shivering thresholds after compensating for changes in mean-skin temperature. Data were analysed using t-tests and presented as means (sds); P<0.05 was taken as significant. RESULTS: Ondensetron plasma concentrations were 278 (57), 234 (55) and 243 (58) ng ml(-1) at the sweating, vasoconstriction and shivering thresholds, respectively; these corresponded to approximately 50 mg of ondansetron which is approximately 10 times the dose used for postoperative nausea and vomiting. Ondansetron did not change the sweating (control 37.4 (0.4) degrees C, ondansetron 37.6 (0.3) degrees C, P=0.16), vasoconstriction (37.0 (0.5) degrees C vs 37.1 (0.3) degrees C; P=0.70), or shivering threshold (36.3 (0.5) degrees C vs 36.3 (0.6) degrees C; P=0.76). No sedation was observed on either study day. CONCLUSIONS: /b>. Ondansetron appears to have little potential for facilitating induction of therapeutic hypothermia.

Aggretin, a heterodimeric C-type lectin from Calloselasma rhodostoma (malayan pit viper), stimulates platelets by binding to alpha 2beta 1 integrin and glycoprotein Ib, activating Syk and phospholipase Cgamma 2, but does not involve the glycoprotein VI/Fc receptor gamma chain collagen receptor

Aggretin, a potent platelet activator, was isolated from Calloselasma rhodostoma venom, and 30-amino acid N-terminal sequences of both subunits were determined. Aggretin belongs to the heterodimeric snake C-type lectin family and is thought to activate platelets by binding to platelet glycoprotein alpha(2)beta(1). We now show that binding to glycoprotein (GP) Ib is also required. Aggretin-induced platelet activation was inhibited by a monoclonal antibody to GPIb as well as by antibodies to alpha(2)beta(1). Binding of both of these platelet receptors to aggretin was confirmed by affinity chromatography. No binding of other major platelet membrane glycoproteins, in particular GPVI, to aggretin was detected. Aggretin also activates platelets from Fc receptor gamma chain (Fcgamma)-deficient mice to a greater extent than those from normal control mice, showing that it does not use the GPVI/Fcgamma pathway. Platelets from Fcgamma-deficient mice expressed fibrinogen receptors normally in response to collagen, although they did not aggregate, indicating that these platelets may partly compensate via other receptors including alpha(2)beta(1) or GPIb for the lack of the Fcgamma pathway. Signaling by aggretin involves a dose-dependent lag phase followed by rapid tyrosine phosphorylation of a number of proteins. Among these are p72(SYK), p125(FAK), and PLCgamma2, whereas, in comparison with collagen and convulxin, the Fcgamma subunit neither is phosphorylated nor coprecipitates with p72(SYK). This supports an independent, GPIb- and integrin-based pathway for activation of p72(SYK) not involving the Fcgamma receptor.

Contractile performance of adult ventricular rat cardiomyocytes is not directly jeopardized by NO/cGMP-dependent induction of pro-apoptotic pathways

The activation of NO/cGMP pathways can induce pro-apoptotic pathways in cardiomyocytes although only a small number of cardiomyocytes fulfill the criteria of apoptosis. The same pathways reduce the contractile performance of cardiomyocytes. In the present study, we tested the hypothesis that exposure of cells to NO/cGMP for 24 h decrease their contractile performance due to an activation of pro-apoptotic pathways. Experiments were performed on freshly isolated and cultured adult ventricular rat cardiomyocytes. Cells were incubated with 8-bromo-cyclo-GMP (100 nmol/L-1 micromol/L), the NO donor SNAP (1 nmol/L-100 micromol/L), or the guanylyl cyclase activator YC-1 (3 micromol/L). Cell shortening, contraction and relaxation velocities, and diastolic cell lengths were determined at beating frequencies of 0.5, 1, and 2 Hz 24 h later. The activation of pro-apoptotic pathways was determined by staining of cardiomyocytes with an antibody directed against active caspase-3 and quantification of the number of apoptotic cells (annexin staining). Caspase-3 activation and an increase in the number of apoptotic cells was observed, but only at the highest concentrations tested (8-bromo-cyclo-GMP: 1-10 mmol/L; SNAP: 1-100 micromol/L). At these concentrations, none of the drugs decreased the mean cell shortening of cardiomyocytes. However, at concentrations lower than those required for induction of apoptotic cell death, the diastolic cell lengths and sarcomere lengths increased but cell shortening decreased. In conclusion, low concentrations of either NO or cGMP cause a desensitization of myofibrils, as indicated by elongated cell shapes, increased sarcomere lengths and reduced load-free cell shortening. High concentrations of NO/cGMP induce caspase-3 activation and increase the number of cells fulfilling the criteria of apoptotic cell death but did not impair cell function. Therefore, induction of apoptotic cell death per se seems not to contribute to the loss of contractile efficiency on the cellular level.