Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

Correcting mortality for loss to follow-up: a nomogram applied to antiretroviral treatment programmes in sub-Saharan Africa

Background The World Health Organization estimates that in sub-Saharan Africa about 4 million HIV-infected patients had started antiretroviral therapy (ART) by the end of 2008. Loss of patients to follow-up and care is an important problem for treatment programmes in this region. As mortality is high in these patients compared to patients remaining in care, ART programmes with high rates of loss to follow-up may substantially underestimate mortality of all patients starting ART. Methods and Findings We developed a nomogram to correct mortality estimates for loss to follow-up, based on the fact that mortality of all patients starting ART in a treatment programme is a weighted average of mortality among patients lost to follow-up and patients remaining in care. The nomogram gives a correction factor based on the percentage of patients lost to follow-up at a given point in time, and the estimated ratio of mortality between patients lost and not lost to follow-up. The mortality observed among patients retained in care is then multiplied by the correction factor to obtain an estimate of programme-level mortality that takes all deaths into account. A web calculator directly calculates the corrected, programme-level mortality with 95% confidence intervals (CIs). We applied the method to 11 ART programmes in sub-Saharan Africa. Patients retained in care had a mortality at 1 year of 1.4% to 12.0%; loss to follow-up ranged from 2.8% to 28.7%; and the correction factor from 1.2 to 8.0. The absolute difference between uncorrected and corrected mortality at 1 year ranged from 1.6% to 9.8%, and was above 5% in four programmes. The largest difference in mortality was in a programme with 28.7% of patients lost to follow-up at 1 year. Conclusions The amount of bias in mortality estimates can be large in ART programmes with substantial loss to follow-up. Programmes should routinely report mortality among patients retained in care and the proportion of patients lost. A simple nomogram can then be used to estimate mortality among all patients who started ART, for a range of plausible mortality rates among patients lost to follow-up.

Intravaginal practices, bacterial vaginosis, and HIV infection in women: individual participant data meta-analysis

Background Identifying modifiable factors that increase women's vulnerability to HIV is a critical step in developing effective female-initiated prevention interventions. The primary objective of this study was to pool individual participant data from prospective longitudinal studies to investigate the association between intravaginal practices and acquisition of HIV infection among women in sub-Saharan Africa. Secondary objectives were to investigate associations between intravaginal practices and disrupted vaginal flora; and between disrupted vaginal flora and HIV acquisition. Methods and Findings We conducted a meta-analysis of individual participant data from 13 prospective cohort studies involving 14,874 women, of whom 791 acquired HIV infection during 21,218 woman years of follow-up. Data were pooled using random-effects meta-analysis. The level of between-study heterogeneity was low in all analyses (I2 values 0.0%–16.1%). Intravaginal use of cloth or paper (pooled adjusted hazard ratio [aHR] 1.47, 95% confidence interval [CI] 1.18–1.83), insertion of products to dry or tighten the vagina (aHR 1.31, 95% CI 1.00–1.71), and intravaginal cleaning with soap (aHR 1.24, 95% CI 1.01–1.53) remained associated with HIV acquisition after controlling for age, marital status, and number of sex partners in the past 3 months. Intravaginal cleaning with soap was also associated with the development of intermediate vaginal flora and bacterial vaginosis in women with normal vaginal flora at baseline (pooled adjusted odds ratio [OR] 1.24, 95% CI 1.04–1.47). Use of cloth or paper was not associated with the development of disrupted vaginal flora. Intermediate vaginal flora and bacterial vaginosis were each associated with HIV acquisition in multivariable models when measured at baseline (aHR 1.54 and 1.69, p<0.001) or at the visit before the estimated date of HIV infection (aHR 1.41 and 1.53, p<0.001), respectively. Conclusions This study provides evidence to suggest that some intravaginal practices increase the risk of HIV acquisition but a direct causal pathway linking intravaginal cleaning with soap, disruption of vaginal flora, and HIV acquisition has not yet been demonstrated. More consistency in the definition and measurement of specific intravaginal practices is warranted so that the effects of specific intravaginal practices and products can be further elucidated.

Mechanisms of osteoclastogenesis inhibition by a novel class of biphenyl-type cannabinoid CB(2) receptor inverse agonists

The cannabinoid CB(2) receptor is known to modulate osteoclast function by poorly understood mechanisms. Here, we report that the natural biphenyl neolignan 4'-O-methylhonokiol (MH) is a CB(2) receptor-selective antiosteoclastogenic lead structure (K(i) < 50 nM). Intriguingly, MH triggers a simultaneous G(i) inverse agonist response and a strong CB(2) receptor-dependent increase in intracellular calcium. The most active inverse agonists from a library of MH derivatives inhibited osteoclastogenesis in RANK ligand-stimulated RAW264.7 cells and primary human macrophages. Moreover, these ligands potently inhibited the osteoclastogenic action of endocannabinoids. Our data show that CB(2) receptor-mediated cAMP formation, but not intracellular calcium, is crucially involved in the regulation of osteoclastogenesis, primarily by inhibiting macrophage chemotaxis and TNF-α expression. MH is an easily accessible CB(2) receptor-selective scaffold that exhibits a novel type of functional heterogeneity.

A research agenda for malaria eradication: basic science and enabling technologies

Today's malaria control efforts are limited by our incomplete understanding of the biology of Plasmodium and of the complex relationships between human populations and the multiple species of mosquito and parasite. Research priorities include the development of in vitro culture systems for the complete life cycle of P. falciparum and P. vivax and the development of an appropriate liver culture system to study hepatic stages. In addition, genetic technologies for the manipulation of Plasmodium need to be improved, the entire parasite metabolome needs to be characterized to identify new druggable targets, and improved information systems for monitoring the changes in epidemiology, pathology, and host-parasite-vector interactions as a result of intensified control need to be established to bridge the gap between bench, preclinical, clinical, and population-based sciences.

Hostile takeover by Plasmodium: reorganization of parasite and host cell membranes during liver stage egress

The protozoan parasite Plasmodium is transmitted by female Anopheles mosquitoes and undergoes obligatory development within a parasitophorous vacuole in hepatocytes before it is released into the bloodstream. The transition to the blood stage was previously shown to involve the packaging of exoerythrocytic merozoites into membrane-surrounded vesicles, called merosomes, which are delivered directly into liver sinusoids. However, it was unclear whether the membrane of these merosomes was derived from the parasite membrane, the parasitophorous vacuole membrane or the host cell membrane. This knowledge is required to determine how phagocytes will be directed against merosomes. Here, we fluorescently label the candidate membranes and use live cell imaging to show that the merosome membrane derives from the host cell membrane. We also demonstrate that proteins in the host cell membrane are lost during merozoite liberation from the parasitophorous vacuole. Immediately after the breakdown of the parasitophorous vacuole membrane, the host cell mitochondria begin to degenerate and protein biosynthesis arrests. The intact host cell plasma membrane surrounding merosomes allows Plasmodium to mask itself from the host immune system and bypass the numerous Kupffer cells on its way into the bloodstream. This represents an effective strategy for evading host defenses before establishing a blood stage infection.

Collapse of telomere homeostasis in hematopoietic cells caused by heterozygous mutations in telomerase genes

Telomerase activity is readily detectable in extracts from human hematopoietic stem and progenitor cells, but appears unable to maintain telomere length with proliferation in vitro and with age in vivo. We performed a detailed study of the telomere length by flow FISH analysis in leukocytes from 835 healthy individuals and 60 individuals with reduced telomerase activity. Healthy individuals showed a broad range in average telomere length in granulocytes and lymphocytes at any given age. The average telomere length declined with age at a rate that differed between age-specific breakpoints and between cell types. Gender differences between leukocyte telomere lengths were observed for all cell subsets studied; interestingly, this trend could already be detected at birth. Heterozygous carriers for mutations in either the telomerase reverse transcriptase (hTERT) or the telomerase RNA template (hTERC) gene displayed striking and comparable telomere length deficits. Further, non-carrier relatives of such heterozygous individuals had somewhat shorter leukocyte telomere lengths than expected; this difference was most profound for granulocytes. Failure to maintain telomere homeostasis as a result of partial telomerase deficiency is thought to trigger cell senescence or cell death, eventually causing tissue failure syndromes. Our data are consistent with these statements and suggest that the likelihood of similar processes occurring in normal individuals increases with age. Our work highlights the essential role of telomerase in the hematopoietic system and supports the notion that telomerase levels in hematopoietic cells, while limiting and unable to prevent overall telomere shortening, are nevertheless crucial to maintain telomere homeostasis with age.

LGI2 truncation causes a remitting focal epilepsy in dogs

One quadrillion synapses are laid in the first two years of postnatal construction of the human brain, which are then pruned until age 10 to 500 trillion synapses composing the final network. Genetic epilepsies are the most common neurological diseases with onset during pruning, affecting 0.5% of 2-10-year-old children, and these epilepsies are often characterized by spontaneous remission. We previously described a remitting epilepsy in the Lagotto romagnolo canine breed. Here, we identify the gene defect and affected neurochemical pathway. We reconstructed a large Lagotto pedigree of around 34 affected animals. Using genome-wide association in 11 discordant sib-pairs from this pedigree, we mapped the disease locus to a 1.7 Mb region of homozygosity in chromosome 3 where we identified a protein-truncating mutation in the Lgi2 gene, a homologue of the human epilepsy gene LGI1. We show that LGI2, like LGI1, is neuronally secreted and acts on metalloproteinase-lacking members of the ADAM family of neuronal receptors, which function in synapse remodeling, and that LGI2 truncation, like LGI1 truncations, prevents secretion and ADAM interaction. The resulting epilepsy onsets at around seven weeks (equivalent to human two years), and remits by four months (human eight years), versus onset after age eight in the majority of human patients with LGI1 mutations. Finally, we show that Lgi2 is expressed highly in the immediate post-natal period until halfway through pruning, unlike Lgi1, which is expressed in the latter part of pruning and beyond. LGI2 acts at least in part through the same ADAM receptors as LGI1, but earlier, ensuring electrical stability (absence of epilepsy) during pruning years, preceding this same function performed by LGI1 in later years. LGI2 should be considered a candidate gene for common remitting childhood epilepsies, and LGI2-to-LGI1 transition for mechanisms of childhood epilepsy remission.

Slc15a4, a gene required for pDC sensing of TLR ligands, is required to control persistent viral infection

Plasmacytoid dendritic cells (pDCs) are the major producers of type I IFN in response to viral infection and have been shown to direct both innate and adaptive immune responses in vitro. However, in vivo evidence for their role in viral infection is lacking. We evaluated the contribution of pDCs to acute and chronic virus infection using the feeble mouse model of pDC functional deficiency. We have previously demonstrated that feeble mice have a defect in TLR ligand sensing. Although pDCs were found to influence early cytokine secretion, they were not required for control of viremia in the acute phase of the infection. However, T cell priming was deficient in the absence of functional pDCs and the virus-specific immune response was hampered. Ultimately, infection persisted in feeble mice. We conclude that pDCs are likely required for efficient T cell priming and subsequent viral clearance. Our data suggest that reduced pDC functionality may lead to chronic infection.

Identification of the bovine Arachnomelia mutation by massively parallel sequencing implicates sulfite oxidase (SUOX) in bone development

Arachnomelia is a monogenic recessive defect of skeletal development in cattle. The causative mutation was previously mapped to a approximately 7 Mb interval on chromosome 5. Here we show that array-based sequence capture and massively parallel sequencing technology, combined with the typical family structure in livestock populations, facilitates the identification of the causative mutation. We re-sequenced the entire critical interval in a healthy partially inbred cow carrying one copy of the critical chromosome segment in its ancestral state and one copy of the same segment with the arachnomelia mutation, and we detected a single heterozygous position. The genetic makeup of several partially inbred cattle provides extremely strong support for the causality of this mutation. The mutation represents a single base insertion leading to a premature stop codon in the coding sequence of the SUOX gene and is perfectly associated with the arachnomelia phenotype. Our findings suggest an important role for sulfite oxidase in bone development.

Capsule type of Streptococcus pneumoniae determines growth phenotype

The polysaccharide capsule of Streptococcus pneumoniae defines over ninety serotypes, which differ in their carriage prevalence and invasiveness for poorly understood reasons. Recently, an inverse correlation between carriage prevalence and oligosaccharide structure of a given capsule has been described. Our previous work suggested a link between serotype and growth in vitro. Here we investigate whether capsule production interferes with growth in vitro and whether this predicts carriage prevalence in vivo. Eighty-one capsule switch mutants were constructed representing nine different serotypes, five of low (4, 7F, 14, 15, 18C) and four of high carriage prevalence (6B, 9V, 19F, 23F). Growth (length of lag phase, maximum optical density) of wildtype strains, nontypeable mutants and capsule switch mutants was studied in nutrient-restricted Lacks medium (MLM) and in rich undefined brain heart infusion broth supplemented with 5% foetal calf serum (BHI+FCS). In MLM growth phenotype depended on, and was transferred with, capsule operon type. Colonization efficiency of mouse nasopharynx also depended on, and was transferred with, capsule operon type. Capsule production interfered with growth, which correlated inversely with serotype-specific carriage prevalence. Serotypes with better growth and higher carriage prevalence produced thicker capsules (by electron microscopy, FITC-dextran exclusion assays and HPLC) than serotypes with delayed growth and low carriage prevalence. However, expression of cpsA, the first capsule gene, (by quantitative RT-PCR) correlated inversely with capsule thickness. Energy spent for capsule production (incorporation of H3-glucose) relative to amount of capsule produced was higher for serotypes with low carriage prevalence. Experiments in BHI+FCS showed overall better bacterial growth and more capsule production than growth in MLM and differences between serotypes were no longer apparent. Production of polysaccharide capsule in S. pneumoniae interferes with growth in nutrient-limiting conditions probably by competition for energy against the central metabolism. Serotype-specific nasopharyngeal carriage prevalence in vivo is predicted by the growth phenotype.

TGF-b2 induction regulates invasiveness of Theileria-transformed leukocytes and disease susceptibility

Theileria parasites invade and transform bovine leukocytes causing either East Coast fever (T. parva), or tropical theileriosis (T. annulata). Susceptible animals usually die within weeks of infection, but indigenous infected cattle show markedly reduced pathology, suggesting that host genetic factors may cause disease susceptibility. Attenuated live vaccines are widely used to control tropical theileriosis and attenuation is associated with reduced invasiveness of infected macrophages in vitro. Disease pathogenesis is therefore linked to aggressive invasiveness, rather than uncontrolled proliferation of Theileria-infected leukocytes. We show that the invasive potential of Theileria-transformed leukocytes involves TGF-b signalling. Attenuated live vaccine lines express reduced TGF-b2 and their invasiveness can be rescued with exogenous TGF-b. Importantly, infected macrophages from disease susceptible Holstein-Friesian (HF) cows express more TGF-b2 and traverse Matrigel with great efficiency compared to those from disease-resistant Sahiwal cattle. Thus, TGF-b2 levels correlate with disease susceptibility. Using fluorescence and time-lapse video microscopy we show that Theileria-infected, disease-susceptible HF macrophages exhibit increased actin dynamics in their lamellipodia and podosomal adhesion structures and develop more membrane blebs. TGF-b2-associated invasiveness in HF macrophages has a transcription-independent element that relies on cytoskeleton remodelling via activation of Rho kinase (ROCK). We propose that a TGF-b autocrine loop confers an amoeboid-like motility on Theileria-infected leukocytes, which combines with MMP-dependent motility to drive invasiveness and virulence.

Spike-based decision learning of nash equilibria in two-player games

Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.