Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Spatio-temporal credit assignment in population learning

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

Percutaneous and Open Retrograde Endovascular Stenting of Symptomatic High-Grade Innominate Artery Stenosis: Technique and Follow-Up

Angioplasty and stenting of the IA have been reported with high technical and clinical success rates, low complication rates and good mid-term patency rates. Different antegrade or retrograde endovascular catheter-based approaches and combinations with surgical exposure of the CCA are used. The purpose of this study was to determine safety, efficacy and mid-term clinical and radiological outcome of the stent-assisted treatment of atherosclerotic stenotic disease of the IA with special focus on the different technical approaches.

Immediate Search in the IDE as an Example of Socio-Technical Congruence in Search-Driven Development

Search-driven development is mainly concerned with code reuse but also with code navigation and debugging. In this essay we look at search-driven navigation in the IDE. We consider Smalltalk-80 as an example of a programming system with search-driven navigation capabilities and explore its human factors. We present how immediate search results lead to a user experience of code browsing rather than one of waiting for and clicking through search results. We explore the socio-technical congruence of immediate search, ie unification of tasks and breakpoints with method calls, which leads to simpler and more extensible development tools. Eventually we conclude with remarks on the socio-technical congruence of search-driven development.

Experiences in Monitoring and Assessment of Sustainable Land Management

Although sustainable land management (SLM) is widely promoted to prevent and mitigate land degradation and desertification, its monitoring and assessment (M&A) has received much less attention. This paper compiles methodological approaches which to date have been little reported in the literature. It draws lessons from these experiences and identifies common elements and future pathways as a basis for a global approach. The paper starts with local level methods where the World Overview of Conservation Approaches and Technologies (WOCAT) framework catalogues SLM case studies. This tool has been included in the local level assessment of Land Degradation Assessment in Drylands (LADA) and in the EU-DESIRE project. Complementary site-based approaches can enhance an ecological process-based understanding of SLM variation. At national and sub-national levels, a joint WOCAT/LADA/DESIRE spatial assessment based on land use systems identifies the status and trends of degradation and SLM, including causes, drivers and impacts on ecosystem services. Expert consultation is combined with scientific evidence and enhanced where necessary with secondary data and indicator databases. At the global level, the Global Environment Facility (GEF) knowledge from the land (KM:Land) initiative uses indicators to demonstrate impacts of SLM investments. Key lessons learnt include the need for a multi-scale approach, making use of common indicators and a variety of information sources, including scientific data and local knowledge through participatory methods. Methodological consistencies allow cross-scale analyses, and findings are analysed and documented for use by decision-makers at various levels. Effective M&A of SLM [e.g. for United Nations Convention to Combat Desertification (UNCCD)] requires a comprehensive methodological framework agreed by the major players.

Differential effects of dexamethasone on the chondrogenesis of mesenchymal stromal cells: influence of microenvironment, tissue origin and growth factor

Mesenchymal stromal cells (MSCs), which reside within various tissues, are utilized in the engineering of cartilage tissue. Dexamethasone (DEX)--a synthetic glucocorticoid--is almost invariably applied to potentiate the growth-factor-induced chondrogenesis of MSCs in vitro, albeit that this effect has been experimentally demonstrated only for transforming-growth-factor-beta (TGF-β)-stimulated bone-marrow-derived MSCs. Clinically, systemic glucocorticoid therapy is associated with untoward side effects (e.g., bone loss and increased susceptibility to infection). Hence, the use of these agents should be avoided or limited. We hypothesize that the influence of DEX on the chondrogenesis of MSCs depends upon their tissue origin and microenvironment [absence or presence of an extracellular matrix (ECM)], as well as upon the nature of the growth factor. We investigated its effects upon the TGF-β1- and bone-morphogenetic-protein 2 (BMP-2)-induced chondrogenesis of MSCs as a function of tissue source (bone marrow vs. synovium) and microenvironment [cell aggregates (no ECM) vs. explants (presence of a natural ECM)]. In aggregates of bone-marrow-derived MSCs, DEX enhanced TGF-β1-induced chondrogenesis by an up-regulation of cartilaginous genes, but had little influence on the BMP-2-induced response. In aggregates of synovial MSCs, DEX exerted no remarkable effect on either TGF-β1- or BMP-2-induced chondrogenesis. In synovial explants, DEX inhibited BMP-2-induced chondrogenesis almost completely, but had little impact on the TGF-β1-induced response. Our data reveal that steroids are not indispensable for the chondrogenesis of MSCs in vitro. Their influence is context dependent (tissue source of the MSCs, their microenvironment and the nature of the growth-factor). This finding has important implications for MSC based approaches to cartilage repair.

[Osteoporosis - whom to treat? The importance of FRAX® in Switzerland]

The WHO fracture risk assessment tool FRAX® is a computer based algorithm that provides models for the assessment of fracture probability in men and women. The approach uses easily obtained clinical risk factors (CRFs) to estimate 10-year probability of a major osteoporotic fracture (hip, clinical spine, humerus or wrist fracture) and the 10-year probability of a hip fracture. The estimate can be used alone or with femoral neck bone mineral density (BMD) to enhance fracture risk prediction. FRAX® is the only risk engine which takes into account the hazard of death as well as that of fracture. Probability of fracture is calculated in men and women from age, body mass index, and dichotomized variables that comprise a prior fragility fracture, parental history of hip fracture, current tobacco smoking, ever long-term use of oral glucocorticoids, rheumatoid arthritis, other causes of secondary osteoporosis, daily alcohol consumption of 3 or more units daily. The relationship between risk factors and fracture probability was constructed using information of nine population-based cohorts from around the world. CRFs for fracture had been identified that provided independent information on fracture risk based on a series of meta-analyses. The FRAX® algorithm was validated in 11 independent cohorts with in excess of 1 million patient-years, including the Swiss SEMOF cohort. Since fracture risk varies markedly in different regions of the world, FRAX® models need to be calibrated to those countries where the epidemiology of fracture and death is known. Models are currently available for 31 countries across the world. The Swiss-specific FRAX® model was developed very soon after the first release of FRAX® in 2008 and was published in 2009, using Swiss epidemiological data, integrating fracture risk and death hazard of our country. Two FRAX®-based approaches may be used to explore intervention thresholds. They have recently been investigated in the Swiss setting. In the first approach the guideline that individuals with a fracture probability equal to or exceeding that of women with a prior fragility fracture should be considered for treatment is translated into thresholds using 10-year fracture probabilities. In that case the threshold is age-dependent and increases from 16 % at the age of 60 ys to 40 % at the age of 80 ys. The second approach is a cost-effectiveness approach. Using a FRAX®-based intervention threshold of 15 % for both, women and men 50 years and older, should permit cost-effective access to therapy to patients at high fracture probability in our country and thereby contribute to further reduce the growing burden of osteoporotic fractures.

Redundancy and recombination in the Echinococcus AgB multigene family: is there any similarity with protozoan contingency genes?

Numerous genetic variants of the Echinococcus antigen B (AgB) are encountered within a single metacestode. This could be a reflection of gene redundancy or the result of a somatic hypermutation process. We evaluate the complexity of the AgB multigene family by characterizing the upstream promoter regions of the 4 already known genes (EgAgB1-EgAgB4) and evaluating their redundancy in the genome of 3 Echinococcus species (E. granulosus, E. ortleppi and E. multilocularis) using PCR-based approaches. We have ascertained that the number of AgB gene copies is quite variable, both within and between species. The most repetitive gene seems to be AgB3, of which there are more than 110 copies in E. ortleppi. For E. granulosus, we have cloned and characterized 10 distinct upstream promoter regions of AgB3 from a single metacestode. Our sequences suggest that AgB1 and AgB3 are involved in gene conversion. These results are discussed in light of the role of gene redundancy and recombination in parasite evasion mechanisms of host immunity, which at present are known for protozoan organisms, but virtually unknown for multicellular parasites.

Flattening Traits

Traits are fine-grained components that can be used to compose classes, while avoiding many of the problems of multiple inheritance and mixin-based approaches. Since most implementations of traits have focused on dynamically-typed languages, the question naturally arises, how can one best introduce traits to statically-typed languages, like Java and C#?

Cellular and immunological basis of the host-parasite relationship during infection with Neospora caninum

Neospora caninum is an apicomplexan parasite that is closely related to Toxoplasma gondii, the causative agent of toxoplasmosis in humans and domestic animals. However, in contrast to T. gondii, N. caninum represents a major cause of abortion in cattle, pointing towards distinct differences in the biology of these two species. There are 3 distinct key features that represent potential targets for prevention of infection or intervention against disease caused by N. caninum. Firstly, tachyzoites are capable of infecting a large variety of host cells in vitro and in vivo. Secondly, the parasite exploits its ability to respond to alterations in living conditions by converting into another stage (tachyzoite-to-bradyzoite or vice versa). Thirdly, by analogy with T. gondii, this parasite has evolved mechanisms that modulate its host cells according to its own requirements, and these must, especially in the case of the bradyzoite stage, involve mechanisms that ensure long-term survival of not only the parasite but also of the host cell. In order to elucidate the molecular and cellular bases of these important features of N. caninum, cell culture-based approaches and laboratory animal models are being exploited. In this review, we will summarize the current achievements related to host cell and parasite cell biology, and will discuss potential applications for prevention of infection and/or disease by reviewing corresponding work performed in murine laboratory infection models and in cattle.

Opportunistic and systematic screening for chlamydia: a study of consultations by young adults in general practice

BACKGROUND: Opportunistic screening for genital chlamydia infection is being introduced in England, but evidence for the effectiveness of this approach is lacking. There are insufficient data about young peoples' use of primary care services to determine the potential coverage of opportunistic screening in comparison with a systematic population-based approach. AIM: To estimate use of primary care services by young men and women; to compare potential coverage of opportunistic chlamydia screening with a systematic postal approach. DESIGN OF STUDY: Population based cross-sectional study. SETTING: Twenty-seven general practices around Bristol and Birmingham. METHOD: A random sample of patients aged 16-24 years were posted a chlamydia screening pack. We collected details of face-to-face consultations from general practice records. Survival and person-time methods were used to estimate the cumulative probability of attending general practice in 1 year and the coverage achieved by opportunistic and systematic postal chlamydia screening. RESULTS: Of 12 973 eligible patients, an estimated 60.4% (95% confidence interval [CI] = 58.3 to 62.5%) of men and 75.3% (73.7 to 76.9%) of women aged 16-24 years attended their practice at least once in a 1-year period. During this period, an estimated 21.3% of patients would not attend their general practice but would be reached by postal screening, 9.2% would not receive a postal invitation but would attend their practice, and 11.8% would be missed by both methods. CONCLUSIONS: Opportunistic and population-based approaches to chlamydia screening would both fail to contact a substantial minority of the target group, if used alone. A pragmatic approach combining both strategies might achieve higher coverage.

Robust postmortem survival of murine vestibular and cochlear stem cells

Potential treatment strategies of neurodegenerative and other diseases with stem cells derived from nonembryonic tissues are much less subjected to ethical criticism than embryonic stem cell-based approaches. Here we report the isolation of inner ear stem cells, which may be useful in cell replacement therapies for hearing loss, after protracted postmortem intervals. We found that neonatal murine inner ear tissues, including vestibular and cochlear sensory epithelia, display remarkably robust cellular survival, even 10 days postmortem. Similarly, isolation of sphere-forming stem cells was possible up to 10 days postmortem. We detected no difference in the proliferation and differentiation potential between stem cells isolated directly after death and up to 5 days postmortem. At longer postmortem intervals, we observed that the potency of sphere-derived cells to spontaneously differentiate into mature cell types diminishes prior to the cells losing their potential for self-renewal. Three-week-old mice also displayed sphere-forming stem cells in all inner ear tissues investigated up to 5 days postmortem. In summary, our results demonstrate that postmortem murine inner ear tissue is suited for isolation of stem cells.

Surgical suturing of articular cartilage induces osteoarthritis-like changes

INTRODUCTION: In clinical tissue-engineering-based approaches to articular cartilage repair, various types of flap are frequently used to retain an implanted construct within the defect, and they are usually affixed by suturing. We hypothesize that the suturing of articular cartilage is associated with a loss of chondrocytes from, and osteoarthritis-like changes within, the perisutural area. MATERIALS AND METHODS: We established a large, partial-thickness defect model in the femoral groove of adult goats. The defects were filled with bovine fibrinogen to support a devitalized flap of autologous synovial tissue, which was sutured to the surrounding articular cartilage with single, interrupted stitches. The perisutural and control regions were analyzed histologically, histochemically and histomorphometrically shortly after surgery and 3 weeks later. RESULTS: Compared to control regions, chondrocytes were lost from the perisutural area even during the first few hours of surgery. During the ensuing 3 weeks, the numerical density of cells in the perisutural area decreased significantly. The cell losses were associated with a loss of proteoglycans from the extracellular matrix. Shortly after surgery, fissures were observed within the walls of the suture channels. By the third week, their surface density had increased significantly and they were filled with avascular mesenchymal tissue. CONCLUSIONS: The suturing of articular cartilage induces severe local damage, which is progressive and reminiscent of that associated with the early stages of osteoarthritis. This damage could be most readily circumvented by adopting an alternative mode of flap affixation, such as gluing with a biological adhesive.