The relationship between obesity, low back pain, and lumbar disc degeneration when genetics and the environment are considered: a systematic review of twin studies

BACKGROUND CONTEXT: The relationships between obesity and low back pain (LBP) and lumbar disc degeneration (LDD) remain unclear. It is possible that familial factors, including genetics and early environment, affect these relationships.PURPOSE: To investigate the relationship between obesity-related measures (eg, weight, body mass index [BMI]) and LBP and LDD using twin studies, where the effect of genetics and early environment can be controlled.STUDY DESIGN: A systematic review with meta-analysis.METHODS: MEDLINE, CINAHL, Scopus, Web of Science, and EMBASE databases were searched from the earliest records to August 2014. All cross-sectional and longitudinal observational twin studies identified by the search strategy were considered for inclusion. Two investigators independently assessed the eligibility, conducted the quality assessment, and extracted the data. Metaanalyses (fixed or random effects, as appropriate) were used to pool studies'estimates of association.RESULTS: In total, 11 articles met the inclusion criteria. Five studies were included in the LBP analysis and seven in the LDD analysis. For the LBP analysis, pooling of the five studies showed that the risk of having LBP for individuals with the highest levels of BMI or weight was almost twice that of people with a lower BMI (odds ratio [OR] 1.8; 95% confidence interval [CI] 1.6-2.0; I-2 = 0%). A dose-response relationship was also identified. When genetics and the effects of a shared early environment were adjusted for using a within-pair twin case-control analysis, pooling of three studies showed a reduced but statistically positive association between obesity and prevalence of LBP (OR 1.5; 95% CI 1.1-2.1; I-2 = 0%). However, the association was further diminished and not significant (OR 1.4; 95% CI 0.8-2.3; I-2 = 0%) when pooling included two studies on monozygotic twin pairs only. Seven studies met the inclusion criteria for LDD. When familial factors were not controlled for, body weight was positively associated with LDD in all five cross-sectional studies. Only two cross-sectional studies investigated the relationship between obesity-related measures and LDD accounting for familial factors, and the results were conflicting. One longitudinal study in LBP and three longitudinal studies in LDD found no increase in risk in obese individuals, whether or not familial factors were controlled for.CONCLUSIONS: Findings from this review suggest that genetics and early environment are possible mechanisms underlying the relationship between obesity and LBP; however, a direct causal link between these conditions appears to be weak. Further longitudinal studies using the twin design are needed to better understand the complex mechanisms underlying the associations between obesity, LBP, and LDD.

Surgical interventions for treating pectus excavatum

Background: Pectus excavatum is characterized by a depression of the anterior chest wall (sternum and lower costal cartilages) and is the most frequently occurring chest wall deformity. The prevalence ranges from 6.28 to 12 cases per 1000 around the world. Generally pectus excavatum is present at birth or is identified after a few weeks or months; however, sometimes it becomes evident only at puberty. The consequence of the condition on a individual's life is variable, some live a normal life and others have physical and psychological symptoms such as: precordial pain after exercises; impairments of pulmonary and cardiac function; shyness and social isolation. For many years, sub-perichondrial resection of the costal cartilages, with or without transverse cuneiform osteotomy of the sternum and placement of a substernal support, called conventional surgery, was the most accepted option for surgical repair of these patients. From 1997 a new surgical repair called, minimally invasive surgery, became available. This less invasive surgical option consists of the retrosternal placement of a curved metal bar, without resections of the costal cartilages or sternum osteotomy, and is performed by videothoracoscopy. However, many aspects that relate to the benefits and harms of both techniques have not been defined. Objectives: To evaluate the effectiveness and safety of the conventional surgery compared with minimally invasive surgery for treating people with pectus excavatum. Search methods: With the aim of increasing the sensitivity of the search strategy we used only terms related to the individual's condition (pectus excavatum); terms related to the interventions, outcomes and types of studies were not included. We searched the Cochrane Central Register of Controlled Trials (CENTRAL), PubMed, Embase, LILACS, and ICTPR. Additionally we searched yet reference lists of articles and conference proceedings. All searches were done without language restriction. Date of the most recent searches: 14 January 2014. Selection criteria: We considered randomized or quasi-randomized controlled trials that compared traditional surgery with minimally invasive surgery for treating pectus excavatum. Data collection and analysis: Two review authors independently assessed the eligibility of the trials identified and agreed trial eligibility after a consensus meeting. The authors also assessed the risk of bias of the eligible trials. Main results: Initially we located 4111 trials from the electronic searches and two further trials from other resources. All trials were added into reference management software and the duplicates were excluded, leaving 2517 studies. The titles and abstracts of these 2517 studies were independently analyzed by two authors and finally eight trials were selected for full text analysis, after which they were all excluded, as they did not fulfil the inclusion criteria. Authors' conclusions: There is no evidence from randomized controlled trials to conclude what is the best surgical option to treat people with pectus excavatum.

O brincar para conhecer o desenvolvimento de crianças com implante coclear

This research analyses the development of children presented with profound deafness and benefited by the cochlear implant, discovery of great significance in hearing health. The work is based, theoretically, on Winnicott, and methodologically, in anamnesis data and playing on a set of ludic scenes, systematically organized. Ten pre-school children with implants, selected by hospital and homogenization eligibility criteria participate in this study encompassing interviews with parents and playful observation sessions with the children, besides Lynn´s Dolls Structured Game. In the children, the results show immaturity, regression to earlier stages of their development, dependency and behavioral disorders, in particular, those related to language, interrelationship and anxiety. In the parents, family disorientation, partly overcome. The children and family participating are assisted by a multidisciplinary health team, at the hospital where they are attended.

Clinical trial design in non-invasive brain stimulation psychiatric research

Major depressive disorder (MDD) trials - investigating either non-pharmacological or pharmacological interventions - have shown mixed results. Many reasons explain this heterogeneity, but one that stands out is the trial design due to specific challenges in the field. We aimed therefore to review the methodology of non-invasive brain stimulation (NIBS) trials and provide a framework to improve clinical trial design. We performed a systematic review for randomized, controlled MDD trials whose intervention was transcranial magnetic stimulation (rTMS) or transcranial direct current stimulation (tDCS) in MEDLINE and other databases from April 2002 to April 2008. We created an unstructured checklist based on CONSORT guidelines to extract items such as power analysis, sham method, blinding assessment, allocation concealment, operational criteria used for MDD, definition of refractory depression and primary study hypotheses. Thirty-one studies were included. We found that the main methodological issues can be divided in to three groups: (1) issues related to phase II/small trials, (2) issues related to MDD trials and, (3) specific issues of NIBS studies. Taken together, they can threaten study validity and lead to inconclusive results. Feasible solutions include: estimating the sample size a priori; measuring the degree of refractoriness of the subjects; specifying the primary hypothesis and statistical tests; controlling predictor variables through stratification randomization methods or using strict eligibility criteria; adjusting the study design to the target population; using adaptive designs and exploring NIBS efficacy employing biological markers. In conclusion, our study summarizes the main methodological issues of NIBS trials and proposes a number of alternatives to manage them. Copyright (C) 2011 John Wiley & Sons, Ltd.

Lipid levels in the second year of life among HIV-infected and HIV-exposed uninfected Latin American children

Background: Dyslipidemia is observed among older children and adults with HIV. We examined nonfasting cholesterol and triglycerides in two groups of 12-23-month-old Latin American children - HIV-infected vs. HIV-exposed but uninfected (HEU). Methods: HIV-infected and HEU children in Latin America and Jamaica were enrolled in an observational cohort. Eligibility for this analysis required having cholesterol and triglyceride results available during the second year of life. Results: HIV-infected (n = 83) children were slightly older at the time of lipid testing than the HEU (n 681). Forty percent of the HIV-infected children were on protease inhibitor-based antiretroviral therapy (ART); 41% were not on ART. There was no statistically significant difference in mean cholesterol concentrations (mg/dl) by HIV status; however, the HIV-infected children had higher mean triglyceride concentrations. The prevalence of high cholesterol (>200 mg/dl) and high triglycerides (>110 mg/dl) was higher among the HIV-infected vs. HEU. Among the HIV-infected children, mean cholesterol and triglyceride concentrations varied by ART. Children receiving no ART had a significantly lower mean cholesterol concentration. Those receiving protease inhibitor-containing ART had a significantly higher mean triglyceride concentration compared to the other two antiretroviral regimen groups. Conclusion: A greater proportion of HIV-infected children at 12-23 months have hyperlipidemia when compared to HEU children, with the highest triglyceride concentrations observed among those receiving protease inhibitor-containing ART, and the lowest cholesterol levels among those not receiving ART. Implications of these findings will require continued follow-up of HIV-infected children who initiate therapy early in life. (C) 2012 Wolters Kluwer Health vertical bar Lippincott Williams & Wilkins

Ultrasonographic and laboratory markers of metabolic and cardiovascular disease risk in obese women with polycystic ovary syndrome

Objective To evaluate whether the presence of polycystic ovary syndrome (PCOS) alters multiple ultrasonographic and laboratory markers of metabolic and cardiovascular disease risk in obese women without any other health condition that could interfere with combined oral contraceptive (COC) eligibility criteria. Methods This was a case- control study evaluating 90 obese women ( body mass index ( BMI) = 30.0 kg/m2 and < 40 kg/m2) aged between 18 and 40 years without any other health condition that could interfere with COC eligibility criteria, of whom 45 had PCOS and 45 were age- matched controls. BMI, waist and hip circumference, arterial blood pressure, fasting insulin and glucose, quantitative insulin sensitivity check index ( QUICKI), highdensity lipoprotein cholesterol, low- density lipoprotein cholesterol, total cholesterol, triglycerides, testosterone, sex hormone- binding globulin, free androgen index ( FAI), carotid stiffness index, intima media thickness, flowmediated dilatation ( FMD) of the brachial artery and non- alcoholic fatty liver disease ( NAFLD) were assessed. Results In women with PCOS, we observed a higher frequency of NAFLD ( 73.3 vs. 46.7%, P < 0.01) and higher FAI ( 10.4 vs. 6.8%, P < 0.01). We also observed a trend towards increased insulin levels ( 10.06 +/- 6.66 vs. 7.45 +/- 5.88 mu IU/mL, P = 0.05), decreased QUICKI ( 0.36 +/- 0.06 vs. 0.39 +/- 0.07, P = 0.05) and decreased FMD ( 7.00 +/- 3.87 vs. 8.41 +/- 3.79%, P = 0.08). No other significant difference was observed. Conclusions NAFLD is frequent in obese women without any other health condition that could interfere with COC eligibility criteria, especially in those with PCOS. This should be considered when choosing the best contraceptive option. Copyright (C) 2012 ISUOG. Published by John Wiley & Sons, Ltd.

"Evaluating the model of classification and valuation of disabilities used in Brazil and defining the elaboration and adoption of a unique model for all the country": Brazilian Interministerial Workgroup Task

The President of Brazil established an Interministerial Work Group in order to “evaluate the model of classification and valuation of disabilities used in Brazil and to define the elaboration and adoption of a unique model for all the country”. Eight Ministries and/or Secretaries participated in the discussion over a period of 10 months, concluding that a proposed model should be based on the United Nations Convention on the Rights of Person with Disabilities, the International Classification of Functioning, Disability and Health, and the ‘support theory’, and organizing a list of recommendations and necessary actions for a Classification, Evaluation and Certification Network with national coverage.

The Emergence of the Post-Socialist Welfare State - The Case of the Baltic States : Estonia, Latvia and Lithuania

This dissertation takes a step towards providing a better understanding of post-socialist welfare state development from a theoretical as well as an empirical perspective. The overall analytical goal of this thesis has been to critically assess the development of social policies in Estonia, Latvia and Lithuania using them as illustrative examples of post-socialist welfare state development in the light of the theories, approaches and typologies that have been developed to study affluent capitalist democracies. The four studies included in this dissertation aspire to a common aim in a number of specific ways. The first study tries to place the ideal-typical welfare state models of the Baltic States within the well-known welfare state typologies. At the same time, it provides a rich overview of the main social security institutions in the three countries by comparing them with each other and with the previous structures of the Soviet period. It examines the social insurance institutions of the Baltic States (old-age pensions, unemployment insurance, short-term benefits, sickness, maternity and parental insurance and family benefits) with respect to conditions of eligibility, replacement rates, financing and contributions. The findings of this study indicate that the Latvian social security system can generally be labelled as a mix of the basic security and corporatist models. The Estonian social security system can generally also be characterised as a mix of the basic security and corporatist models, even if there are some weak elements of the targeted model in it. It appears that the institutional changes developing in the social security system of Lithuania have led to a combination of the basic security and targeted models of the welfare state. Nevertheless, as the example of the three Baltic States shows, there is diversity in how these countries solve problems within the field of social policy. In studying the social security schemes in detail, some common features were found that could be attributed to all three countries. Therefore, the critical analysis of the main social security institutions of the Baltic States in this study gave strong supporting evidence in favour of identifying the post-socialist regime type that is already gaining acceptance within comparative welfare state research. Study Two compares the system of social maintenance and insurance in the Soviet Union, which was in force in the three Baltic countries before their independence, with the currently existing social security systems. The aim of the essay is to highlight the forces that have influenced the transformation of the social policy from its former highly universal, albeit authoritarian, form, to the less universal, social insurance-based systems of present-day Estonia, Latvia and Lithuania. This study demonstrates that the welfare–economy nexus is not the only important factor in the development of social programs. The results of this analysis revealed that people's attitudes towards distributive justice and the developmental level of civil society also play an important part in shaping social policies. The shift to individualism in people’s mentality and the decline of the labour movement, or, to be more precise, the decline in trade union membership and influence, does nothing to promote the development of social rights in the Baltic countries and hinders the expansion of social policies. The legacy of the past has been another important factor in shaping social programs. It can be concluded that social policy should be studied as if embedded not only in the welfare-economy nexus, but also in the societal, historical and cultural nexus of a given society. Study Three discusses the views of the state elites on family policy within a wider theoretical setting covering family policy and social policy in a broader sense and attempts to expand this analytical framework to include other post-socialist countries. The aim of this essay is to explore the various views of the state elites in the Baltics concerning family policy and, in particular, family benefits as one of the possible explanations for the observed policy differences. The qualitative analyses indicate that the Baltic States differ significantly with regard to the motives behind their family policies. Lithuanian decision-makers seek to reduce poverty among families with children and enhance the parents’ responsibility for bringing up their children. Latvian policy-makers act so as to increase the birth rate and create equal opportunities for children from all families. Estonian policy-makers seek to create equal opportunities for all children and the desire to enhance gender equality is more visible in the case of Estonia in comparison with the other two countries. It is strongly arguable that there is a link between the underlying motives and the kinds of family benefits in a given country. This study, thus, indicates how intimately the attitudes of the state bureaucrats, policy-makers, political elite and researchers shape social policy. It confirms that family policy is a product of the prevailing ideology within a country, while the potential influence of globalisation and Europeanisation is detectable too. The final essay takes into account the opinions of welfare users and examines the performances of the institutionalised family benefits by relying on the recipients’ opinions regarding these benefits. The opinions of the populations as a whole regarding government efforts to help families are compared with those of the welfare users. Various family benefits are evaluated according to the recipients' satisfaction with those benefits as well as the contemporaneous levels of subjective satisfaction with the welfare programs related to the absolute level of expenditure on each program. The findings of this paper indicate that, in Latvia, people experience a lower level of success regarding state-run family insurance institutions, as compared to those in Lithuania and Estonia. This is deemed to be because the cash benefits for families and children in Latvia are, on average, seen as marginally influencing the overall financial situation of the families concerned. In Lithuania and Estonia, the overwhelming majority think that the family benefit systems improve the financial situation of families. It appears that recipients evaluated universal family benefits as less positive than targeted benefits. Some universal benefits negatively influenced the level of general satisfaction with the family benefits system provided in the countries being researched. This study puts forward a discussion about whether universalism is always more legitimate than targeting. In transitional economies, in which resources are highly constrained, some forms of universal benefits could turn out to be very expensive in relative terms, without being seen as useful or legitimate forms of help to families. In sum, by closely examining the different aspects of social policy, this dissertation goes beyond the over-generalisation of Eastern European welfare state development and, instead, takes a more detailed look at what is really going on in these countries through the examples of Lithuania, Latvia and Estonia. In addition, another important contribution made by this study is that it revives ‘western’ theoretical knowledge through ‘eastern’ empirical evidence and provides the opportunity to expand the theoretical framework for post-socialist societies.

I limiti all'autonomia procedurale degli Stati membri dell'UE in materia risarcitoria

This work analyses the limits that the principle of State liability for damages suffered by individuals because of breach of EU law poses to the procedural autonomy of the Member States of the EU. The introductory part of this work is dedicated to the general character of the limitations EU law poses to the State’s competence in procedural matters. The first part of the research, instead, focuses on the specific limits that european law poses on the rules of procedure related to the legal regime of the right to compensation and its operating conditions; in particular, this first part explores respectively the “substantive” and “procedural” limits that EU law poses to the State’s autonomy to regulate actions for damages for breaches of EU law. The substantial limits concern the conditions of eligibility of liability and the constitutive conditions of the right to compensation; the procedural limits to the action for damages refer to the concrete organization and characteristics of the judicial action. The second part of the research is devoted to rules of procedure governing the relations between judicial remedies explicitly aimed at protecting the right to reparation and other remedies that may be relevant, both europeans and nationals. The first chapter of the second part of this work focuses on the rules governing the relations between the action for damages brought up at the national level and the remedies provided by european Treaties; finally, I explore the relations between the action for damages brought up at the national level and other remedies present in the same national juridical order. I reconstruct all the limits to the procedural autonomy of Member States concerning the right to compensation; consequently, I verify that those limits represent part of the system of internal procedures, able to guarantee the respect of european law.

Das OtCM-System zur elektronischen Compliance-Messung : Qualitätsuntersuchung der OtCM-Blister und e-Dispenser ; Validierung und Nutzenevaluation im Vergleich mit MEMS

Im Rahmen der vorliegenden Arbeit wurden Eignung und Nutzen des „Objective therapy Compliance Measurement“ (OtCMTM)-Systems, einer innovativen Weiterentwicklung im Bereich der elektronischen Compliance-Messung, untersucht. Unter experimentellen Bedingungen wurden Funktionalität und Verlässlichkeit der elektronischen OtCMTM-Blisterpackungen überprüft, um deren Eignung für den klinischen Einsatz zu zeigen. Funktionalität (≥90% lesbare Blister), Richtigkeit (≤2% Fehler) und Robustheit waren bei den OtCMTM-Blistern der Version 3 gegeben, nachdem die Fehler der Versionen 1 und 2 in Zusammenarbeit mit dem Hersteller TCG identifiziert und eliminiert worden waren. Der als Alternative zu den elektronischen Blistern für die Verpackung von klinischen Prüfmustern entwickelte OtCMTM e-Dispenser wurde bezüglich Funktionalität und Anwenderfreundlichkeit in einer Pilotstudie untersucht. Dabei wurde ein Optimierungsbedarf festgestellt. In einer klinischen Studie wurde das OtCMTM-System mit dem als „Goldstandard“ geltenden MEMS® verglichen. Vergleichskriterien waren Datenqualität, Akzeptanz und Anwenderfreundlichkeit, Zeitaufwand bei der Bereitstellung der Medikation und Datenauswertung, sowie Validität. Insgesamt 40 Patienten, die mit Rekawan® retard 600mg behandelt wurden, nahmen an der offenen, randomisierten, prospektiven Studie teil. Das OtCMTM-System zeigte sich bezüglich Validität, Akzeptanz und Anwenderfreundlichkeit mit MEMS® vergleichbar. Eine erwartete Zeitersparnis wurde mit dem OtCMTM-System gegenüber MEMS® nicht erreicht. Vorteile des OtCMTM-Systems sind eine höhere Datenqualität und die Möglichkeit zum Einsatz in der Telemedizin.

How to assess the external validity of therapeutic trials: a conceptual approach

Background External validity of study results is an important issue from a clinical point of view. From a methodological point of view, however, the concept of external validity is more complex than it seems to be at first glance. Methods Methodological review to address the concept of external validity. Results External validity refers to the question whether results are generalizable to persons other than the population in the original study. The only formal way to establish the external validity would be to repeat the study for that specific target population. We propose a three-way approach for assessing the external validity for specified target populations. (i) The study population might not be representative for the eligibility criteria that were intended. It should be addressed whether the study population differs from the intended source population with respect to characteristics that influence outcome. (ii) The target population will, by definition, differ from the study population with respect to geographical, temporal and ethnical conditions. Pondering external validity means asking the question whether these differences may influence study results. (iii) It should be assessed whether the study's conclusions can be generalized to target populations that do not meet all the eligibility criteria. Conclusion Judging the external validity of study results cannot be done by applying given eligibility criteria to a single target population. Rather, it is a complex reflection in which prior knowledge, statistical considerations, biological plausibility and eligibility criteria all have place.

Outcomes in patients waiting for antiretroviral treatment in the Free State Province, South Africa: prospective linkage study

Objective: In South Africa, many HIV-infected patients experience delays in accessing antiretroviral therapy (ART). We examined pretreatment mortality and access to treatment in patients waiting for ART. Design: Cohort of HIV-infected patients assessed for ART eligibility at 36 facilities participating in the Comprehensive HIV and AIDS Management (CHAM) program in the Free State Province. Methods: Proportion of patients initiating ART, pre-ART mortality and risk factors associated with these outcomes were estimated using competing risks survival analysis. Results: Forty-four thousand, eight hundred and forty-four patients enrolled in CHAM between May 2004 and December 2007, of whom 22 083 (49.2%) were eligible for ART; pre-ART mortality was 53.2 per 100 person-years [95% confidence interval (CI) 51.8–54.7]. Median CD4 cell count at eligibility increased from 87 cells/ml in 2004 to 101 cells/ml in 2007. Two years after eligibility an estimated 67.7% (67.1–68.4%) of patients had started ART, and 26.2% (25.6–26.9%) died before starting ART. Among patients with CD4 cell counts below 25 cells/ml at eligibility, 48% died before ART and 51% initiated ART. Men were less likely to start treatment and more likely to die than women. Patients in rural clinics or clinics with low staffing levels had lower rates of starting treatment and higher mortality compared with patients in urban/peri-urban clinics, or better staffed clinics. Conclusions: Mortality is high in eligible patients waiting for ART in the Free State Province. The most immunocompromised patients had the lowest probability of starting ART and the highest risk of pre-ART death. Prioritization of these patients should reduce waiting times and pre-ART mortality.

Effects of glucosamine, chondroitin, or placebo in patients with osteoarthritis of hip or knee: network meta-analysis

OBJECTIVE: To determine the effect of glucosamine, chondroitin, or the two in combination on joint pain and on radiological progression of disease in osteoarthritis of the hip or knee. Design Network meta-analysis. Direct comparisons within trials were combined with indirect evidence from other trials by using a Bayesian model that allowed the synthesis of multiple time points. MAIN OUTCOME MEASURE: Pain intensity. Secondary outcome was change in minimal width of joint space. The minimal clinically important difference between preparations and placebo was prespecified at -0.9 cm on a 10 cm visual analogue scale. DATA SOURCES: Electronic databases and conference proceedings from inception to June 2009, expert contact, relevant websites. Eligibility criteria for selecting studies Large scale randomised controlled trials in more than 200 patients with osteoarthritis of the knee or hip that compared glucosamine, chondroitin, or their combination with placebo or head to head. Results 10 trials in 3803 patients were included. On a 10 cm visual analogue scale the overall difference in pain intensity compared with placebo was -0.4 cm (95% credible interval -0.7 to -0.1 cm) for glucosamine, -0.3 cm (-0.7 to 0.0 cm) for chondroitin, and -0.5 cm (-0.9 to 0.0 cm) for the combination. For none of the estimates did the 95% credible intervals cross the boundary of the minimal clinically important difference. Industry independent trials showed smaller effects than commercially funded trials (P=0.02 for interaction). The differences in changes in minimal width of joint space were all minute, with 95% credible intervals overlapping zero. Conclusions Compared with placebo, glucosamine, chondroitin, and their combination do not reduce joint pain or have an impact on narrowing of joint space. Health authorities and health insurers should not cover the costs of these preparations, and new prescriptions to patients who have not received treatment should be discouraged.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.