Poor Reliability between Cochrane Reviewers and Blinded External Reviewers When Applying the Cochrane Risk of Bias Tool in Physical Therapy Trials

OBJECTIVES To test the inter-rater reliability of the RoB tool applied to Physical Therapy (PT) trials by comparing ratings from Cochrane review authors with those of blinded external reviewers. METHODS Randomized controlled trials (RCTs) in PT were identified by searching the Cochrane Database of Systematic Reviews for meta-analysis of PT interventions. RoB assessments were conducted independently by 2 reviewers blinded to the RoB ratings reported in the Cochrane reviews. Data on RoB assessments from Cochrane reviews and other characteristics of reviews and trials were extracted. Consensus assessments between the two reviewers were then compared with the RoB ratings from the Cochrane reviews. Agreement between Cochrane and blinded external reviewers was assessed using weighted kappa (κ). RESULTS In total, 109 trials included in 17 Cochrane reviews were assessed. Inter-rater reliability on the overall RoB assessment between Cochrane review authors and blinded external reviewers was poor (κ  =  0.02, 95%CI: -0.06, 0.06]). Inter-rater reliability on individual domains of the RoB tool was poor (median κ  = 0.19), ranging from κ  =  -0.04 ("Other bias") to κ  =  0.62 ("Sequence generation"). There was also no agreement (κ  =  -0.29, 95%CI: -0.81, 0.35]) in the overall RoB assessment at the meta-analysis level. CONCLUSIONS Risk of bias assessments of RCTs using the RoB tool are not consistent across different research groups. Poor agreement was not only demonstrated at the trial level but also at the meta-analysis level. Results have implications for decision making since different recommendations can be reached depending on the group analyzing the evidence. Improved guidelines to consistently apply the RoB tool and revisions to the tool for different health areas are needed.

Intergroup bias in third-party punishment stems from both ingroup favoritism and outgroup discrimination

Social norms pervade almost every aspect of social interaction. If they are violated, not only legal institutions, but other members of society as well, punish, i.e., inflict costs on the wrongdoer. Sanctioning occurs even when the punishers themselves were not harmed directly and even when it is costly for them. There is evidence for intergroup bias in this third-party punishment: third-parties, who share group membership with victims, punish outgroup perpetrators more harshly than ingroup perpetrators. However, it is unknown whether a discriminatory treatment of outgroup perpetrators (outgroup discrimination) or a preferential treatment of ingroup perpetrators (ingroup favoritism) drives this bias. To answer this question, the punishment of outgroup and ingroup perpetrators must be compared to a baseline, i.e., unaffiliated perpetrators. By applying a costly punishment game, we found stronger punishment of outgroup versus unaffiliated perpetrators and weaker punishment of ingroup versus unaffiliated perpetrators. This demonstrates that both ingroup favoritism and outgroup discrimination drive intergroup bias in third-party punishment of perpetrators that belong to distinct social groups.

Negative bias of processing ambiguously cued emotional stimuli.

Daily we cope with upcoming potentially disadvantageous events. Therefore, it makes sense to be prepared for the worst case. Such a 'pessimistic' bias is reflected in brain activation during emotion processing. Healthy individuals underwent functional neuroimaging while viewing emotional stimuli that were earlier cued ambiguously or unambiguously concerning their emotional valence. Presentation of ambiguously announced pleasant pictures compared with unambiguously announced pleasant pictures resulted in increased activity in the ventrolateral prefrontal, premotor and temporal cortex, and in the caudate nucleus. This was not the case for the respective negative conditions. This indicates that pleasant stimuli after ambiguous cueing provided 'unexpected' emotional input, resulting in the adaptation of brain activity. It strengthens the hypothesis of a 'pessimistic' bias of brain activation toward ambiguous emotional events.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

Drug-eluting stents vs. bare metal stents in patients with cardiogenic shock: a comparison by propensity score analysis.

BACKGROUND In patients with cardiogenic shock, data on the comparative safety and efficacy of drug-eluting stents (DESs) vs. bare metal stents (BMSs) are lacking. We sought to assess the performance of DESs compared with BMSs among patients with cardiogenic shock undergoing percutaneous coronary intervention (PCI). METHODS Out of 236 patients with acute coronary syndromes complicated by cardiogenic shock, 203 were included in the final analysis. The primary endpoint included death, and the secondary endpoint of major adverse cardiac and cerebrovascular events (MACCEs) included the composite of death, myocardial infarction, any repeat revascularization and stroke. Patients were followed for a minimum of 30 days and up to 4 years. As stent assignment was not random, we performed a propensity score analysis to minimize potential bias. RESULTS Among patients treated with DESs, there was a lower risk of the primary and secondary endpoints compared with BMSs at 30 days (29 vs. 56%, P < 0.001; 34 vs. 58%, P = 0.001, respectively) and during long-term follow-up [hazard ratio 0.43, 95% confidence interval (CI) 0.29-0.65, P < 0.001; hazard ratio 0.49, 95% CI 0.34-0.71, P < 0.001, respectively]. After propensity score adjustment, all-cause mortality was reduced among patients treated with DESs compared with BMSs both at 30 days [adjusted odds ratio (OR) 0.26, 95% CI 0.11-0.62; P = 0.002] and during long-term follow-up (adjusted hazard ratio 0.40, 95% CI 0.22-0.72; P = 0.002). The rate of MACCE was lower among patients treated with DESs compared with those treated with BMSs at 30 days (adjusted OR 0.42, 95% CI 0.19-0.95; P = 0.036). The difference in MACCEs between devices approached significance during long-term follow-up (adjusted hazard ratio 0.60, 95% CI 0.34-1.01; P = 0.052). CONCLUSION DESs appear to be associated with improved clinical outcomes, including a reduction in all-cause mortality compared with BMSs among patients undergoing PCI for cardiogenic shock, possibly because of a pacification of the infarct-related artery by anti-inflammatory drug. The results of this observational study require confirmation in an appropriately powered randomized trial.

Can eye movements bias memory recall?

A large body of research suggests that when we retrieve visual information from memory, we look back to the location where we encoded these objects. It has been proposed that the oculomotor trace we act out during encoding is stored in long-term memory, along other contents of the episodic representation. If memory recall triggers the eyes to revisit the location where the stimulus was encoded, is there also an effect in the reverse direction? Can eye movements trigger memory recall? In Experiment 1 participants encoded two faces at two different locations on the computer screen. Then, the average face (morph) of these two faces appeared in either of the two encoding locations and participants had to indicate whether it resembles more the first or second face. In Experiment 2 the morph appeared in a new location, but participants had to repeat one of the oculomotor traces that was used during encoding. Participants’ morph perception was influenced both by the location and the eye-movement it was presented with. Our results suggest that eye-movements can bias memory recall, but only in a short-lasting and rather fragile way.

School-age children show a bias toward fantasy classifications after playing a platform game

We investigated the influence of playing a video game on children’s ability to distinguish between fantasy and reality. School-age children played a platform game for 15 min and then performed a fantasy/reality distinction task in which they were to judge whether images (from the platform game and from other games) were fantasy images or reality images. Unlike those in the control group (who played a memory game), the children in the experimental group showed a response bias toward mistakenly classifying reality images from the video game as fantasy images (as determined by means of an analysis based on signal detection theory). We conclude that playing the video game exerted a short-term influence on children’s ability to distinguish between fantasy and reality.

Car Design and Road Crossing Behaviour

Humans possess a highly developed sensitivity for facial features. This sensitivity is also deployed to non-human beings and inanimate objects such as cars. In the present study we aimed to investigate whether car design has a bearing on the behaviour of pedestrians. Methods: An immersive virtual reality environment with a zebra crossing was used to determine a) whether the minimum accepted distance for crossing the street is bigger for cars with dominant appearance than for cars with friendly appearance (Block 1) and b) whether the speed of dominant cars are overestimated compared to friendly cars (Block 2). In Block 1, the participant's task was to cross the road in front of an approaching car at the latest moment. The point of time when entering and leaving the street was measured. In Block 2 they were asked to estimate the speed of each passing car. An independent sample rated dominant cars as being more dominant, angry and hostile than friendly cars. Results: None of the predictions regarding the car design was confirmed. Instead, there was an effect of starting position: From the centre island, participants entered the road significantly later (smaller accepted distance) and left the road later than when starting from the pavement. Consistently, the speed of the cars was estimated significantly lower when standing on the centre island compared to the pavement. When entering the visual size of the cars as factor (instead of dominance), we found that participants started to cross the road significantly later in front of small cars compared to big cars and that the speed of smaller cars was overestimated compared to big cars (size-speed bias). Conclusions: Car size and starting position, not car design seem to have an influence on road crossing behaviour.

Minimum carbon payment along an aridity gradient for dryland afforestation in non annex I countries with stochastic weather and prices

Recent findings demonstrate that trees in deserts are efficient carbon sinks. It remains however unknown whether the Clean Development Mechanism will accelerate the planting of trees in Non Annex I dryland countries. We estimated the price of carbon at which a farmer would be indifferent between his customary activity and the planting of trees to trade carbon credits, along an aridity gradient. Carbon yields were simulated by means of the CO2FIX v3.1 model for Pinus halepensis with its respective yield classes along the gradient (Arid – 100mm to Dry Sub Humid conditions – 900mm). Wheat and pasture yields were predicted on somewhat similar nitrogen-based quadratic models, using 30 years of weather data to simulate moisture stress. Stochastic production, input and output prices were afterwards simulated on a Monte Carlo matrix. Results show that, despite the high levels of carbon uptake, carbon trading by afforesting is unprofitable anywhere along the gradient. Indeed, the price of carbon would have to raise unrealistically high, and the certification costs would have to drop significantly, to make the Clean Development Mechanism worthwhile for non annex I dryland countries farmers. From a government agency's point of view the Clean Development Mechanism is attractive. However, such agencies will find it difficult to demonstrate “additionality”, even if the rule may be somewhat flexible. Based on these findings, we will further discuss why the Clean Development Mechanism, a supposedly pro-poor instrument, fails to assist farmers in Non Annex I dryland countries living at minimum subsistence level.

8p23 beta-defensin copy number determination by single-locus pseudogene-based paralog ratio tests risk bias due to low-frequency sequence variations.

BACKGROUND The copy number variation (CNV) in beta-defensin genes (DEFB) on human chromosome 8p23 has been proposed to contribute to the phenotypic differences in inflammatory diseases. However, determination of exact DEFB CN is a major challenge in association studies. Quantitative real-time PCR (qPCR), paralog ratio tests (PRT) and multiplex ligation-dependent probe amplification (MLPA) have been extensively used to determine DEFB CN in different laboratories, but inter-method inconsistencies were observed frequently. In this study we asked which one is superior among the three methods for DEFB CN determination. RESULTS We developed a clustering approach for MLPA and PRT to statistically correlate data from a single experiment. Then we compared qPCR, a newly designed PRT and MLPA for DEFB CN determination in 285 DNA samples. We found MLPA had the best convergence and clustering results of the raw data and the highest call rate. In addition, the concordance rates between MLPA or PRT and qPCR (32.12% and 37.99%, respectively) were unacceptably low with underestimated CN by qPCR. Concordance rate between MLPA and PRT (90.52%) was high but PRT systematically underestimated CN by one in a subset of samples. In these samples a sequence variant which caused complete PCR dropout of the respective DEFB cluster copies was found in one primer binding site of one of the targeted paralogous pseudogenes. CONCLUSION MLPA is superior to PRT and even more to qPCR for DEFB CN determination. Although the applied PRT provides in most cases reliable results, such a test is particularly sensitive to low-frequency sequence variations preferably accumulating in loci like pseudogenes which are most likely not under selective pressure. In the light of the superior performance of multiplex assays, the drawbacks of such single PRTs could be overcome by combining more test markers.

How narcissistic self-construction sabotages their relationships: Accuracy and bias in partner perception

Trapped by their superordinate goal to maintain a grandiose self, narcissists navigate their relationships wishing to re-assure themselves of their specialness and superiority, using strategies bound to fail. Dyadic analyses of diary data (14 days, 83 couples) showed that narcissists focus on their partners’ shortcomings, becoming experts at detecting negative forms of support, while simultaneously minimizing their partner’s positive intentions. Partners of narcissists, in contrast, appear to have a positive bias in ongoing relationships and seem to compensate for some of the narcissists’ shortcomings. Narcissists’ focus on their own selfconstruction purposes also extends to their parenting roles. In a retrospective study, narcissistic mothers were found to demand superior performance, desiring to show-off their child, yet concurrently attributing their child’s success to themselves. They also exercise psychological control and role-reversal, requiring the child to cater to their needs, and to admire them, while simultaneously oblivious to their child’s needs and desires.