Poor Reliability between Cochrane Reviewers and Blinded External Reviewers When Applying the Cochrane Risk of Bias Tool in Physical Therapy Trials

OBJECTIVES To test the inter-rater reliability of the RoB tool applied to Physical Therapy (PT) trials by comparing ratings from Cochrane review authors with those of blinded external reviewers. METHODS Randomized controlled trials (RCTs) in PT were identified by searching the Cochrane Database of Systematic Reviews for meta-analysis of PT interventions. RoB assessments were conducted independently by 2 reviewers blinded to the RoB ratings reported in the Cochrane reviews. Data on RoB assessments from Cochrane reviews and other characteristics of reviews and trials were extracted. Consensus assessments between the two reviewers were then compared with the RoB ratings from the Cochrane reviews. Agreement between Cochrane and blinded external reviewers was assessed using weighted kappa (κ). RESULTS In total, 109 trials included in 17 Cochrane reviews were assessed. Inter-rater reliability on the overall RoB assessment between Cochrane review authors and blinded external reviewers was poor (κ  =  0.02, 95%CI: -0.06, 0.06]). Inter-rater reliability on individual domains of the RoB tool was poor (median κ  = 0.19), ranging from κ  =  -0.04 ("Other bias") to κ  =  0.62 ("Sequence generation"). There was also no agreement (κ  =  -0.29, 95%CI: -0.81, 0.35]) in the overall RoB assessment at the meta-analysis level. CONCLUSIONS Risk of bias assessments of RCTs using the RoB tool are not consistent across different research groups. Poor agreement was not only demonstrated at the trial level but also at the meta-analysis level. Results have implications for decision making since different recommendations can be reached depending on the group analyzing the evidence. Improved guidelines to consistently apply the RoB tool and revisions to the tool for different health areas are needed.

Low-frequency drug-resistant HIV-1 and risk of virological failure to first-line NNRTI-based ART: a multicohort European case-control study using centralized ultrasensitive 454 pyrosequencing.

OBJECTIVES It is still debated if pre-existing minority drug-resistant HIV-1 variants (MVs) affect the virological outcomes of first-line NNRTI-containing ART. METHODS This Europe-wide case-control study included ART-naive subjects infected with drug-susceptible HIV-1 as revealed by population sequencing, who achieved virological suppression on first-line ART including one NNRTI. Cases experienced virological failure and controls were subjects from the same cohort whose viraemia remained suppressed at a matched time since initiation of ART. Blinded, centralized 454 pyrosequencing with parallel bioinformatic analysis in two laboratories was used to identify MVs in the 1%-25% frequency range. ORs of virological failure according to MV detection were estimated by logistic regression. RESULTS Two hundred and sixty samples (76 cases and 184 controls), mostly subtype B (73.5%), were used for the analysis. Identical MVs were detected in the two laboratories. 31.6% of cases and 16.8% of controls harboured pre-existing MVs. Detection of at least one MV versus no MVs was associated with an increased risk of virological failure (OR = 2.75, 95% CI = 1.35-5.60, P = 0.005); similar associations were observed for at least one MV versus no NRTI MVs (OR = 2.27, 95% CI = 0.76-6.77, P = 0.140) and at least one MV versus no NNRTI MVs (OR = 2.41, 95% CI = 1.12-5.18, P = 0.024). A dose-effect relationship between virological failure and mutational load was found. CONCLUSIONS Pre-existing MVs more than double the risk of virological failure to first-line NNRTI-based ART.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study.

OBJECTIVE There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria. DESIGN Meta-epidemiological study within Cochrane Database of Systematic Reviews. METHODS Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis. RESULTS Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6. CONCLUSIONS The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

Contribution of APOBEC3G/F Activity to the Development of Low-Abundance Drug-Resistant HIV-1 variants.

Plasma drug-resistant minority HIV-1 variants (DRMV) increase the risk of virological failure to first-line NNRTI antiretroviral therapy (ART). The origin of DRMVs in ART-naive patients, however, remains unclear. In a large pan-European case-control study investigating the clinical relevance of pre-existing DRMVs using 454 pyrosequencing, the six most prevalent plasma DRMVs detected corresponded to G-to-A nucleotide mutations (V90I, V106I, V108I, E138K, M184I and M230I). Here, we evaluated if such DRMVs could have emerged from APOBEC3G/F activity. Out of 236 ART-naïve evaluated subjects, APOBEC3G/F hypermutation signatures were detected in plasma viruses of 14 (5.9%) individuals. Samples with minority E138K, M184I, and M230I mutations, but not those with V90I, V106I, or V108I were significantly associated with APOBEC3G/F activity (Fisher's p<0.005), defined as presence of >0.5% of sample sequences with an APOBEC3G/F signature. Mutations E138K, M184I and M230I co-occurred in the same sequence as APOBEC3G/F signatures in 3/9 (33%), 5/11 (45%) and 4/8 (50%) of samples, respectively; such linkage was not found for V90I, V106I or V108I. In-frame STOP codons were observed in 1.5% of all clonal sequences; 14.8% of them co-occurred with APOBEC3G/F signatures. APOBEC3G/F-associated E138K, M184I and M230I appeared within clonal sequences containing in-frame STOP codons in 2/3 (66%), 5/5 (100%) and 4/4 (100%) of the samples. In a reanalysis of the parent case-control study, presence of APOBEC3G/F signatures was not associated with virological failure. In conclusion, the contribution of APOBEC3G/F editing to the development of DRMVs is very limited and does not affect the efficacy of NNRTI ART.

Endoscopic and Open Release Similarly Safe for the Treatment of Carpal Tunnel Syndrome. A Systematic Review and Meta-Analysis.

BACKGROUND The Endoscopic Release of Carpal Tunnel Syndrome (ECTR) is a minimal invasive approach for the treatment of Carpal Tunnel Syndrome. There is scepticism regarding the safety of this technique, based on the assumption that this is a rather "blind" procedure and on the high number of severe complications that have been reported in the literature. PURPOSE To evaluate whether there is evidence supporting a higher risk after ECTR in comparison to the conventional open release. METHODS We searched MEDLINE (January 1966 to November 2013), EMBASE (January 1980 to November 2013), the Cochrane Neuromuscular Disease Group Specialized Register (November 2013) and CENTRAL (2013, issue 11 in The Cochrane Library). We hand-searched reference lists of included studies. We included all randomized or quasi-randomized controlled trials (e.g. study using alternation, date of birth, or case record number) that compare any ECTR with any OCTR technique. Safety was assessed by the incidence of major, minor and total number of complications, recurrences, and re-operations.The total time needed before return to work or to return to daily activities was also assessed. We synthesized data using a random-effects meta-analysis in STATA. We conducted a sensitivity analysis for rare events using binomial likelihood. We judged the conclusiveness of meta-analysis calculating the conditional power of meta-analysis. CONCLUSIONS ECTR is associated with less time off work or with daily activities. The assessment of major complications, reoperations and recurrence of symptoms does not favor either of the interventions. There is an uncertain advantage of ECTR with respect to total minor complications (more transient paresthesia but fewer skin-related complications). Future studies are unlikely to alter these findings because of the rarity of the outcome. The effect of a learning curve might be responsible for reduced recurrences and reoperations with ECTR in studies that are more recent, although formal statistical analysis failed to provide evidence for such an association. LEVEL OF EVIDENCE I.

Addition of docetaxel, zoledronic acid, or both to first-line long-term hormone therapy in prostate cancer (STAMPEDE): survival results from an adaptive, multiarm, multistage, platform randomised controlled trial

BACKGROUND Long-term hormone therapy has been the standard of care for advanced prostate cancer since the 1940s. STAMPEDE is a randomised controlled trial using a multiarm, multistage platform design. It recruits men with high-risk, locally advanced, metastatic or recurrent prostate cancer who are starting first-line long-term hormone therapy. We report primary survival results for three research comparisons testing the addition of zoledronic acid, docetaxel, or their combination to standard of care versus standard of care alone. METHODS Standard of care was hormone therapy for at least 2 years; radiotherapy was encouraged for men with N0M0 disease to November, 2011, then mandated; radiotherapy was optional for men with node-positive non-metastatic (N+M0) disease. Stratified randomisation (via minimisation) allocated men 2:1:1:1 to standard of care only (SOC-only; control), standard of care plus zoledronic acid (SOC + ZA), standard of care plus docetaxel (SOC + Doc), or standard of care with both zoledronic acid and docetaxel (SOC + ZA + Doc). Zoledronic acid (4 mg) was given for six 3-weekly cycles, then 4-weekly until 2 years, and docetaxel (75 mg/m(2)) for six 3-weekly cycles with prednisolone 10 mg daily. There was no blinding to treatment allocation. The primary outcome measure was overall survival. Pairwise comparisons of research versus control had 90% power at 2·5% one-sided α for hazard ratio (HR) 0·75, requiring roughly 400 control arm deaths. Statistical analyses were undertaken with standard log-rank-type methods for time-to-event data, with hazard ratios (HRs) and 95% CIs derived from adjusted Cox models. This trial is registered at ClinicalTrials.gov (NCT00268476) and ControlledTrials.com (ISRCTN78818544). FINDINGS 2962 men were randomly assigned to four groups between Oct 5, 2005, and March 31, 2013. Median age was 65 years (IQR 60-71). 1817 (61%) men had M+ disease, 448 (15%) had N+/X M0, and 697 (24%) had N0M0. 165 (6%) men were previously treated with local therapy, and median prostate-specific antigen was 65 ng/mL (IQR 23-184). Median follow-up was 43 months (IQR 30-60). There were 415 deaths in the control group (347 [84%] prostate cancer). Median overall survival was 71 months (IQR 32 to not reached) for SOC-only, not reached (32 to not reached) for SOC + ZA (HR 0·94, 95% CI 0·79-1·11; p=0·450), 81 months (41 to not reached) for SOC + Doc (0·78, 0·66-0·93; p=0·006), and 76 months (39 to not reached) for SOC + ZA + Doc (0·82, 0·69-0·97; p=0·022). There was no evidence of heterogeneity in treatment effect (for any of the treatments) across prespecified subsets. Grade 3-5 adverse events were reported for 399 (32%) patients receiving SOC, 197 (32%) receiving SOC + ZA, 288 (52%) receiving SOC + Doc, and 269 (52%) receiving SOC + ZA + Doc. INTERPRETATION Zoledronic acid showed no evidence of survival improvement and should not be part of standard of care for this population. Docetaxel chemotherapy, given at the time of long-term hormone therapy initiation, showed evidence of improved survival accompanied by an increase in adverse events. Docetaxel treatment should become part of standard of care for adequately fit men commencing long-term hormone therapy. FUNDING Cancer Research UK, Medical Research Council, Novartis, Sanofi-Aventis, Pfizer, Janssen, Astellas, NIHR Clinical Research Network, Swiss Group for Clinical Cancer Research.

Similar outcome of upfront-unrelated and matched sibling stem cell transplantation in idiopathic paediatric aplastic anaemia. A study on behalf of the UK Paediatric BMT Working Party, Paediatric Diseases Working Party and Severe Aplastic Anaemia Working Party of EBMT

Abstract We explored the feasibility of unrelated donor haematopoietic stem cell transplant (HSCT) upfront without prior immunosuppressive therapy (IST) in paediatric idiopathic severe aplastic anaemia (SAA). This cohort was then compared to matched historical controls who had undergone first-line therapy with a matched sibling/family donor (MSD) HSCT (n = 87) or IST with horse antithymocyte globulin and ciclosporin (n = 58) or second-line therapy with unrelated donor HSCT post-failed IST (n = 24). The 2-year overall survival in the upfront cohort was 96 ± 4% compared to 91 ± 3% in the MSD controls (P = 0·30) and 94 ± 3% in the IST controls (P = 0·68) and 74 ± 9% in the unrelated donor HSCT post-IST failure controls (P = 0·02).The 2-year event-free survival in the upfront cohort was 92 ± 5% compared to 87 ± 4% in MSD controls (P = 0·37), 40 ± 7% in IST controls (P = 0·0001) and 74 ± 9% in the unrelated donor HSCT post-IST failure controls (n = 24) (P = 0·02). Outcomes for upfront-unrelated donor HSCT in paediatric idiopathic SAA were similar to MSD HSCT and superior to IST and unrelated donor HSCT post-IST failure. Front-line therapy with matched unrelated donor HSCT is a novel treatment approach and could be considered as first-line therapy in selected paediatric patients who lack a MSD. © 2015 John Wiley & Sons Ltd.