Poor Reliability between Cochrane Reviewers and Blinded External Reviewers When Applying the Cochrane Risk of Bias Tool in Physical Therapy Trials

OBJECTIVES To test the inter-rater reliability of the RoB tool applied to Physical Therapy (PT) trials by comparing ratings from Cochrane review authors with those of blinded external reviewers. METHODS Randomized controlled trials (RCTs) in PT were identified by searching the Cochrane Database of Systematic Reviews for meta-analysis of PT interventions. RoB assessments were conducted independently by 2 reviewers blinded to the RoB ratings reported in the Cochrane reviews. Data on RoB assessments from Cochrane reviews and other characteristics of reviews and trials were extracted. Consensus assessments between the two reviewers were then compared with the RoB ratings from the Cochrane reviews. Agreement between Cochrane and blinded external reviewers was assessed using weighted kappa (κ). RESULTS In total, 109 trials included in 17 Cochrane reviews were assessed. Inter-rater reliability on the overall RoB assessment between Cochrane review authors and blinded external reviewers was poor (κ  =  0.02, 95%CI: -0.06, 0.06]). Inter-rater reliability on individual domains of the RoB tool was poor (median κ  = 0.19), ranging from κ  =  -0.04 ("Other bias") to κ  =  0.62 ("Sequence generation"). There was also no agreement (κ  =  -0.29, 95%CI: -0.81, 0.35]) in the overall RoB assessment at the meta-analysis level. CONCLUSIONS Risk of bias assessments of RCTs using the RoB tool are not consistent across different research groups. Poor agreement was not only demonstrated at the trial level but also at the meta-analysis level. Results have implications for decision making since different recommendations can be reached depending on the group analyzing the evidence. Improved guidelines to consistently apply the RoB tool and revisions to the tool for different health areas are needed.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

Current State of Reporting Pain Outcomes in Cochrane Reviews of Chronic Musculoskeletal Pain Conditions and Considerations for an OMERACT Research Agenda.

OBJECTIVE To assess the current state of reporting of pain outcomes in Cochrane reviews on chronic musculoskeletal painful conditions and to elicit opinions of patients, healthcare practitioners, and methodologists on presenting pain outcomes to patients, clinicians, and policymakers. METHODS We identified all reviews in the Cochrane Library of chronic musculoskeletal pain conditions from Cochrane review groups (Back, Musculoskeletal, and Pain, Palliative, and Supportive Care) that contained a summary of findings (SoF) table. We extracted data on reported pain domains and instruments and conducted a survey and interviews on considerations for SoF tables (e.g., pain domains, presentation of results). RESULTS Fifty-seven SoF tables in 133 Cochrane reviews were eligible. SoF tables reported pain in 56/57, with all presenting results for pain intensity (20 different outcome instruments), pain interference in 8 SoF tables (5 different outcome instruments), and pain frequency in 1 multiple domain instrument. Other domains like pain quality or pain affect were not reported. From the survey and interviews [response rate 80% (36/45)], we derived 4 themes for a future research agenda: pain domains, considerations for assessing truth, discrimination, and feasibility; clinically important thresholds for responder analyses and presenting results; and establishing hierarchies of outcome instruments. CONCLUSION There is a lack of standardization in the domains of pain selected and the manner that pain outcomes are reported in SoF tables, hampering efforts to synthesize evidence. Future research should focus on the themes identified, building partnerships to achieve consensus and develop guidance on best practices for reporting pain outcomes.

Can decision making in general surgery be based on evidence? An empirical study of Cochrane Reviews

BACKGROUND: This empirical study analyzes the current status of Cochrane Reviews (CRs) and their strength of recommendation for evidence-based decision making in the field of general surgery. METHODS: Systematic literature search of the Cochrane Database of Systematic Reviews and the Cochrane Collaboration's homepage to identify available CRs on surgical topics. Quantitative and qualitative characteristics, utilization, and formulated treatment recommendations were evaluated by 2 independent reviewers. Association of review characteristics with treatment recommendation was analyzed using univariate and multivariate logistic regression models. RESULTS: Ninety-three CRs, including 1,403 primary studies and 246,473 patients, were identified. Mean number of included primary studies per CR was 15.1 (standard deviation [SD] 14.5) including 2,650 (SD 3,340) study patients. Two and a half (SD 8.3) nonrandomized trials were included per analyzed CR. Seventy-two (77%) CRs were published or updated in 2005 or later. Explicit treatment recommendations were given in 45 (48%). Presence of a treatment recommendation was associated with the number of included primary studies and the proportion of randomized studies. Utilization of surgical CRs remained low and showed large inter-country differences. The most surgical CRs were accessed in UK, USA, and Australia, followed by several Western and Eastern European countries. CONCLUSION: Only a minority of available CRs address surgical questions and their current usage is low. Instead of unsystematically increasing the number of surgical CRs it would be far more efficient to focus the review process on relevant surgical questions. Prioritization of CRs needs valid methods which should be developed by the scientific surgical community.

Active surveillance for rheumatic heart disease in endemic regions: a systematic review and meta-analysis of prevalence among children and adolescents

BACKGROUND Rheumatic heart disease accounts for up to 250 000 premature deaths every year worldwide and can be regarded as a physical manifestation of poverty and social inequality. We aimed to estimate the prevalence of rheumatic heart disease in endemic countries as assessed by different screening modalities and as a function of age. METHODS We searched Medline, Embase, the Latin American and Caribbean System on Health Sciences Information, African Journals Online, and the Cochrane Database of Systematic Reviews for population-based studies published between Jan 1, 1993, and June 30, 2014, that reported on prevalence of rheumatic heart disease among children and adolescents (≥5 years to <18 years). We assessed prevalence of clinically silent and clinically manifest rheumatic heart disease in random effects meta-analyses according to screening modality and geographical region. We assessed the association between social inequality and rheumatic heart disease with the Gini coefficient. We used Poisson regression to analyse the effect of age on prevalence of rheumatic heart disease and estimated the incidence of rheumatic heart disease from prevalence data. FINDINGS We included 37 populations in the systematic review and meta-analysis. The pooled prevalence of rheumatic heart disease detected by cardiac auscultation was 2·9 per 1000 people (95% CI 1·7-5·0) and by echocardiography it was 12·9 per 1000 people (8·9-18·6), with substantial heterogeneity between individual reports for both screening modalities (I(2)=99·0% and 94·9%, respectively). We noted an association between social inequality expressed by the Gini coefficient and prevalence of rheumatic heart disease (p=0·0002). The prevalence of clinically silent rheumatic heart disease (21·1 per 1000 people, 95% CI 14·1-31·4) was about seven to eight times higher than that of clinically manifest disease (2·7 per 1000 people, 1·6-4·4). Prevalence progressively increased with advancing age, from 4·7 per 1000 people (95% CI 0·0-11·2) at age 5 years to 21·0 per 1000 people (6·8-35·1) at 16 years. The estimated incidence was 1·6 per 1000 people (0·8-2·3) and remained constant across age categories (range 2·5, 95% CI 1·3-3·7 in 5-year-old children to 1·7, 0·0-5·1 in 15-year-old adolescents). We noted no sex-related differences in prevalence (p=0·829). INTERPRETATION We found a high prevalence of rheumatic heart disease in endemic countries. Although a reduction in social inequalities represents the cornerstone of community-based prevention, the importance of early detection of silent rheumatic heart disease remains to be further assessed. FUNDING UBS Optimus Foundation.

PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study.

OBJECTIVE There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria. DESIGN Meta-epidemiological study within Cochrane Database of Systematic Reviews. METHODS Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis. RESULTS Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6. CONCLUSIONS The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

Reporting and handling missing outcome data in mental health: a systematic review of Cochrane systematic reviews and meta-analyses

OBJECTIVES The purpose of the study was to provide empirical evidence about the reporting of methodology to address missing outcome data and the acknowledgement of their impact in Cochrane systematic reviews in the mental health field. METHODS Systematic reviews published in the Cochrane Database of Systematic Reviews after January 1, 2009 by three Cochrane Review Groups relating to mental health were included. RESULTS One hundred ninety systematic reviews were considered. Missing outcome data were present in at least one included study in 175 systematic reviews. Of these 175 systematic reviews, 147 (84%) accounted for missing outcome data by considering a relevant primary or secondary outcome (e.g., dropout). Missing outcome data implications were reported only in 61 (35%) systematic reviews and primarily in the discussion section by commenting on the amount of the missing outcome data. One hundred forty eligible meta-analyses with missing data were scrutinized. Seventy-nine (56%) of them had studies with total dropout rate between 10 and 30%. One hundred nine (78%) meta-analyses reported to have performed intention-to-treat analysis by including trials with imputed outcome data. Sensitivity analysis for incomplete outcome data was implemented in less than 20% of the meta-analyses. CONCLUSIONS Reporting of the techniques for handling missing outcome data and their implications in the findings of the systematic reviews are suboptimal.

The Character of Christian-Muslim Encounter: Essays in Honour of David Thomas

The Character of Christian-Muslim Encounter is a Festschrift in honour of David Thomas, Professor of Christianity and Islam, and Nadir Dinshaw Professor of Inter Religious Relations, at the University of Birmingham, UK. The Editors have put together a collection of over 30 contributions from colleagues of Professor Thomas that commences with a biographical sketch and representative tribute provided by a former doctoral student, and comprises a series of wide-ranging academic papers arranged to broadly reflect three dimensions of David Thomas’ academic and professional work – studies in and of Islam; Christian-Muslim relations; the Church and interreligious engagement. These are set in the context of a focussed theme – the character of Christian-Muslim encounters – and cast within a broad chronological framework.

Complexity of miR-223 regulation by CEBPA in human AML

microRNA-223 (miR-223) can trigger normal granulopoiesis. miR-223 expression is regulated by two distinct CEBPA (CCAAT/enhancer binding protein-alpha) sites. Here, we report that miR-223 is largely suppressed in cells from acute myeloid leukemia (AML) patients. By sequencing, we found that miR-223 suppression in AML is not caused by DNA sequence alterations, nor is it mediated by promoter hypermethylation. The analysis of the individual contribution of both CEBPA sites to miR-223 regulation identified the site upstream of the miR-223 primary transcript as the predominant regulatory element. Our results suggest that miR-223 suppression in AML is caused by impaired miR-223 upstream factors.

Morphology of the nasopalatine canal and dental implant surgery: a radiographic analysis of 100 consecutive patients using limited cone-beam computed tomography

To analyze the dimensions and anatomic characteristics of the nasopalatine canal and the corresponding buccal bone plate of the alveolar process, using limited cone-beam computed tomography (CBCT) imaging.

Expression of human cathepsin L or human cathepsin V in mouse thymus mediates positive selection of T helper cells in cathepsin L knock-out mice

A genetic deficiency of the cysteine protease cathepsin L (Ctsl) in mice results in impaired positive selection of conventional CD4+ T helper cells as a result of an incomplete processing of the MHC class II associated invariant chain or incomplete proteolytic generation of positively selecting peptide ligands. The human genome encodes, in contrast to the mouse genome, for two cathepsin L proteases, namely cathepsin L (CTSL) and cathepsin V (CTSV; alternatively cathepsin L2). In the human thymic cortex, CTSV is the predominately expressed protease as compared to CTSL or other cysteine cathepsins. In order to analyze the functions of CTSL and CTSV in the positive selection of CD4+ T cells we employed Ctsl knock-out mice crossed either with transgenic mice expressing CTSL under the control of its genuine human promoter or with transgenic mice expressing CTSV under the control of the keratin 14 (K14) promoter, which drives expression to the cortical epithelium. Both human proteases are expressed in the thymus of the transgenic mice, and independent expression of both CTSL and CTSV rescues the reduced frequency of CD4+ T cells in Ctsl-deficient mice. Moreover, the expression of the human cathepsins does not change the number of CD4+CD25+Foxp3+ regulatory T cells, but the normalization of the frequency of conventional CD4+ T cell in the transgenic mice results in a rebalancing of conventional T cells and regulatory T cells. We conclude that the functional differences of CTSL and CTSV in vivo are not mainly determined by their inherent biochemical properties, but rather by their tissue specific expression pattern.

Joint lavage for osteoarthritis of the knee

Background Osteoarthritis is the most common form of joint disorder and a leading cause of pain and physical disability. Observational studies suggested a benefit for joint lavage, but recent, sham-controlled trials yielded conflicting results, suggesting joint lavage not to be effective. Objectives To compare joint lavage with sham intervention, placebo or non-intervention control in terms of effects on pain, function and safety outcomes in patients with knee osteoarthritis. Search methods We searched CENTRAL, MEDLINE, EMBASE, and CINAHL up to 3 August 2009, checked conference proceedings, reference lists, and contacted authors. Selection criteria We included studies if they were randomised or quasi-randomised trials that compared arthroscopic and non-arthroscopic joint lavage with a control intervention in patients with osteoarthritis of the knee. We did not apply any language restrictions. Data collection and analysis Two independent review authors extracted data using standardised forms. We contacted investigators to obtain missing outcome information. We calculated standardised mean differences (SMDs) for pain and function, and risk ratios for safety outcomes. We combined trials using inverse-variance random-effects meta-analysis. Main results We included seven trials with 567 patients. Three trials examined arthroscopic joint lavage, two non-arthroscopic joint lavage and two tidal irrigation. The methodological quality and the quality of reporting was poor and we identified a moderate to large degree of heterogeneity among the trials (I2 = 65%). We found little evidence for a benefit of joint lavage in terms of pain relief at three months (SMD -0.11, 95% CI -0.42 to 0.21), corresponding to a difference in pain scores between joint lavage and control of 0.3 cm on a 10-cm visual analogue scale (VAS). Results for improvement in function at three months were similar (SMD -0.10, 95% CI -0.30 to 0.11), corresponding to a difference in function scores between joint lavage and control of 0.2 cm on a WOMAC disability sub-scale from 0 to 10. For pain, estimates of effect sizes varied to some degree depending on the type of lavage, but this variation was likely to be explained by differences in the credibility of control interventions: trials using sham interventions to closely mimic the process of joint lavage showed a null-effect. Reporting on adverse events and drop out rates was unsatisfactory, and we were unable to draw conclusions for these secondary outcomes. Authors' conclusions Joint lavage does not result in a relevant benefit for patients with knee osteoarthritis in terms of pain relief or improvement of function.

Therapeutic ultrasound for osteoarthritis of the knee or hip

Osteoarthritis is the most common form of joint disease and the leading cause of pain and physical disability in the elderly. Therapeutic ultrasound is one of several physical therapy modalities suggested for the management of pain and loss of function due to osteoarthritis (OA).