Two decades of exceptional achievements : does the evidence support nurses to favour Cochrane systematic reviews over other systematic reviews?

Systematic reviews (SRs) are increasingly recognised as the standard approach in summarising health research and influence clinical nursing practice and health care decisions (Coster and Norman, 2009, Grimshaw and Russell, 1993 and Griffiths and Norman, 2005). High quality SRs should have a clearly stated set of objectives with pre-defined eligibility criteria for studies; an explicit reproducible methodology; a systematic search that attempts to identify all studies that would meet the eligibility criteria; an assessment of the validity of the findings of the included studies; the assessment of risk of bias; and a systematic presentation and synthesis of the characteristics of findings of the included study (Higgins and Green, 2011). Although SRs are highly regarded and are expected to be rigorous, just as other research, their quality may vary (Choi et al., 2001 and Hoving et al., 2001)...

Do Cochrane summaries help student midwives understand the findings of Cochrane systematic reviews: the BRIEF randomised trial

BACKGROUND: Abstracts and plain language summaries (PLS) are often the first, and sometimes the only, point of contact between readers and systematic reviews. It is important to identify how these summaries are used and to know the impact of different elements, including the authors' conclusions. The trial aims to assess whether (a) the abstract or the PLS of a Cochrane Review is a better aid for midwifery students in assessing the evidence, (b) inclusion of authors' conclusions helps them and (c) there is an interaction between the type of summary and the presence or absence of the conclusions.

METHODS: Eight hundred thirteen midwifery students from nine universities in the UK and Ireland were recruited to this 2 × 2 factorial trial (abstract versus PLS, conclusions versus no conclusions). They were randomly allocated to one of four groups and asked to recall knowledge after reading one of four summary formats of two Cochrane Reviews, one with clear findings and one with uncertain findings. The primary outcome was the proportion of students who identified the appropriate statement to describe the main findings of the two reviews as assessed by an expert panel.

RESULTS: There was no statistically significant difference in correct response between the abstract and PLS groups in the clear finding example (abstract, 59.6 %; PLS, 64.2 %; risk difference 4.6 %; CI -0.2 to 11.3) or the uncertain finding example (42.7 %, 39.3 %, -3.4 %, -10.1 to 3.4). There was no significant difference between the conclusion and no conclusion groups in the example with clear findings (conclusions, 63.3 %; no conclusions, 60.5 %; 2.8 %; -3.9 to 9.5), but there was a significant difference in the example with uncertain findings (44.7 %; 37.3 %; 7.3 %; 0.6 to 14.1, p = 0.03). PLS without conclusions in the uncertain finding review had the lowest proportion of correct responses (32.5 %). Prior knowledge and belief predicted student response to the clear finding review, while years of midwifery education predicted response to the uncertain finding review.

CONCLUSIONS: Abstracts with and without conclusions generated similar student responses. PLS with conclusions gave similar results to abstracts with and without conclusions. Removing the conclusions from a PLS with uncertain findings led to more problems with interpretation.

Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?

The aims of this study were to assess and compare the methodological quality of Cochrane and non-Cochrane systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) using AMSTAR and to compare the prevalence of meta-analysis in both review types. A literature search was undertaken to identify SRs that consisted of hand-searching five major orthodontic journals [American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics and Orthodontics and Craniofacial Research (February 2002 to July 2011)] and the Cochrane Database of Systematic Reviews from January 2000 to July 2011. Methodological quality of the included reviews was gauged using the AMSTAR tool involving 11 key methodological criteria with a score of 0 or 1 given for each criterion. A cumulative grade was given for the paper overall (0-11); an overall score of 4 or less represented poor methodological quality, 5-8 was considered fair and 9 or greater was deemed to be good. In total, 109 SRs were identified in the five major journals and on the CDSR. Of these, 26 (23.9%) were in the CDSR. The mean overall AMSTAR score was 6.2 with 21.1% of reviews satisfying 9 or more of the 11 criteria; a similar prevalence of poor reviews (22%) was also noted. Multiple linear regression indicated that reviews published in the CDSR (P < 0.01); and involving meta-analysis (β = 0.50, 95% confidence interval 0.72, 2.07, P < 0.001) showed greater concordance with AMSTAR.

Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews

OBJECTIVES To assess discrepancies in the analyzed outcomes between protocols and published reviews within Cochrane oral health systematic reviews (COHG) on the Cochrane Database of Systematic Reviews (CDSR). STUDY DESIGN AND SETTING All COHG systematic reviews on the CDSR and the corresponding protocols were retrieved in November 2014 and information on the reported outcomes was recorded. Data was collected at the systematic review level by two reviewers independently. RESULTS One hundred and fifty two reviews were included. In relation to primary outcomes, 11.2% were downgraded to secondary outcomes, 9.9% were omitted altogether in the final publication and new primary outcomes were identified in 18.4% of publications. For secondary outcomes, 2% were upgraded to primary, 12.5% were omitted and 30.9% were newly introduced in the publication. Overall, 45.4% of reviews had at least one discrepancy when compared to the protocol; these were reported in 14.5% reviews. The number of review updates appears to be associated with discrepancies between final review and protocol (OR: 3.18, 95% CI: 1.77, 5.74, p<0.001). The risk of reporting significant results was lower for both downgraded outcomes [RR: 0.52, 95% CI: 0.17, 1.58, p = 0.24] and upgraded or newly introduced outcomes [RR: 0.77, 95% CI: 0.36, 1.64, p = 0.50] compared to outcomes with no discrepancies. The risk of reporting significant results was higher for upgraded or newly introduced outcomes compared to downgraded outcomes (RR = 1.19, 95% CI: 0.65, 2.16, p = 0.57). None of the comparisons reached statistical significance. CONCLUSION While no evidence of selective outcome reporting was found in this study, based on the present analysis of SRs published within COHG systematic reviews, discrepancies between outcomes in pre-published protocols and final reviews continue to be common. Solutions such as the use of standardized outcomes to reduce the prevalence of this issue may need to be explored.

Reporting and handling missing outcome data in mental health: a systematic review of Cochrane systematic reviews and meta-analyses

OBJECTIVES The purpose of the study was to provide empirical evidence about the reporting of methodology to address missing outcome data and the acknowledgement of their impact in Cochrane systematic reviews in the mental health field. METHODS Systematic reviews published in the Cochrane Database of Systematic Reviews after January 1, 2009 by three Cochrane Review Groups relating to mental health were included. RESULTS One hundred ninety systematic reviews were considered. Missing outcome data were present in at least one included study in 175 systematic reviews. Of these 175 systematic reviews, 147 (84%) accounted for missing outcome data by considering a relevant primary or secondary outcome (e.g., dropout). Missing outcome data implications were reported only in 61 (35%) systematic reviews and primarily in the discussion section by commenting on the amount of the missing outcome data. One hundred forty eligible meta-analyses with missing data were scrutinized. Seventy-nine (56%) of them had studies with total dropout rate between 10 and 30%. One hundred nine (78%) meta-analyses reported to have performed intention-to-treat analysis by including trials with imputed outcome data. Sensitivity analysis for incomplete outcome data was implemented in less than 20% of the meta-analyses. CONCLUSIONS Reporting of the techniques for handling missing outcome data and their implications in the findings of the systematic reviews are suboptimal.

The benefits and challenges of conducting an overview of systematic reviews in public health: a focus on physical activity

Introduction The ultimate aim of Cochrane systematic reviews is to inform policy and practice decisions for better health outcomes. However, due to the increasing numbers of scientific publications, wading through the available evidence of both individual studies and systematic reviews can be challenging and overwhelming even for avid authors and readers. This paper briefly describes the first overview (a systematic review of reviews) of the Cochrane Public Health Group (CPHG) in development and proposes a way forward for the methodologies under consideration.

Systematic reviews showed insufficient evidence for clinical practice in 2004: What about in 2011? the next appeal for the evidence-based medicine age

Rationale and aim The aims of the Cochrane systematic reviews are to make readily available and up-to-date information for clinical practice, offering consistent evidence and straightforward recommendations. In 2004, we evaluated the conclusions from Cochrane systematic reviews of randomized controlled trials in terms of their recommendations for clinical practice and found that 47.83% of them had insufficient evidence for use in clinical practice. We proposed to reanalyze the reviews to evaluate whether this percentage had significantly decreased. Methods A cross-sectional study of systematic reviews published in the Cochrane Library (Issue 7, 2011) was conducted. We randomly selected reviews across all 52 Cochrane Collaborative Review Groups. Results We analyzed 1128 completed systematic reviews. Of these, 45.30% concluded that the interventions studied were likely to be beneficial, of which only 2.04% recommended no further research. In total, 45.04% of the reviews reported that the evidence did not support either benefit or harm, of which 0.8% did not recommend further studies and 44.24% recommended additional studies; the latter has decreased from our previous study with a difference of 3.59%. Conclusion Only a small number of the Cochrane collaboration's systematic reviews support clinical interventions with no need for additional research. A larger number of high-quality randomized clinical trials are necessary to change the 'insufficient evidence' scenario for clinical practice illustrated by the Cochrane database. It is recommended that we should produce higher-quality primary studies in active collaboration and consultation with global scholars and societies so that this can represent a major component of methodological advance in this context. © 2012 John Wiley & Sons Ltd.

A PRISMA assessment of the reporting quality of systematic reviews in orthodontics

Abstract Objectives: To assess the reporting quality of Cochrane and non-Cochrane systematic reviews (SR) in orthodontics and to compare the reporting quality (PRISMA score) with methodological quality (AMSTAR criteria). Materials and Methods: Systematic reviews (n  =  109) published between January 2000 and July 2011 in five leading orthodontic journals were identified and included. The quality of reporting of the included reviews was assessed by two authors in accordance with the PRISMA guidelines. Each article was assigned a cumulative grade based on fulfillment of the applicable criteria, and an overall percentage score was assigned. Descriptive statistics and simple and multiple linear regression analyses were undertaken. Results: The mean overall PRISMA score was 64.1% (95% confidence interval [CI], 62%-65%). The quality of reporting was considerably better in reviews published in the Cochrane Database of Systematic Reviews (P < .001) than in non-Cochrane reviews. Both multivariable and univariable analysis indicated that journal of publication and number of authors was significantly associated with the PRISMA score. The association between AMSTAR score and modified PRISMA score was also found to be highly statistically significant. Conclusion: Compliance of orthodontic SRs published in orthodontic journals with PRISMA guidelines was deficient in several areas. The quality of reporting assessed using PRISMA guidelines was significantly better in orthodontic SRs published in the Cochrane Database of Systematic Reviews.

Updated method guidelines for cochrane musculoskeletal group systematic reviews and metaanalyses.

The Cochrane Musculoskeletal Group (CMSG), one of 53 groups of the not-for-profit, international Cochrane Collaboration, prepares, maintains, and disseminates systematic reviews of treatments for musculoskeletal diseases. It is important that authors conducting CMSG reviews and the readers of our reviews be aware of and use updated, state-of-the-art systematic review methodology. One hundred sixty reviews have been published. Previous method guidelines for systematic reviews of interventions in the musculoskeletal field published in 2006 have been substantially updated to incorporate methodological advances that are mandatory or highly desirable in Cochrane reviews and knowledge translation advances. The methodological advances include new guidance on searching, new risk-of-bias assessment, grading the quality of the evidence, the new Summary of Findings table, and comparative effectiveness using network metaanalysis. Method guidelines specific to musculoskeletal disorders are provided by CMSG editors for various aspects of undertaking a systematic review. These method guidelines will help improve the quality of reporting and ensure high standards of conduct as well as consistency across CMSG reviews.

Re-examining the evidence in radiation dermatitis management literature : an overview and a critical appraisal of systematic reviews

Purpose: To provide an overview and a critical appraisal of systematic reviews (SRs) of published interventions for the prevention/management of radiation dermatitis. Methods and Materials: We searched Medline, CINAHL, Embase, and the Cochrane Library. We also manually searched through individual reference lists of potentially eligible articles and a number of key journals in the topic area. Two authors screened all potential articles and included eligible SRs. Two authors critically appraised and extracted key findings from the included reviews using AMSTAR (the measurement tool for “assessment of multiple systematic reviews”). Results: Of 1837 potential titles, 6 SRs were included. A number of interventions have been reported to be potentially beneficial for managing radiation dermatitis. Interventions evaluated in these reviews included skin care advice, steroidal/nonsteroidal topical agents, systemic therapies, modes of radiation delivery, and dressings. However, all the included SRs reported that there is insufficient evidence supporting any single effective intervention. The methodological quality of the included studies varied, and methodological shortfalls in these reviews might create biases to the overall results or recommendations for clinical practice. Conclusions: An up-to-date high-quality SR in the prevention/management of radiation dermatitis is needed to guide practice and direction for future research. We recommend that clinicians or guideline developers critically evaluate the information of SRs in their decision making.