Reporting completeness of abstracts of systematic reviews published in leading dental specialty journals

The aim of this study was to investigate the reporting completeness of systematic review (SR) abstracts in leading dental specialty journals. Electronic and supplementary hand searching were undertaken to identify SRs published in seven dental specialty journals and in the Cochrane Database of Systematic Reviews. Abstract reporting completeness was evaluated using a checklist derived from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (prisma) guidelines. Descriptive statistics followed by univariate and multivariate analyses were conducted. Two-hundred and eighteen SR abstracts were identified. Reporting of interventions (94%), objectives (96%), data sources (81%), eligibility criteria (77%), and conclusions (97%) was adequate in the majority of reviews. However, inadequate reporting of participants (18%), results (42%), effect size (14%), level of significance (60%), and trial registration (100%) was commonplace. The mean overall reporting score was 79.1% (95% CI, 77.6-80.6). Only journal of publication was a significant predictor of overall reporting, with inferior results for all journals relative to Cochrane reviews, with scores ranging from -4.3% (95% CI, -8.74 to 0.08) to -35.6% (95% CI, -42.0 to -24.3) for the International Journal of Prosthodontics and the British Journal of Oral and Maxillofacial Surgery, respectively. Improved reporting of dental SR abstracts is needed and should be encouraged, as these abstracts may underpin influential clinical decisions.

Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?

The aims of this study were to assess and compare the methodological quality of Cochrane and non-Cochrane systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) using AMSTAR and to compare the prevalence of meta-analysis in both review types. A literature search was undertaken to identify SRs that consisted of hand-searching five major orthodontic journals [American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics and Orthodontics and Craniofacial Research (February 2002 to July 2011)] and the Cochrane Database of Systematic Reviews from January 2000 to July 2011. Methodological quality of the included reviews was gauged using the AMSTAR tool involving 11 key methodological criteria with a score of 0 or 1 given for each criterion. A cumulative grade was given for the paper overall (0-11); an overall score of 4 or less represented poor methodological quality, 5-8 was considered fair and 9 or greater was deemed to be good. In total, 109 SRs were identified in the five major journals and on the CDSR. Of these, 26 (23.9%) were in the CDSR. The mean overall AMSTAR score was 6.2 with 21.1% of reviews satisfying 9 or more of the 11 criteria; a similar prevalence of poor reviews (22%) was also noted. Multiple linear regression indicated that reviews published in the CDSR (P < 0.01); and involving meta-analysis (β = 0.50, 95% confidence interval 0.72, 2.07, P < 0.001) showed greater concordance with AMSTAR.

Effectiveness and cost-effectiveness of traditional and new partner notification technologies for curable sexually transmitted infections: observational study, systematic reviews and mathematical modelling

BACKGROUND Partner notification is essential to the comprehensive case management of sexually transmitted infections. Systematic reviews and mathematical modelling can be used to synthesise information about the effects of new interventions to enhance the outcomes of partner notification. OBJECTIVE To study the effectiveness and cost-effectiveness of traditional and new partner notification technologies for curable sexually transmitted infections (STIs). DESIGN Secondary data analysis of clinical audit data; systematic reviews of randomised controlled trials (MEDLINE, EMBASE and Cochrane Central Register of Controlled Trials) published from 1 January 1966 to 31 August 2012 and of studies of health-related quality of life (HRQL) [MEDLINE, EMBASE, ISI Web of Knowledge, NHS Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA)] published from 1 January 1980 to 31 December 2011; static models of clinical effectiveness and cost-effectiveness; and dynamic modelling studies to improve parameter estimation and examine effectiveness. SETTING General population and genitourinary medicine clinic attenders. PARTICIPANTS Heterosexual women and men. INTERVENTIONS Traditional partner notification by patient or provider referral, and new partner notification by expedited partner therapy (EPT) or its UK equivalent, accelerated partner therapy (APT). MAIN OUTCOME MEASURES Population prevalence; index case reinfection; and partners treated per index case. RESULTS Enhanced partner therapy reduced reinfection in index cases with curable STIs more than simple patient referral [risk ratio (RR) 0.71; 95% confidence interval (CI) 0.56 to 0.89]. There are no randomised trials of APT. The median number of partners treated for chlamydia per index case in UK clinics was 0.60. The number of partners needed to treat to interrupt transmission of chlamydia was lower for casual than for regular partners. In dynamic model simulations, > 10% of partners are chlamydia positive with look-back periods of up to 18 months. In the presence of a chlamydia screening programme that reduces population prevalence, treatment of current partners achieves most of the additional reduction in prevalence attributable to partner notification. Dynamic model simulations show that cotesting and treatment for chlamydia and gonorrhoea reduce the prevalence of both STIs. APT has a limited additional effect on prevalence but reduces the rate of index case reinfection. Published quality-adjusted life-year (QALY) weights were of insufficient quality to be used in a cost-effectiveness study of partner notification in this project. Using an intermediate outcome of cost per infection diagnosed, doubling the efficacy of partner notification from 0.4 to 0.8 partners treated per index case was more cost-effective than increasing chlamydia screening coverage. CONCLUSIONS There is evidence to support the improved clinical effectiveness of EPT in reducing index case reinfection. In a general heterosexual population, partner notification identifies new infected cases but the impact on chlamydia prevalence is limited. Partner notification to notify casual partners might have a greater impact than for regular partners in genitourinary clinic populations. Recommendations for future research are (1) to conduct randomised controlled trials using biological outcomes of the effectiveness of APT and of methods to increase testing for human immunodeficiency virus (HIV) and STIs after APT; (2) collection of HRQL data should be a priority to determine QALYs associated with the sequelae of curable STIs; and (3) standardised parameter sets for curable STIs should be developed for mathematical models of STI transmission that are used for policy-making. FUNDING The National Institute for Health Research Health Technology Assessment programme.

Updated method guidelines for cochrane musculoskeletal group systematic reviews and metaanalyses.

The Cochrane Musculoskeletal Group (CMSG), one of 53 groups of the not-for-profit, international Cochrane Collaboration, prepares, maintains, and disseminates systematic reviews of treatments for musculoskeletal diseases. It is important that authors conducting CMSG reviews and the readers of our reviews be aware of and use updated, state-of-the-art systematic review methodology. One hundred sixty reviews have been published. Previous method guidelines for systematic reviews of interventions in the musculoskeletal field published in 2006 have been substantially updated to incorporate methodological advances that are mandatory or highly desirable in Cochrane reviews and knowledge translation advances. The methodological advances include new guidance on searching, new risk-of-bias assessment, grading the quality of the evidence, the new Summary of Findings table, and comparative effectiveness using network metaanalysis. Method guidelines specific to musculoskeletal disorders are provided by CMSG editors for various aspects of undertaking a systematic review. These method guidelines will help improve the quality of reporting and ensure high standards of conduct as well as consistency across CMSG reviews.

Systematic reviews published in higher impact clinical journals were of higher quality.

OBJECTIVES To compare the methodological quality of systematic reviews (SRs) published in high- and low-impact factor (IF) Core Clinical Journals. In addition, we aimed to record the implementation of aspects of reporting, including Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram, reasons for study exclusion, and use of recommendations for interventions such as Grading of Recommendations Assessment, Development and Evaluation (GRADE). STUDY DESIGN AND SETTING We searched PubMed for systematic reviews published in Core Clinical Journals between July 1 and December 31, 2012. We evaluated the methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR) tool. RESULTS Over the 6-month period, 327 interventional systematic reviews were identified with a mean AMSTAR score of 63.3% (standard deviation, 17.1%), when converted to a percentage scale. We identified deficiencies in relation to a number of quality criteria including delineation of excluded studies and assessment of publication bias. We found that SRs published in higher impact journals were undertaken more rigorously with higher percentage AMSTAR scores (per IF unit: β = 0.68%; 95% confidence interval: 0.32, 1.04; P < 0.001), a discrepancy likely to be particularly relevant when differences in IF are large. CONCLUSION Methodological quality of SRs appears to be better in higher impact journals. The overall quality of SRs published in many Core Clinical Journals remains suboptimal.

Systematic reviews and meta-analyses of randomized trials: principles and pitfalls

Systematic reviews and meta-analyses allow for a more transparent and objective appraisal of the evidence. They may decrease the number of false-negative results and prevent delays in the introduction of effective interventions into clinical practice. However, as for any other tool, their misuse can result in severely misleading results. In this article, we discuss the main steps that should be taken when conducting systematic reviews and meta-analyses, namely the preparation of a review protocol, identification of eligible trials, and data extraction, pooling of treatment effects across trials, investigation of potential reasons for differences in treatment effects across trials, and complete reporting of the review methods and findings. We also discuss common pitfalls that should be avoided, including the use of quality assessment tools to derive summary quality scores, pooling of data across trials as if they belonged to a single large trial, and inappropriate uses of meta-regression that could result in misleading estimates of treatment effects because of regression to the mean or the ecological fallacy. If conducted and reported properly, systematic reviews and meta-analyses will increase our understanding of the strengths and weaknesses of the available evidence, which may eventually facilitate clinical decision making.

Time relevance, citation of reporting guidelines, and breadth of literature search in systematic reviews in orthodontics

INTRODUCTION As the importance of systematic review (SR) conclusions relies upon the scientific rigor of methods and the currency of evidence, we aimed to investigate the currency of orthodontic SRs using as proxy the time from the initial search to publication. Additionally, SR information regarding reporting guidelines, registration, and literature searches were recorded when available. MATERIALS AND METHODS A systematic PubMed search was carried out using the Clinical Queries page to identify orthodontic SRs cited between 1 January 2008 and 7 November 2013. Data related to reporting guidelines, review registration, dates of review processing, literature search, and abstract reporting were retrieved and classified by journal type. Survival analysis was used to assess the time to reach predefined manuscript stages for orthodontic and non-orthodontic journals. RESULTS One hundred twenty seven of the originally identified 585 SRs were considered eligible. The median interval from search until publication was 13.2 months (interquartile range: IQR = 9.7 months) irrespective of the journal type. There was evidence (P = 0.05) that SRs published by non-orthodontic journals appeared in PubMed faster than in orthodontic journals (non-orthodontic: median = 6.5 months; IQR = 5.7 months; orthodontic: median = 10.2 months; IQR = 5.6 months) from submission to publication and from acceptance to publication (non-orthodontic: median = 1.5 months; IQR = 2.4 months; orthodontic: median = 6.0 months; IQR = 6.2 months; P < 0.001). More than half of these SRs did not cite adherence to any reporting guidelines, whereas all but five studies were not prospectively registered. Search of unpublished research was undertaken in approximately 21 per cent and 29 per cent of the SRs published in non-orthodontic and orthodontic periodicals, respectively. CONCLUSIONS This study indicates that SR users should be aware that median time for orthodontic SRs from search to publication is 13.2 months. SRs published in non-orthodontic journals are likely to be more current in terms of submission until time to publication and acceptance until time to publication compared with those published in orthodontic journals.

Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews

OBJECTIVES To assess discrepancies in the analyzed outcomes between protocols and published reviews within Cochrane oral health systematic reviews (COHG) on the Cochrane Database of Systematic Reviews (CDSR). STUDY DESIGN AND SETTING All COHG systematic reviews on the CDSR and the corresponding protocols were retrieved in November 2014 and information on the reported outcomes was recorded. Data was collected at the systematic review level by two reviewers independently. RESULTS One hundred and fifty two reviews were included. In relation to primary outcomes, 11.2% were downgraded to secondary outcomes, 9.9% were omitted altogether in the final publication and new primary outcomes were identified in 18.4% of publications. For secondary outcomes, 2% were upgraded to primary, 12.5% were omitted and 30.9% were newly introduced in the publication. Overall, 45.4% of reviews had at least one discrepancy when compared to the protocol; these were reported in 14.5% reviews. The number of review updates appears to be associated with discrepancies between final review and protocol (OR: 3.18, 95% CI: 1.77, 5.74, p<0.001). The risk of reporting significant results was lower for both downgraded outcomes [RR: 0.52, 95% CI: 0.17, 1.58, p = 0.24] and upgraded or newly introduced outcomes [RR: 0.77, 95% CI: 0.36, 1.64, p = 0.50] compared to outcomes with no discrepancies. The risk of reporting significant results was higher for upgraded or newly introduced outcomes compared to downgraded outcomes (RR = 1.19, 95% CI: 0.65, 2.16, p = 0.57). None of the comparisons reached statistical significance. CONCLUSION While no evidence of selective outcome reporting was found in this study, based on the present analysis of SRs published within COHG systematic reviews, discrepancies between outcomes in pre-published protocols and final reviews continue to be common. Solutions such as the use of standardized outcomes to reduce the prevalence of this issue may need to be explored.

The Quality of the Evidence According to GRADE Is Predominantly Low or Very Low in Oral Health Systematic Reviews

OBJECTIVES The main objective was to assess the credibility of the evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) in oral health systematic reviews on the Cochrane Database of Systematic Reviews (CDSR) and elsewhere. STUDY DESIGN AND SETTING Systematic Reviews or meta-analyses (January 2008-December 2013) from 14 high impact general dental and specialty dental journals and the Cochrane Database of Systematic Reviews were screened for meta-analyses. Data was collected at the systematic review, meta-analysis and trial level. Two reviewers applied and agreed on the GRADE rating for the selected meta-analyses. RESULTS From the 510 systematic reviews initially identified 91 reviews (41 Cochrane and 50 non-Cochrane) were eligible for inclusion. The quality of evidence was high in 2% and moderate in 18% of the included meta-analyses with no difference between Cochrane and non-Cochrane reviews, journal impact factor or year of publication. The most common domains prompting downgrading of the evidence were study limitations (risk of bias) and imprecision (risk of play of chance). CONCLUSION The quality of the evidence in oral health assessed using GRADE is predominantly low or very low suggesting a pressing need for more randomised clinical trials and other studies of higher quality in order to inform clinical decisions thereby reducing the risk of instituting potentially ineffective and/or harmful therapies.

Use of quality assessment tools in systematic reviews was varied and inconsistent

OBJECTIVES To assess the use of quality assessment tools among a cross-section of systematic reviews (SRs) and to further evaluate whether quality was used as a parameter in the decision to include primary studies within subsequent meta-analysis. STUDY DESIGN AND SETTING We searched PubMed for SRs (interventional, observational, and diagnostic) published in Core Clinical Journals between January 1 and March 31, 2014. RESULTS Three hundred nine SRs were identified. Quality assessment was undertaken in 222 (71.8%) with isolated use of the Cochrane risk of bias tool (26.1%, n = 58) and the Newcastle-Ottawa Scale (15.3%, n = 34) most common. A threshold level of primary study quality for subsequent meta-analysis was used in 12.9% (40 of 309) of reviews. Overall, fifty-four combinations of quality assessment tools were identified with a similar preponderance of tools used among observational and interventional reviews. Multiple tools were used in 11.7% (n = 36) of SRs overall. CONCLUSION We found that quality assessment tools were used in a majority of SRs; however, a threshold level of quality for meta-analysis was stipulated in just 12.9% (n = 40). This cross-sectional analysis provides further evidence of the need for more active or intuitive editorial processes to enhance the reporting of SRs.

Reporting and handling missing outcome data in mental health: a systematic review of Cochrane systematic reviews and meta-analyses

OBJECTIVES The purpose of the study was to provide empirical evidence about the reporting of methodology to address missing outcome data and the acknowledgement of their impact in Cochrane systematic reviews in the mental health field. METHODS Systematic reviews published in the Cochrane Database of Systematic Reviews after January 1, 2009 by three Cochrane Review Groups relating to mental health were included. RESULTS One hundred ninety systematic reviews were considered. Missing outcome data were present in at least one included study in 175 systematic reviews. Of these 175 systematic reviews, 147 (84%) accounted for missing outcome data by considering a relevant primary or secondary outcome (e.g., dropout). Missing outcome data implications were reported only in 61 (35%) systematic reviews and primarily in the discussion section by commenting on the amount of the missing outcome data. One hundred forty eligible meta-analyses with missing data were scrutinized. Seventy-nine (56%) of them had studies with total dropout rate between 10 and 30%. One hundred nine (78%) meta-analyses reported to have performed intention-to-treat analysis by including trials with imputed outcome data. Sensitivity analysis for incomplete outcome data was implemented in less than 20% of the meta-analyses. CONCLUSIONS Reporting of the techniques for handling missing outcome data and their implications in the findings of the systematic reviews are suboptimal.

The evidence from systematic reviews and meta-analyses published in orthodontic literature. Where do we stand?

AIM To analyse meta-analyses included in systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) focusing on orthodontic literature and to assess the quality of the existing evidence. MATERIALS AND METHODS Electronic searching was undertaken to identify SRs published in five major orthodontic journals and the CDSR between January 2000 and June 2014. Quality assessment of the overall body of evidence from meta-analyses was conducted using the Grading of Recommendations Assessment, Development and Evaluation working group (GRADE) tool. RESULTS One hundred and fifty-seven SRs were identified; meta-analysis was present in 43 of these (27.4 per cent). The highest proportion of SRs that included a meta-analysis was found in Orthodontics and Craniofacial Research (6/13; 46.1 per cent), followed by the CDSR (12/33; 36.4 per cent) and the American Journal of Orthodontics and Dentofacial Orthopaedics (15/44; 34.1 per cent). Class II treatment was the most commonly addressed topic within SRs in orthodontics (n = 18/157; 11.5 per cent). The number of trials combined to produce a summary estimate was small for most meta-analyses with a median of 4 (range: 2-52). Only 21 per cent (n = 9) of included meta-analyses were considered to have a high/moderate quality of evidence according to GRADE, while the majority were of low or very low quality (n = 34; 79.0 per cent). CONCLUSIONS Overall, approximately one quarter of orthodontic SRs included quantitative synthesis, with a median of four trials per meta-analysis. The overall quality of evidence from the selected orthodontic SRs was predominantly low to very low indicating the relative lack of high quality of evidence from SRs to inform clinical practice guidelines.