An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS)

Systematic reviews are not an assembly of anecdotes but a distillation of current best available evidence on a particular topic and as such have an important role to play in evidence-based healthcare. A substantial proportion of these systematic reviews focus on interventions, and are able to provide clinicians with the opportunity to understand and translate the best available evidence on the effects of these healthcare interventions into clinical practice. The importance of systematic reviews in summarising and identifying the gaps in evidence which might inform new research initiatives is also widely acknowledged. Their potential impact on practice and research makes their methodological quality especially important as it may directly influence their utility for clinicians, patients and policy makers. The objectives of this study were to identify systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS) and to evaluate their methodological quality using the evaluation tool, AMSTAR.

A PRISMA assessment of the reporting quality of systematic reviews in orthodontics

Abstract Objectives: To assess the reporting quality of Cochrane and non-Cochrane systematic reviews (SR) in orthodontics and to compare the reporting quality (PRISMA score) with methodological quality (AMSTAR criteria). Materials and Methods: Systematic reviews (n  =  109) published between January 2000 and July 2011 in five leading orthodontic journals were identified and included. The quality of reporting of the included reviews was assessed by two authors in accordance with the PRISMA guidelines. Each article was assigned a cumulative grade based on fulfillment of the applicable criteria, and an overall percentage score was assigned. Descriptive statistics and simple and multiple linear regression analyses were undertaken. Results: The mean overall PRISMA score was 64.1% (95% confidence interval [CI], 62%-65%). The quality of reporting was considerably better in reviews published in the Cochrane Database of Systematic Reviews (P < .001) than in non-Cochrane reviews. Both multivariable and univariable analysis indicated that journal of publication and number of authors was significantly associated with the PRISMA score. The association between AMSTAR score and modified PRISMA score was also found to be highly statistically significant. Conclusion: Compliance of orthodontic SRs published in orthodontic journals with PRISMA guidelines was deficient in several areas. The quality of reporting assessed using PRISMA guidelines was significantly better in orthodontic SRs published in the Cochrane Database of Systematic Reviews.

Can decision making in general surgery be based on evidence? An empirical study of Cochrane Reviews

BACKGROUND: This empirical study analyzes the current status of Cochrane Reviews (CRs) and their strength of recommendation for evidence-based decision making in the field of general surgery. METHODS: Systematic literature search of the Cochrane Database of Systematic Reviews and the Cochrane Collaboration's homepage to identify available CRs on surgical topics. Quantitative and qualitative characteristics, utilization, and formulated treatment recommendations were evaluated by 2 independent reviewers. Association of review characteristics with treatment recommendation was analyzed using univariate and multivariate logistic regression models. RESULTS: Ninety-three CRs, including 1,403 primary studies and 246,473 patients, were identified. Mean number of included primary studies per CR was 15.1 (standard deviation [SD] 14.5) including 2,650 (SD 3,340) study patients. Two and a half (SD 8.3) nonrandomized trials were included per analyzed CR. Seventy-two (77%) CRs were published or updated in 2005 or later. Explicit treatment recommendations were given in 45 (48%). Presence of a treatment recommendation was associated with the number of included primary studies and the proportion of randomized studies. Utilization of surgical CRs remained low and showed large inter-country differences. The most surgical CRs were accessed in UK, USA, and Australia, followed by several Western and Eastern European countries. CONCLUSION: Only a minority of available CRs address surgical questions and their current usage is low. Instead of unsystematically increasing the number of surgical CRs it would be far more efficient to focus the review process on relevant surgical questions. Prioritization of CRs needs valid methods which should be developed by the scientific surgical community.

Reporting completeness of abstracts of systematic reviews published in leading dental specialty journals

The aim of this study was to investigate the reporting completeness of systematic review (SR) abstracts in leading dental specialty journals. Electronic and supplementary hand searching were undertaken to identify SRs published in seven dental specialty journals and in the Cochrane Database of Systematic Reviews. Abstract reporting completeness was evaluated using a checklist derived from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (prisma) guidelines. Descriptive statistics followed by univariate and multivariate analyses were conducted. Two-hundred and eighteen SR abstracts were identified. Reporting of interventions (94%), objectives (96%), data sources (81%), eligibility criteria (77%), and conclusions (97%) was adequate in the majority of reviews. However, inadequate reporting of participants (18%), results (42%), effect size (14%), level of significance (60%), and trial registration (100%) was commonplace. The mean overall reporting score was 79.1% (95% CI, 77.6-80.6). Only journal of publication was a significant predictor of overall reporting, with inferior results for all journals relative to Cochrane reviews, with scores ranging from -4.3% (95% CI, -8.74 to 0.08) to -35.6% (95% CI, -42.0 to -24.3) for the International Journal of Prosthodontics and the British Journal of Oral and Maxillofacial Surgery, respectively. Improved reporting of dental SR abstracts is needed and should be encouraged, as these abstracts may underpin influential clinical decisions.

Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?

The aims of this study were to assess and compare the methodological quality of Cochrane and non-Cochrane systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) using AMSTAR and to compare the prevalence of meta-analysis in both review types. A literature search was undertaken to identify SRs that consisted of hand-searching five major orthodontic journals [American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics and Orthodontics and Craniofacial Research (February 2002 to July 2011)] and the Cochrane Database of Systematic Reviews from January 2000 to July 2011. Methodological quality of the included reviews was gauged using the AMSTAR tool involving 11 key methodological criteria with a score of 0 or 1 given for each criterion. A cumulative grade was given for the paper overall (0-11); an overall score of 4 or less represented poor methodological quality, 5-8 was considered fair and 9 or greater was deemed to be good. In total, 109 SRs were identified in the five major journals and on the CDSR. Of these, 26 (23.9%) were in the CDSR. The mean overall AMSTAR score was 6.2 with 21.1% of reviews satisfying 9 or more of the 11 criteria; a similar prevalence of poor reviews (22%) was also noted. Multiple linear regression indicated that reviews published in the CDSR (P < 0.01); and involving meta-analysis (β = 0.50, 95% confidence interval 0.72, 2.07, P < 0.001) showed greater concordance with AMSTAR.

Effectiveness and cost-effectiveness of traditional and new partner notification technologies for curable sexually transmitted infections: observational study, systematic reviews and mathematical modelling

BACKGROUND Partner notification is essential to the comprehensive case management of sexually transmitted infections. Systematic reviews and mathematical modelling can be used to synthesise information about the effects of new interventions to enhance the outcomes of partner notification. OBJECTIVE To study the effectiveness and cost-effectiveness of traditional and new partner notification technologies for curable sexually transmitted infections (STIs). DESIGN Secondary data analysis of clinical audit data; systematic reviews of randomised controlled trials (MEDLINE, EMBASE and Cochrane Central Register of Controlled Trials) published from 1 January 1966 to 31 August 2012 and of studies of health-related quality of life (HRQL) [MEDLINE, EMBASE, ISI Web of Knowledge, NHS Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA)] published from 1 January 1980 to 31 December 2011; static models of clinical effectiveness and cost-effectiveness; and dynamic modelling studies to improve parameter estimation and examine effectiveness. SETTING General population and genitourinary medicine clinic attenders. PARTICIPANTS Heterosexual women and men. INTERVENTIONS Traditional partner notification by patient or provider referral, and new partner notification by expedited partner therapy (EPT) or its UK equivalent, accelerated partner therapy (APT). MAIN OUTCOME MEASURES Population prevalence; index case reinfection; and partners treated per index case. RESULTS Enhanced partner therapy reduced reinfection in index cases with curable STIs more than simple patient referral [risk ratio (RR) 0.71; 95% confidence interval (CI) 0.56 to 0.89]. There are no randomised trials of APT. The median number of partners treated for chlamydia per index case in UK clinics was 0.60. The number of partners needed to treat to interrupt transmission of chlamydia was lower for casual than for regular partners. In dynamic model simulations, > 10% of partners are chlamydia positive with look-back periods of up to 18 months. In the presence of a chlamydia screening programme that reduces population prevalence, treatment of current partners achieves most of the additional reduction in prevalence attributable to partner notification. Dynamic model simulations show that cotesting and treatment for chlamydia and gonorrhoea reduce the prevalence of both STIs. APT has a limited additional effect on prevalence but reduces the rate of index case reinfection. Published quality-adjusted life-year (QALY) weights were of insufficient quality to be used in a cost-effectiveness study of partner notification in this project. Using an intermediate outcome of cost per infection diagnosed, doubling the efficacy of partner notification from 0.4 to 0.8 partners treated per index case was more cost-effective than increasing chlamydia screening coverage. CONCLUSIONS There is evidence to support the improved clinical effectiveness of EPT in reducing index case reinfection. In a general heterosexual population, partner notification identifies new infected cases but the impact on chlamydia prevalence is limited. Partner notification to notify casual partners might have a greater impact than for regular partners in genitourinary clinic populations. Recommendations for future research are (1) to conduct randomised controlled trials using biological outcomes of the effectiveness of APT and of methods to increase testing for human immunodeficiency virus (HIV) and STIs after APT; (2) collection of HRQL data should be a priority to determine QALYs associated with the sequelae of curable STIs; and (3) standardised parameter sets for curable STIs should be developed for mathematical models of STI transmission that are used for policy-making. FUNDING The National Institute for Health Research Health Technology Assessment programme.

Researcher allegiance in psychotherapy outcome research: an overview of reviews

Researcher allegiance (RA) is widely discussed as a risk of bias in psychotherapy outcome research. The relevance attached to RA bias is related to meta-analyses demonstrating an association of RA with treatment effects. However, recent meta-analyses have yielded mixed results. To provide more clarity on the magnitude and robustness of the RA-outcome association this article reports on a meta-meta-analysis summarizing all available meta-analytic estimates of the RA-outcome association. Random-effects methods were used. Primary study overlap was controlled. Thirty meta-analyses were included. The mean RA-outcome association was r=.262 (p=.002, I(2)=28.98%), corresponding to a moderate effect size. The RA-outcome association was robust across several moderating variables including characteristics of treatment, population, and the type of RA assessment. Allegiance towards the RA bias hypothesis moderated the RA-outcome association. The findings of this meta-meta-analysis suggest that the RA-outcome association is substantial and robust. Implications for psychotherapy outcome research are discussed.

Assimilation and contrast in web product reviews: devaluing the recommendation of a proficient but dissimilar reviewer

When viewing web-consumer reviews consumers encounter the reviewers in an anonymous environment. Although their interactions are only virtual they still exchange social information, e.g. often reviewers refer to their proficiency or consumption motives within the review texts. Do these social information harm the viewers’ perception of the recommended products? The present study addresses this question by applying the paradigm of social comparison (Mussweiler, 2003) to web-consumer reviews. In a laboratory experiment with a student sample (n = 120) we manipulated the perceived similarity between reviewer and viewer and the perceived proficiency of the reviewer. A measurement of achievement goals (Elliott & McGregor, 2001) and average number of hours of study prior to the experiment allowed to introduce the reviewer as high [low] in proficiency and similar [dissimilar] in achievement goals. As predicted, the viewer’s evaluation of the recommended products differed as a function of this social information. Contrasting with the reviewer led to devaluing the products recommended by a proficient but dissimilar reviewer. However, against our prediction social comparison with the reviewer did not affect the viewer`s self-evaluation. Whether social information in web-product reviews affects the viewer`s self-evaluation and induces both social comparison processes remains an open question. Future studies aim to address this by manipulating the informational focus of the viewer, rather than the perceived similarity between viewer and reviewer. So far, the present study extends the application of social comparison to consumption environments and contributes to the understanding of the virtual social identity.

Updated method guidelines for cochrane musculoskeletal group systematic reviews and metaanalyses.

The Cochrane Musculoskeletal Group (CMSG), one of 53 groups of the not-for-profit, international Cochrane Collaboration, prepares, maintains, and disseminates systematic reviews of treatments for musculoskeletal diseases. It is important that authors conducting CMSG reviews and the readers of our reviews be aware of and use updated, state-of-the-art systematic review methodology. One hundred sixty reviews have been published. Previous method guidelines for systematic reviews of interventions in the musculoskeletal field published in 2006 have been substantially updated to incorporate methodological advances that are mandatory or highly desirable in Cochrane reviews and knowledge translation advances. The methodological advances include new guidance on searching, new risk-of-bias assessment, grading the quality of the evidence, the new Summary of Findings table, and comparative effectiveness using network metaanalysis. Method guidelines specific to musculoskeletal disorders are provided by CMSG editors for various aspects of undertaking a systematic review. These method guidelines will help improve the quality of reporting and ensure high standards of conduct as well as consistency across CMSG reviews.