Parochialism or self-consciousness? Internationality in medical history journals 1997-2006

Research councils, universities and funding agencies are increasingly asking for tools to measure the quality of research in the humanities. One of their preferred methods is a ranking of journals according to their supposed level of internationality. Our quantitative survey of seventeen major journals of medical history reveals the futility of such an approach. Most journals have a strong national character with a dominance of native language, authors and topics. The most common case is a paper written by a local author in his own language on a national subject regarding the nineteenth or twentieth century. American and British journals are taken notice of internationally but they only rarely mention articles from other history of medicine journals. Continental European journals show a more international review of literature, but are in their turn not noticed globally. Increasing specialisation and fragmentation has changed the role of general medical history journals. They run the risk of losing their function as international platforms of discourse on general and theoretical issues and major trends in historiography, to international collections of papers. Journal editors should therefore force their authors to write a more international report, and authors should be encouraged to submit papers of international interest and from a more general, transnational and methodological point of view.

Sample sizes of studies on diagnostic accuracy: literature survey

OBJECTIVES: To determine sample sizes in studies on diagnostic accuracy and the proportion of studies that report calculations of sample size. DESIGN: Literature survey. DATA SOURCES: All issues of eight leading journals published in 2002. METHODS: Sample sizes, number of subgroup analyses, and how often studies reported calculations of sample size were extracted. RESULTS: 43 of 8999 articles were non-screening studies on diagnostic accuracy. The median sample size was 118 (interquartile range 71-350) and the median prevalence of the target condition was 43% (27-61%). The median number of patients with the target condition--needed to calculate a test's sensitivity--was 49 (28-91). The median number of patients without the target condition--needed to determine a test's specificity--was 76 (27-209). Two of the 43 studies (5%) reported a priori calculations of sample size. Twenty articles (47%) reported results for patient subgroups. The number of subgroups ranged from two to 19 (median four). No studies reported that sample size was calculated on the basis of preplanned analyses of subgroups. CONCLUSION: Few studies on diagnostic accuracy report considerations of sample size. The number of participants in most studies on diagnostic accuracy is probably too small to analyse variability of measures of accuracy across patient subgroups.

Bibliographic study showed improving methodology of meta-analyses published in leading journals 1993-2002

OBJECTIVE: To assess the methodology of meta-analyses published in leading general and specialist medical journals over a 10-year period. STUDY DESIGN AND SETTING: Volumes 1993-2002 of four general medicine journals and four specialist journals were searched by hand for meta-analyses including at least five controlled trials. Characteristics were assessed using a standardized questionnaire. RESULTS: A total of 272 meta-analyses, which included a median of 11 trials (range 5-195), were assessed. Most (81%) were published in general medicine journals. The median (range) number of databases searched increased from 1 (1-9) in 1993/1994 to 3.5 (1-21) in 2001/2002, P<0.0001. The proportion of meta-analyses including searches by hand (10% in 1993/1994, 25% in 2001/2002, P=0.005), searches of the grey literature (29%, 51%, P=0.010 by chi-square test), and of trial registers (10%, 32%, P=0.025) also increased. Assessments of the quality of trials also became more common (45%, 70%, P=0.008), including whether allocation of patients to treatment groups had been concealed (24%, 60%, P=0.001). The methodological and reporting quality was consistently higher in general medicine compared to specialist journals. CONCLUSION: Many meta-analyses published in leading journals have important methodological limitations. The situation has improved in recent years but considerable room for further improvements remains.

Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?

The aims of this study were to assess and compare the methodological quality of Cochrane and non-Cochrane systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) using AMSTAR and to compare the prevalence of meta-analysis in both review types. A literature search was undertaken to identify SRs that consisted of hand-searching five major orthodontic journals [American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics and Orthodontics and Craniofacial Research (February 2002 to July 2011)] and the Cochrane Database of Systematic Reviews from January 2000 to July 2011. Methodological quality of the included reviews was gauged using the AMSTAR tool involving 11 key methodological criteria with a score of 0 or 1 given for each criterion. A cumulative grade was given for the paper overall (0-11); an overall score of 4 or less represented poor methodological quality, 5-8 was considered fair and 9 or greater was deemed to be good. In total, 109 SRs were identified in the five major journals and on the CDSR. Of these, 26 (23.9%) were in the CDSR. The mean overall AMSTAR score was 6.2 with 21.1% of reviews satisfying 9 or more of the 11 criteria; a similar prevalence of poor reviews (22%) was also noted. Multiple linear regression indicated that reviews published in the CDSR (P < 0.01); and involving meta-analysis (β = 0.50, 95% confidence interval 0.72, 2.07, P < 0.001) showed greater concordance with AMSTAR.

Are sample sizes clear and justified in RCTs published in dental journals?

Sample size calculations are advocated by the CONSORT group to justify sample sizes in randomized controlled trials (RCTs). The aim of this study was primarily to evaluate the reporting of sample size calculations, to establish the accuracy of these calculations in dental RCTs and to explore potential predictors associated with adequate reporting. Electronic searching was undertaken in eight leading specific and general dental journals. Replication of sample size calculations was undertaken where possible. Assumed variances or odds for control and intervention groups were also compared against those observed. The relationship between parameters including journal type, number of authors, trial design, involvement of methodologist, single-/multi-center study and region and year of publication, and the accuracy of sample size reporting was assessed using univariable and multivariable logistic regression. Of 413 RCTs identified, sufficient information to allow replication of sample size calculations was provided in only 121 studies (29.3%). Recalculations demonstrated an overall median overestimation of sample size of 15.2% after provisions for losses to follow-up. There was evidence that journal, methodologist involvement (OR = 1.97, CI: 1.10, 3.53), multi-center settings (OR = 1.86, CI: 1.01, 3.43) and time since publication (OR = 1.24, CI: 1.12, 1.38) were significant predictors of adequate description of sample size assumptions. Among journals JCP had the highest odds of adequately reporting sufficient data to permit sample size recalculation, followed by AJODO and JDR, with 61% (OR = 0.39, CI: 0.19, 0.80) and 66% (OR = 0.34, CI: 0.15, 0.75) lower odds, respectively. Both assumed variances and odds were found to underestimate the observed values. Presentation of sample size calculations in the dental literature is suboptimal; incorrect assumptions may have a bearing on the power of RCTs.

Time relevance, citation of reporting guidelines, and breadth of literature search in systematic reviews in orthodontics

INTRODUCTION As the importance of systematic review (SR) conclusions relies upon the scientific rigor of methods and the currency of evidence, we aimed to investigate the currency of orthodontic SRs using as proxy the time from the initial search to publication. Additionally, SR information regarding reporting guidelines, registration, and literature searches were recorded when available. MATERIALS AND METHODS A systematic PubMed search was carried out using the Clinical Queries page to identify orthodontic SRs cited between 1 January 2008 and 7 November 2013. Data related to reporting guidelines, review registration, dates of review processing, literature search, and abstract reporting were retrieved and classified by journal type. Survival analysis was used to assess the time to reach predefined manuscript stages for orthodontic and non-orthodontic journals. RESULTS One hundred twenty seven of the originally identified 585 SRs were considered eligible. The median interval from search until publication was 13.2 months (interquartile range: IQR = 9.7 months) irrespective of the journal type. There was evidence (P = 0.05) that SRs published by non-orthodontic journals appeared in PubMed faster than in orthodontic journals (non-orthodontic: median = 6.5 months; IQR = 5.7 months; orthodontic: median = 10.2 months; IQR = 5.6 months) from submission to publication and from acceptance to publication (non-orthodontic: median = 1.5 months; IQR = 2.4 months; orthodontic: median = 6.0 months; IQR = 6.2 months; P < 0.001). More than half of these SRs did not cite adherence to any reporting guidelines, whereas all but five studies were not prospectively registered. Search of unpublished research was undertaken in approximately 21 per cent and 29 per cent of the SRs published in non-orthodontic and orthodontic periodicals, respectively. CONCLUSIONS This study indicates that SR users should be aware that median time for orthodontic SRs from search to publication is 13.2 months. SRs published in non-orthodontic journals are likely to be more current in terms of submission until time to publication and acceptance until time to publication compared with those published in orthodontic journals.

The evidence from systematic reviews and meta-analyses published in orthodontic literature. Where do we stand?

AIM To analyse meta-analyses included in systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) focusing on orthodontic literature and to assess the quality of the existing evidence. MATERIALS AND METHODS Electronic searching was undertaken to identify SRs published in five major orthodontic journals and the CDSR between January 2000 and June 2014. Quality assessment of the overall body of evidence from meta-analyses was conducted using the Grading of Recommendations Assessment, Development and Evaluation working group (GRADE) tool. RESULTS One hundred and fifty-seven SRs were identified; meta-analysis was present in 43 of these (27.4 per cent). The highest proportion of SRs that included a meta-analysis was found in Orthodontics and Craniofacial Research (6/13; 46.1 per cent), followed by the CDSR (12/33; 36.4 per cent) and the American Journal of Orthodontics and Dentofacial Orthopaedics (15/44; 34.1 per cent). Class II treatment was the most commonly addressed topic within SRs in orthodontics (n = 18/157; 11.5 per cent). The number of trials combined to produce a summary estimate was small for most meta-analyses with a median of 4 (range: 2-52). Only 21 per cent (n = 9) of included meta-analyses were considered to have a high/moderate quality of evidence according to GRADE, while the majority were of low or very low quality (n = 34; 79.0 per cent). CONCLUSIONS Overall, approximately one quarter of orthodontic SRs included quantitative synthesis, with a median of four trials per meta-analysis. The overall quality of evidence from the selected orthodontic SRs was predominantly low to very low indicating the relative lack of high quality of evidence from SRs to inform clinical practice guidelines.