The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content

Background: The COSMIN checklist (COnsensus-based Standards for the selection of health status Measurement INstruments) was developed in an international Delphi study to evaluate the methodological quality of studies on measurement properties of health-related patient reported outcomes (HR-PROs). In this paper, we explain our choices for the design requirements and preferred statistical methods for which no evidence is available in the literature or on which the Delphi panel members had substantial discussion. Methods: The issues described in this paper are a reflection of the Delphi process in which 43 panel members participated. Results: The topics discussed are internal consistency (relevance for reflective and formative models, and distinction with unidimensionality), content validity (judging relevance and comprehensiveness), hypotheses testing as an aspect of construct validity (specificity of hypotheses), criterion validity (relevance for PROs), and responsiveness (concept and relation to validity, and (in) appropriate measures).Conclusions: We expect that this paper will contribute to a better understanding of the rationale behind the items, thereby enhancing the acceptance and use of the COSMIN checklist.

An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS)

Systematic reviews are not an assembly of anecdotes but a distillation of current best available evidence on a particular topic and as such have an important role to play in evidence-based healthcare. A substantial proportion of these systematic reviews focus on interventions, and are able to provide clinicians with the opportunity to understand and translate the best available evidence on the effects of these healthcare interventions into clinical practice. The importance of systematic reviews in summarising and identifying the gaps in evidence which might inform new research initiatives is also widely acknowledged. Their potential impact on practice and research makes their methodological quality especially important as it may directly influence their utility for clinicians, patients and policy makers. The objectives of this study were to identify systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS) and to evaluate their methodological quality using the evaluation tool, AMSTAR.

Quality of Reporting of Observational Longitudinal Research

Observational longitudinal research is particularly useful for assessing etiology and prognosis and for providing evidence for clinical decision making. However, there are no structured reporting requirements for studies of this design to assist authors, editors, and readers. The authors developed and tested a checklist of criteria related to threats to the internal and external validity of observational longitudinal studies. The checklist criteria concerned recruitment, data collection, biases, and data analysis and descriptive issues relevant to study rationale, study population, and generalizability. Two raters independently assessed 49 randomly selected articles describing stroke research published from 1999 to 2003 in six journals: American Journal of Epidemiology, Journal of Epidemiology and Community Health, Stroke, Annals of Neurology, Archives of Physical Medicine and Rehabilitation, and American Journal of Physical Medicine and Rehabilitation. On average, 17 of the 33 checklist criteria were reported. Criteria describing the study design were better reported than those related to internal validity. No relation was found between study type (etiologic or prognostic) or word count and quality of reporting. A flow diagram for summarizing participant flow through a study was developed. Editors and authors should consider using a checklist and flow diagram when reporting on observational longitudinal research.

A PRISMA assessment of the reporting quality of systematic reviews in orthodontics

Abstract Objectives: To assess the reporting quality of Cochrane and non-Cochrane systematic reviews (SR) in orthodontics and to compare the reporting quality (PRISMA score) with methodological quality (AMSTAR criteria). Materials and Methods: Systematic reviews (n  =  109) published between January 2000 and July 2011 in five leading orthodontic journals were identified and included. The quality of reporting of the included reviews was assessed by two authors in accordance with the PRISMA guidelines. Each article was assigned a cumulative grade based on fulfillment of the applicable criteria, and an overall percentage score was assigned. Descriptive statistics and simple and multiple linear regression analyses were undertaken. Results: The mean overall PRISMA score was 64.1% (95% confidence interval [CI], 62%-65%). The quality of reporting was considerably better in reviews published in the Cochrane Database of Systematic Reviews (P < .001) than in non-Cochrane reviews. Both multivariable and univariable analysis indicated that journal of publication and number of authors was significantly associated with the PRISMA score. The association between AMSTAR score and modified PRISMA score was also found to be highly statistically significant. Conclusion: Compliance of orthodontic SRs published in orthodontic journals with PRISMA guidelines was deficient in several areas. The quality of reporting assessed using PRISMA guidelines was significantly better in orthodontic SRs published in the Cochrane Database of Systematic Reviews.

Matched-pair study showed higher quality of placebo-controlled trials in Western phytotherapy than conventional medicine

OBJECTIVES: Herbal medicine (phytotherapy) is widely used, but the evidence for its effectiveness is a matter of ongoing debate. We compared the quality and results of trials of Western phytotherapy and conventional medicine. STUDY DESIGN AND SETTING: A random sample of placebo-controlled trials of Western phytotherapy was identified in a comprehensive literature search (19 electronic databases). Conventional medicine trials matched for condition and type of outcome were selected from the Cochrane Central Controlled Trials Register (issue 1, 2003). Data were extracted in duplicate. Trials described as double-blind, with adequate generation of allocation sequence and adequate concealment of allocation were assumed to be of higher methodological quality. RESULTS: Eighty-nine herbal medicine and 89 matched conventional medicine trials were analyzed. Studies of Western herbalism were smaller, less likely to be published in English, and less likely to be indexed in MEDLINE than their counterparts from conventional medicine. Nineteen (21%) herbal and four (5%) conventional medicine trials were of higher quality. In both groups, smaller trials showed more beneficial treatment effects than larger trials. CONCLUSIONS: Our findings challenge the widely held belief that the quality of the evidence on the effectiveness of herbal medicine is generally inferior to the evidence available for conventional medicine.

Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?

The aims of this study were to assess and compare the methodological quality of Cochrane and non-Cochrane systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) using AMSTAR and to compare the prevalence of meta-analysis in both review types. A literature search was undertaken to identify SRs that consisted of hand-searching five major orthodontic journals [American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics and Orthodontics and Craniofacial Research (February 2002 to July 2011)] and the Cochrane Database of Systematic Reviews from January 2000 to July 2011. Methodological quality of the included reviews was gauged using the AMSTAR tool involving 11 key methodological criteria with a score of 0 or 1 given for each criterion. A cumulative grade was given for the paper overall (0-11); an overall score of 4 or less represented poor methodological quality, 5-8 was considered fair and 9 or greater was deemed to be good. In total, 109 SRs were identified in the five major journals and on the CDSR. Of these, 26 (23.9%) were in the CDSR. The mean overall AMSTAR score was 6.2 with 21.1% of reviews satisfying 9 or more of the 11 criteria; a similar prevalence of poor reviews (22%) was also noted. Multiple linear regression indicated that reviews published in the CDSR (P < 0.01); and involving meta-analysis (β = 0.50, 95% confidence interval 0.72, 2.07, P < 0.001) showed greater concordance with AMSTAR.

Systematic reviews published in higher impact clinical journals were of higher quality.

OBJECTIVES To compare the methodological quality of systematic reviews (SRs) published in high- and low-impact factor (IF) Core Clinical Journals. In addition, we aimed to record the implementation of aspects of reporting, including Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram, reasons for study exclusion, and use of recommendations for interventions such as Grading of Recommendations Assessment, Development and Evaluation (GRADE). STUDY DESIGN AND SETTING We searched PubMed for systematic reviews published in Core Clinical Journals between July 1 and December 31, 2012. We evaluated the methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR) tool. RESULTS Over the 6-month period, 327 interventional systematic reviews were identified with a mean AMSTAR score of 63.3% (standard deviation, 17.1%), when converted to a percentage scale. We identified deficiencies in relation to a number of quality criteria including delineation of excluded studies and assessment of publication bias. We found that SRs published in higher impact journals were undertaken more rigorously with higher percentage AMSTAR scores (per IF unit: β = 0.68%; 95% confidence interval: 0.32, 1.04; P < 0.001), a discrepancy likely to be particularly relevant when differences in IF are large. CONCLUSION Methodological quality of SRs appears to be better in higher impact journals. The overall quality of SRs published in many Core Clinical Journals remains suboptimal.

PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study.

OBJECTIVE There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria. DESIGN Meta-epidemiological study within Cochrane Database of Systematic Reviews. METHODS Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis. RESULTS Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6. CONCLUSIONS The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

Do Psychosocial Interventions Improve Quality of Life and Wellbeing in Adults with Neuromuscular Disorders? A Systematic Review and Narrative Synthesis

Background: Quality of life and well-being are frequently restricted in adults with neuromuscular disorders. As such, identification of appropriate interventions is imperative. Objective: The objective of this paper was to systematically review and critically appraise quantitative studies (RCTs, controlled trials and cohort studies) of psychosocial interventions designed to improve quality of life and well-being in adults with neuromuscular disorders. Method: A systematic review of the published and unpublished literature was conducted. Studies meeting inclusion criteria were appraised using a validated quality assessment tool and results presented in a narrative synthesis. Results: Out of 3,136 studies identified, ten studies met criteria for inclusion within the review. Included studies comprised a range of interventions including: cognitive behavioural therapy, dignity therapy, hypnosis, expressive disclosure, gratitude lists, group psychoeducation and psychologically informed rehabilitation. Five of the interventions were for patients with Amyotrophic Lateral Sclerosis (ALS). The remainder were for patients with post-polio syndrome, muscular dystrophies and mixed disorders, such as Charcot-Marie-Tooth disease, myasthenia gravis and myotonic dystrophy. Across varied interventions and neuromuscular disorders, seven studies reported a short-term beneficial effect of intervention on quality of life and well-being. Whilst such findings are encouraging, widespread issues with the methodological quality of these studies significantly compromised the results. Conclusion: There is no strong evidence that psychosocial interventions improve quality of life and well-being in adults with neuromuscular disorders, due to a paucity of high quality research in this field. Multi-site, randomised controlled trials with active controls, standardised outcome measurement and longer term follow-ups are urgently required.

Atypical antipsychotics in the treatment of pathological aggression in children and adolescents: literature review and clinical recommendations

Objective: To review the literature about the use of atypical antipsychotics in the treatment of pathological aggression in children and adolescents. Method: The databases MEDLINE, SciELO, and LILACS were searched for publications in Portuguese or English from 1992 to August 2011 using the following keywords: mental disease, child, adolescent, treatment, atypical antipsychotic, aggressive behavior, aggression, and violent behavior. Results: Sixty-seven studies of good methodological quality and clinical interest and relevance were identified. Studies including children and adolescents were relatively limited, because few atypical antipsychotics have been approved by the Food and Drug Administration (FDA). All the medications included in this review (risperidone, olanzapine, quetiapine, ziprasidone, aripiprazole and clozapine) have some effectiveness in treating aggression in children and adolescents, and choices should be based on clinical indications and side effects. Conclusions: There are few studies about the effectiveness and safety of atypical antipsychotics for the pediatric population, and further randomized controlled studies with larger groups of patients and more diagnostic categories, such as severe conduct disorder and oppositional defiant disorder, should be conducted to confirm the results reported up to date and to evaluate the impact of long-term use.

A quantitative and qualitative evaluation of reports of clinical trials published in six Brazilian dental journals indexed in the Scientific Electronic Library Online (SciELO)

INTRODUCTION: Open access publishing is becoming increasingly popular within the biomedical sciences. SciELO, the Scientific Electronic Library Online, is a digital library covering a selected collection of Brazilian scientific journals many of which provide open access to full-text articles.This library includes a number of dental journals some of which may include reports of clinical trials in English, Portuguese and/or Spanish. Thus, SciELO could play an important role as a source of evidence for dental healthcare interventions especially if it yields a sizeable number of high quality reports. OBJECTIVE: The aim of this study was to identify reports of clinical trials by handsearching of dental journals that are accessible through SciELO, and to assess the overall quality of these reports. MATERIAL AND METHODS: Electronic versions of six Brazilian dental Journals indexed in SciELO were handsearched at www.scielo.br in September 2008. Reports of clinical trials were identified and classified as controlled clinical trials (CCTs - prospective, experimental studies comparing 2 or more healthcare interventions in human beings) or randomized controlled trials (RCTs - a random allocation method is clearly reported), according to Cochrane eligibility criteria. CRITERIA TO ASSESS METHODOLOGICAL QUALITY INCLUDED: method of randomization, concealment of treatment allocation, blinded outcome assessment, handling of withdrawals and losses and whether an intention-to-treat analysis had been carried out. RESULTS: The search retrieved 33 CCTs and 43 RCTs. A majority of the reports provided no description of either the method of randomization (75.3%) or concealment of the allocation sequence (84.2%). Participants and outcome assessors were reported as blinded in only 31.2% of the reports. Withdrawals and losses were only clearly described in 6.5% of the reports and none mentioned an intention-to-treat analysis or any similar procedure. CONCLUSIONS: The results of this study indicate that a substantial number of reports of trials and systematic reviews are available in the dental journals listed in SciELO, and that these could provide valuable evidence for clinical decision making. However, it is clear that the quality of a number of these reports is of some concern and that improvement in the conduct and reporting of these trials could be achieved if authors adhered to internationally accepted guidelines, e.g. the CONSORT statement.

QUADAS e STARD: avaliação da qualidade de estudos de acurácia de testes diagnósticos

OBJETIVO: Comparar duas abordagens baseadas em critérios do Quality Assessment of Diagnostic Accuracy Studies (QUADAS) e do Standards for Reporting Studies of Diagnostic Accuracy (STARD) na avaliação de qualidade de estudos de validação do teste rápido OptiMal®, para diagnóstico de malária. MÉTODOS: Foi realizada busca de artigos de validação do teste rápido na base bibliográfica Medline acessada pelo PubMed, no ano de 2007. Treze artigos foram recuperados na busca. Foram combinados 12 critérios do QUADAS e três do STARD para comparação com os critérios do QUADAS isoladamente. Foi considerado que artigos de regular a boa qualidade atenderiam pelo menos 50% dos critérios do QUADAS. RESULTADOS: Dos 13 artigos recuperados, 12 cumpriram pelo menos 50% dos critérios do QUADAS, e apenas dois atenderam à combinação dos critérios. Considerando-se a combinação dos dois critérios (> 6 QUADAS e > 3STARD), dois estudos (15,4%) apresentaram boa qualidade metodológica. A seleção de artigos usando a combinação proposta variou de dois a oito artigos, dependendo do número de itens considerados como ponto de corte. CONCLUSÕES: A combinação do QUADAS com o STARD tem o potencial de conferir maior rigor nas avaliações da qualidade de artigos publicados sobre validação de testes diagnósticos em malária, por incorporar a checagem de informações relevantes não alcançáveis pelo uso do QUADAS isoladamente.

A systematic review with procedural assessments and meta-analysis of Low Level Laser Therapy in lateral elbow tendinopathy (tennis elbow)

Background: Recent reviews have indicated that low level level laser therapy (LLLT) is ineffective in lateral elbow tendinopathy (LET) without assessing validity of treatment procedures and doses or the influence of prior steroid injections. Methods: Systematic review with meta-analysis, with primary outcome measures of pain relief and/or global improvement and subgroup analyses of methodological quality, wavelengths and treatment procedures. Results: 18 randomised placebo-controlled trials (RCTs) were identified with 13 RCTs (730 patients) meeting the criteria for meta-analysis. 12 RCTs satisfied half or more of the methodological criteria. Publication bias was detected by Egger's graphical test, which showed a negative direction of bias. Ten of the trials included patients with poor prognosis caused by failed steroid injections or other treatment failures, or long symptom duration or severe baseline pain. The weighted mean difference (WMD) for pain relief was 10.2 mm [95% CI: 3.0 to 17.5] and the RR for global improvement was 1.36 [1.16 to 1.60]. Trials which targeted acupuncture points reported negative results, as did trials with wavelengths 820, 830 and 1064 nm. In a subgroup of five trials with 904 nm lasers and one trial with 632 nm wavelength where the lateral elbow tendon insertions were directly irradiated, WMD for pain relief was 17.2 mm [95% CI: 8.5 to 25.9] and 14.0 mm [95% CI: 7.4 to 20.6] respectively, while RR for global pain improvement was only reported for 904 nm at 1.53 [95% CI: 1.28 to 1.83]. LLLT doses in this subgroup ranged between 0.5 and 7.2 Joules. Secondary outcome measures of painfree grip strength, pain pressure threshold, sick leave and follow-up data from 3 to 8 weeks after the end of treatment, showed consistently significant results in favour of the same LLLT subgroup (p < 0.02). No serious side-effects were reported. Conclusion: LLLT administered with optimal doses of 904 nm and possibly 632 nm wavelengths directly to the lateral elbow tendon insertions, seem to offer short-term pain relief and less disability in LET, both alone and in conjunction with an exercise regimen. This finding contradicts the conclusions of previous reviews which failed to assess treatment procedures, wavelengths and optimal doses.

Reaching those most in need: a review of diabetes self-management interventions in disadvantaged populations

There has been increased recognition of the importance of developing diabetes self-management education (DSME) interventions that are effective with under-served and minority populations. Despite several recent studies in this area, there is to our knowledge no systematic review or synthesis of what has been learned from this research. An electronic literature search identified five formative evaluations and ten controlled DSME intervention trials focused on under-served (low-income, minority or aged) populations. The RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) evaluation framework was used to evaluate the controlled studies on the dimensions of reach, efficacy, adoption, implementation, and maintenance. Fifty percent of the studies identified reported on the percentage of patients who participated, and the percentages were highly variable. The methodological quality of the articles was generally good and the short-term results were encouraging, especially on behavioral outcomes. Data on adoption (representativeness of settings and clinicians who participate) and implementation were almost never reported. Studies of modalities in addition to group meetings are needed to increase the reach of DSME with under-served populations. The promising formative evaluation work that has been conducted needs to be extended for more systematic study of the process of intervention implementation and adaptation with special populations. Studies that explicitly address the community context and that address multiple issues related to public health impact of DSME interventions are recommended to enhance long-term results. Copyright (C) 2002 John Wiley Sons, Ltd.

Qualidade de vida relacionada com o exercício em indivíduos com asma: revisão sistemática

Background: A asma condiciona o dia-a-dia do indivíduo asmático do ponto de vista clínico e emocional demonstrando-se muitas vezes como um subtractivo da qualidade de vida (QV). Alguns estudos, com particular incidência nos últimos dez anos, para além de demonstrarem os benefícios da actividade física na componente clínica da doença, têm analisado o seu efeito na QV dos asmáticos. Objectivo: Analisar os efeitos da actividade física na QV de indivíduos com asma tendo por base uma revisão da literatura actual. Métodos: Foi conduzida uma pesquisa dos randomized controlled trials (RCT) compreendidos entre Janeiro de 2000 e Agosto de 2010, bem como as citações e as referências bibliográficas de cada estudo nas principais bases de dados de ciências da saúde (Academic Search Complete, DOAJ, Elsevier – Science Direct, Highwire Press, PubMed, Scielo Global, Scirus, Scopus, SpringerLink, Taylor & Francis e Wiley Interscience) com as palavras-chave: asthma, quality of life, QoL, physical activity, exercise, breathing, training e programme em todas as combinações possíveis. Os estudos foram analisados independentemente por dois revisores quanto aos critérios de inclusão e qualidade dos estudos. Resultados: Dos 1075 estudos identificados apenas onze foram incluídos. Destes, seis apresentaram um score 5/10, três 6/10 e dois 7/10 segundo a escala PEDro. Cinco destes estudos foram realizados em crianças entre os 7 e os 15 anos e os restantes em adultos. Os programas de intervenção dividiram-se em programas de treino aeróbio e programas de exercícios respiratórios. Todos programas de treino aeróbio apresentaram melhorias na QV demonstrando uma influência positiva do treino aeróbio na asma. Principais conclusões: Há uma tendência notória do benefício dos programas de treino aeróbio na QV dos indivíduos asmáticos. Os programas de exercícios respiratórios foram poucos e heterogéneos impossibilitando uma conclusão positiva quanto à sua recomendação para a melhoria da QV nesta patologia. Há uma grande necessidade de mais RCT com rigor metodológico.