PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study.

OBJECTIVE There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria. DESIGN Meta-epidemiological study within Cochrane Database of Systematic Reviews. METHODS Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis. RESULTS Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6. CONCLUSIONS The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

An analysis of protocols and publications suggested that most discontinuations of clinical trials were not based on preplanned interim analyses or stopping rules.

OBJECTIVES To investigate the frequency of interim analyses, stopping rules, and data safety and monitoring boards (DSMBs) in protocols of randomized controlled trials (RCTs); to examine these features across different reasons for trial discontinuation; and to identify discrepancies in reporting between protocols and publications. STUDY DESIGN AND SETTING We used data from a cohort of RCT protocols approved between 2000 and 2003 by six research ethics committees in Switzerland, Germany, and Canada. RESULTS Of 894 RCT protocols, 289 prespecified interim analyses (32.3%), 153 stopping rules (17.1%), and 257 DSMBs (28.7%). Overall, 249 of 894 RCTs (27.9%) were prematurely discontinued; mostly due to reasons such as poor recruitment, administrative reasons, or unexpected harm. Forty-six of 249 RCTs (18.4%) were discontinued due to early benefit or futility; of those, 37 (80.4%) were stopped outside a formal interim analysis or stopping rule. Of 515 published RCTs, there were discrepancies between protocols and publications for interim analyses (21.1%), stopping rules (14.4%), and DSMBs (19.6%). CONCLUSION Two-thirds of RCT protocols did not consider interim analyses, stopping rules, or DSMBs. Most RCTs discontinued for early benefit or futility were stopped without a prespecified mechanism. When assessing trial manuscripts, journals should require access to the protocol.

Addressing missing participant outcome data in dental clinical trials

Missing outcome data are common in clinical trials and despite a well-designed study protocol, some of the randomized participants may leave the trial early without providing any or all of the data, or may be excluded after randomization. Premature discontinuation causes loss of information, potentially resulting in attrition bias leading to problems during interpretation of trial findings. The causes of information loss in a trial, known as mechanisms of missingness, may influence the credibility of the trial results. Analysis of trials with missing outcome data should ideally be handled with intention to treat (ITT) rather than per protocol (PP) analysis. However, true ITT analysis requires appropriate assumptions and imputation of missing data. Using a worked example from a published dental study, we highlight the key issues associated with missing outcome data in clinical trials, describe the most recognized approaches to handling missing outcome data, and explain the principles of ITT and PP analysis.

Clinical end points in coronary stent trials: a case for standardized definitions

BACKGROUND: Although most clinical trials of coronary stents have measured nominally identical safety and effectiveness end points, differences in definitions and timing of assessment have created confusion in interpretation. METHODS AND RESULTS: The Academic Research Consortium is an informal collaboration between academic research organizations in the United States and Europe. Two meetings, in Washington, DC, in January 2006 and in Dublin, Ireland, in June 2006, sponsored by the Academic Research Consortium and including representatives of the US Food and Drug Administration and all device manufacturers who were working with the Food and Drug Administration on drug-eluting stent clinical trial programs, were focused on consensus end point definitions for drug-eluting stent evaluations. The effort was pursued with the objective to establish consistency among end point definitions and provide consensus recommendations. On the basis of considerations from historical legacy to key pathophysiological mechanisms and relevance to clinical interpretability, criteria for assessment of death, myocardial infarction, repeat revascularization, and stent thrombosis were developed. The broadly based consensus end point definitions in this document may be usefully applied or recognized for regulatory and clinical trial purposes. CONCLUSION: Although consensus criteria will inevitably include certain arbitrary features, consensus criteria for clinical end points provide consistency across studies that can facilitate the evaluation of safety and effectiveness of these devices.

Clinical endpoints in peripheral endovascular revascularization trials: a case for standardized definitions

BACKGROUND: Endovascular therapy is a rapidly expanding option for the treatment of patients with peripheral arterial disease (PAD), leading to a myriad of published studies reporting on various revascularization strategies. However, these reports are often difficult to interpret and compare because they do not utilize uniform clinical endpoint definitions. Moreover, few of these studies describe clinical outcomes from a patients' perspective. METHODS AND RESULTS: The DEFINE Group is a collaborative effort of an ad-hoc multidisciplinary team from various specialties involved in peripheral arterial disease therapy in Europe and the United States. DEFINE's goal was to arrive at a broad based consensus for baseline and endpoint definitions in peripheral endovascular revascularization trials for chronic lower limb ischemia. In this project, which started in 2006, the individual team members reviewed the existing pertinent literature. Following this, a series of telephone conferences and face-to-face meetings were held to agree upon definitions. Input was also obtained from regulatory (United States Food and Drug Administration) and industry (device manufacturers with an interest in peripheral endovascular revascularization) stakeholders, respectively. The efforts resulted in the current document containing proposed baseline and endpoint definitions in chronic lower limb PAD. Although the consensus has inevitably included certain arbitrary choices and compromises, adherence to these proposed standard definitions would provide consistency across future trials, thereby facilitating evaluation of clinical effectiveness and safety of various endovascular revascularization techniques. CONCLUSION: This current document is based on a broad based consensus involving relevant stakeholders from the medical community, industry and regulatory bodies. It is proposed that the consensus document may have value for study design of future clinical trials in chronic lower limb ischemia as well as for regulatory purposes.

Standardized definitions and clinical endpoints in trials investigating endovascular repair of aortic dissections.

OBJECTIVES Endovascular therapy is a rapidly expanding option for the treatment of patients with aortic dissection (AD) and various studies have been published. These trials, however, are often difficult to interpret and compare because they do not utilize uniform clinical endpoint definitions. METHODS The DEFINE Group is a collaborative effort of an ad hoc multidisciplinary team from various specialties involved in AD therapy in Europe and the United States. DEFINE's goal was to arrive at a broad based consensus for baseline and endpoint definitions in trials for endovascular therapy of various vascular pathologies. In this project, which started in December 2006, the individual team members reviewed the existing pertinent literature. Following this, a series of telephone conferences and face-to-face meetings were held to agree upon definitions. Input was also obtained from regulatory (United States Food and Drug Administration) and industry (device manufacturers with an interest in peripheral endovascular revascularization) stakeholders, respectively. RESULTS These efforts resulted in the present document containing proposed baseline and endpoint definitions for clinical and morphological outcomes. Although the consensus has inevitably included certain arbitrary consensus choices and compromises, adherence to these proposed standard definitions would provide consistency across future trials, thereby facilitating evaluation of clinical effectiveness and safety of various endovascular revascularization techniques. CONCLUSIONS This current document is based on a broad based consensus involving relevant stakeholders from the medical community, industry and regulatory bodies. It is proposed that the consensus document may have value for study design of future clinical trials in endovascular AD therapy as well as for regulatory purposes.

Clinical Perspectives from Randomized Phase 3 Trials on Prostate Cancer: An Analysis of the ClinicalTrials.gov Database

AbstractBackground It is not easy to overview pending phase 3 trials on prostate cancer (PCa), and awareness of these trials would benefit clinicians. Objective To identify all phase 3 trials on {PCa} registered in the ClinicalTrials.gov database with pending results. Design and setting On September 29, 2014, a database was established from the records for 175 538 clinical trials registered on ClinicalTrials.gov. A search of this database for the substring “prostat” identified 2951 prostate trials. Phase 3 trials accounted for 441 studies, of which 333 concerned only PCa. We selected only ongoing or completed trials with pending results, that is, for which the primary endpoint had not been published in a peer-reviewed medical journal. Results and limitations We identified 123 phase 3 trials with pending results. Trials were conducted predominantly in North America (n = 63; 51) and Europe (n = 47; 38). The majority were on nonmetastatic disease (n = 82; 67), with 37 (30) on metastatic disease and four trials (3) including both. In terms of intervention, systemic treatment was most commonly tested (n = 71; 58), followed by local treatment 34 (28), and both systemic and local treatment (n = 11; 9), with seven (6) trials not classifiable. The 71 trials on systemic treatment included androgen deprivation therapy (n = 34; 48), chemotherapy (n = 15; 21), immunotherapy (n = 9; 13), other systemic drugs (n = 9; 13), radiopharmaceuticals (n = 2; 3), and combinations (n = 2; 3). Local treatments tested included radiation therapy (n = 27; 79), surgery (n = 5; 15), and both (n = 2; 2). A limitation is that not every clinical trial is registered on ClinicalTrials.gov. Conclusion There are many {PCa} phase 3 trials with pending results, most of which address questions regarding systemic treatments for both nonmetastatic and metastatic disease. Radiation therapy and androgen deprivation therapy are the interventions most commonly tested for local and systemic treatment, respectively. Patient summary This report describes all phase 3 trials on prostate cancer registered in the ClinicalTrials.gov database with pending results. Most of these trials address questions regarding systemic treatments for both nonmetastatic and metastatic disease. Radiation therapy and androgen deprivation therapy are the interventions most commonly tested for local and systemic treatment, respectively.

Measurement issues in trials of pediatric acute diarrheal diseases: a systematic review

BACKGROUND: Worldwide, diarrheal diseases rank second among conditions that afflict children. Despite the disease burden, there is limited consensus on how to define and measure pediatric acute diarrhea in trials. OBJECTIVES: In RCTs of children involving acute diarrhea as the primary outcome, we documented (1) how acute diarrhea and its resolution were defined, (2) all primary outcomes, (3) the psychometric properties of instruments used to measure acute diarrhea and (4) the methodologic quality of included trials, as reported. METHODS: We searched CENTRAL, Embase, Global Health, and Medline from inception to February 2009. English-language RCTs of children younger than 19 years that measured acute diarrhea as a primary outcome were chosen. RESULTS: We identified 138 RCTs reporting on 1 or more primary outcomes related to pediatric acute diarrhea/diseases. Included trials used 64 unique definitions of diarrhea, 69 unique definitions of diarrhea resolution, and 46 unique primary outcomes. The majority of included trials evaluated short-term clinical disease activity (incidence and duration of diarrhea), laboratory outcomes, or a composite of these end points. Thirty-two trials used instruments (eg, single and multidomain scoring systems) to support assessment of disease activity. Of these, 3 trials stated that their instrument was valid; however, none of the trials (or their citations) reported evidence of this validity. The overall methodologic quality of included trials was good. CONCLUSIONS: Even in what would be considered methodologically sound clinical trials, definitions of diarrhea, primary outcomes, and instruments employed in RCTs of pediatric acute diarrhea are heterogeneous, lack evidence of validity, and focus on indices that may not be important to participants.

Implementation of raltegravir in routine clinical practice: selection criteria for choosing this drug, virologic response rates, and characteristics of failures

Raltegravir (RAL) achieved remarkable virologic suppression rates in randomized-clinical trials, but today efficacy data and factors for treatment failures in a routine clinical care setting are limited.

Sample size estimation: an overview with applications to orthodontic clinical trial designs

Proper sample size estimation is an important part of clinical trial methodology and closely related to the precision and power of the trial's results. Trials with sufficient sample sizes are scientifically and ethically justified and more credible compared with trials with insufficient sizes. Planning clinical trials with inadequate sample sizes might be considered as a waste of time and resources, as well as unethical, since patients might be enrolled in a study in which the expected results will not be trusted and are unlikely to have an impact on clinical practice. Because of the low emphasis of sample size calculation in clinical trials in orthodontics, it is the objective of this article to introduce the orthodontic clinician to the importance and the general principles of sample size calculations for randomized controlled trials to serve as guidance for study designs and as a tool for quality assessment when reviewing published clinical trials in our specialty. Examples of calculations are shown for 2-arm parallel trials applicable to orthodontics. The working examples are analyzed, and the implications of design or inherent complexities in each category are discussed.

Reporting of research quality characteristics of studies published in 6 major clinical dental specialty journals

The objective of this article was to record reporting characteristics related to study quality of research published in major specialty dental journals with the highest impact factor (Journal of Endodontics, Journal of Oral and Maxillofacial Surgery, American Journal of Orthodontics and Dentofacial Orthopedics; Pediatric Dentistry, Journal of Clinical Periodontology, and International Journal of Prosthetic Dentistry). The included articles were classified into the following 3 broad subject categories: (1) cross-sectional (snap-shot), (2) observational, and (3) interventional. Multinomial logistic regression was conducted for effect estimation using the journal as the response and randomization, sample calculation, confounding discussed, multivariate analysis, effect measurement, and confidence intervals as the explanatory variables. The results showed that cross-sectional studies were the dominant design (55%), whereas observational investigations accounted for 13%, and interventions/clinical trials for 32%. Reporting on quality characteristics was low for all variables: random allocation (15%), sample size calculation (7%), confounding issues/possible confounders (38%), effect measurements (16%), and multivariate analysis (21%). Eighty-four percent of the published articles reported a statistically significant main finding and only 13% presented confidence intervals. The Journal of Clinical Periodontology showed the highest probability of including quality characteristics in reporting results among all dental journals.

Effect of platform switching on peri-implant bone levels: a randomized clinical trial

The concept of platform switching has been introduced to implant dentistry based on observations of reduced peri-implant bone loss. However, randomized clinical trials are still lacking. This study aimed to test the hypothesis that platform switching has a positive impact on crestal bone-level changes.