The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study

OBJECTIVE: To evaluate the association of adequate allocation concealment and patient blinding with estimates of treatment benefits in osteoarthritis trials. METHODS: We performed a meta-epidemiologic study of 16 meta-analyses with 175 trials that compared therapeutic interventions with placebo or nonintervention control in patients with hip or knee osteoarthritis. We calculated effect sizes from the differences in means of pain intensity between groups at the end of followup divided by the pooled SD and compared effect sizes between trials with and trials without adequate methodology. RESULTS: Effect sizes tended to be less beneficial in 46 trials with adequate allocation concealment compared with 112 trials with inadequate or unclear concealment of allocation (difference -0.15; 95% confidence interval [95% CI] -0.31, 0.02). Selection bias associated with inadequate or unclear concealment of allocation was most pronounced in meta-analyses with large estimated treatment benefits (P for interaction < 0.001), meta-analyses with high between-trial heterogeneity (P = 0.009), and meta-analyses of complementary medicine (P = 0.019). Effect sizes tended to be less beneficial in 64 trials with adequate blinding of patients compared with 58 trials without (difference -0.15; 95% CI -0.39, 0.09), but differences were less consistent and disappeared after accounting for allocation concealment. Detection bias associated with a lack of adequate patient blinding was most pronounced for nonpharmacologic interventions (P for interaction < 0.001). CONCLUSION: Results of osteoarthritis trials may be affected by selection and detection bias. Adequate concealment of allocation and attempts to blind patients will minimize these biases.

What is the influence of randomisation sequence generation and allocation concealment on treatment effects of physical therapy trials? A meta-epidemiological study.

OBJECTIVE To determine if adequacy of randomisation and allocation concealment is associated with changes in effect sizes (ES) when comparing physical therapy (PT) trials with and without these methodological characteristics. DESIGN Meta-epidemiological study. PARTICIPANTS A random sample of randomised controlled trials (RCTs) included in meta-analyses in the PT discipline were identified. INTERVENTION Data extraction including assessments of random sequence generation and allocation concealment was conducted independently by two reviewers. To determine the association between sequence generation, and allocation concealment and ES, a two-level analysis was conducted using a meta-meta-analytic approach. PRIMARY AND SECONDARY OUTCOME MEASURES association between random sequence generation and allocation concealment and ES in PT trials. RESULTS 393 trials included in 43 meta-analyses, analysing 44 622 patients contributed to this study. Adequate random sequence generation and appropriate allocation concealment were accomplished in only 39.7% and 11.5% of PT trials, respectively. Although trials with inappropriate allocation concealment tended to have an overestimate treatment effect when compared with trials with adequate concealment of allocation, the difference was non-statistically significant (ES=0.12; 95% CI -0.06 to 0.30). When pooling our results with those of Nuesch et al, we obtained a pooled statistically significant value (ES=0.14; 95% CI 0.02 to 0.26). There was no difference in ES in trials with appropriate or inappropriate random sequence generation (ES=0.02; 95% CI -0.12 to 0.15). CONCLUSIONS Our results suggest that when evaluating risk of bias of primary RCTs in PT area, systematic reviewers and clinicians implementing research into practice should pay attention to these biases since they could exaggerate treatment effects. Systematic reviewers should perform sensitivity analysis including trials with low risk of bias in these domains as primary analysis and/or in combination with less restrictive analyses. Authors and editors should make sure that allocation concealment and random sequence generation are properly reported in trial reports.

Randomization in clinical trials in orthodontics: its significance in research design and methods to achieve it

Randomization is a key step in reducing selection bias during the treatment allocation phase in randomized clinical trials. The process of randomization follows specific steps, which include generation of the randomization list, allocation concealment, and implementation of randomization. The phenomenon in the dental and orthodontic literature of characterizing treatment allocation as random is frequent; however, often the randomization procedures followed are not appropriate. Randomization methods assign, at random, treatment to the trial arms without foreknowledge of allocation by either the participants or the investigators thus reducing selection bias. Randomization entails generation of random allocation, allocation concealment, and the actual methodology of implementing treatment allocation randomly and unpredictably. Most popular randomization methods include some form of restricted and/or stratified randomization. This article introduces the reasons, which make randomization an integral part of solid clinical trial methodology, and presents the main randomization schemes applicable to clinical trials in orthodontics.

Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials

Published evidence suggests that aspects of trial design lead to biased intervention effect estimates, but findings from different studies are inconsistent. This study combined data from 7 meta-epidemiologic studies and removed overlaps to derive a final data set of 234 unique meta-analyses containing 1973 trials. Outcome measures were classified as "mortality," "other objective," "or subjective," and Bayesian hierarchical models were used to estimate associations of trial characteristics with average bias and between-trial heterogeneity. Intervention effect estimates seemed to be exaggerated in trials with inadequate or unclear (vs. adequate) random-sequence generation (ratio of odds ratios, 0.89 [95% credible interval {CrI}, 0.82 to 0.96]) and with inadequate or unclear (vs. adequate) allocation concealment (ratio of odds ratios, 0.93 [CrI, 0.87 to 0.99]). Lack of or unclear double-blinding (vs. double-blinding) was associated with an average of 13% exaggeration of intervention effects (ratio of odds ratios, 0.87 [CrI, 0.79 to 0.96]), and between-trial heterogeneity was increased for such studies (SD increase in heterogeneity, 0.14 [CrI, 0.02 to 0.30]). For each characteristic, average bias and increases in between-trial heterogeneity were driven primarily by trials with subjective outcomes, with little evidence of bias in trials with objective and mortality outcomes. This study is limited by incomplete trial reporting, and findings may be confounded by other study design characteristics. Bias associated with study design characteristics may lead to exaggeration of intervention effect estimates and increases in between-trial heterogeneity in trials reporting subjectively assessed outcomes.

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

OBJECTIVE: To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. DESIGN: Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses. DATA SOURCES: 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. MAIN OUTCOME MEASURES: Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates. RESULTS: In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. CONCLUSIONS: The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

Systemic prokinetic pharmacologic treatment for postoperative adynamic ileus following abdominal surgery in adults

BACKGROUND: Postoperative adynamic bowel atony interferes with recovery following abdominal surgery. Prokinetic pharmacologic drugs are widely used to accelerate postoperative recovery. OBJECTIVES: To evaluate the benefits and harms of systemic acting prokinetic drugs to treat postoperative adynamic ileus in patients undergoing abdominal surgery. SEARCH STRATEGY: Trials were identified by computerised searches of the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, and the Cochrane Colorectal Cancer Group specialised register. The reference lists of included trials and review articles were tracked and authors contacted. SELECTION CRITERIA: Randomised controlled parallel-group trials (RCT) comparing the effect of systemically acting prokinetic drugs against placebo or no intervention. DATA COLLECTION AND ANALYSIS: Four reviewers independently extracted the data and assessed trial quality. Trial authors were contacted for additional information if needed. MAIN RESULTS: Thirty-nine RCTs met the inclusion criteria contributing a total of 4615 participants. Most trials enrolled a small number of patients and showed moderate to poor (reporting of) methodological quality, in particular regarding allocation concealment and intention-to-treat analysis. Fifteen systemic acting prokinetic drugs were investigated and ten comparisons could be summarized. Six RCTs support the effect of Alvimopan, a novel peripheral mu receptor antagonist. However, the trials do not meet reporting guidelines and the drug is still in an investigational stage. Erythromycin showed homogenous and consistent absence of effect across all included trials and outcomes. The evidence is insufficient to recommend the use of cholecystokinin-like drugs, cisapride, dopamine-antagonists, propranolol or vasopressin. Effects are either inconsistent across outcomes, or trials are too small and often of poor methodological quality. Cisapride has been withdrawn from the market due to adverse cardiac events in many countries. Intravenous lidocaine and neostigmine might show a potential effect, but more evidence on clinically relevant outcomes is needed. Heterogeneity among included trials was seen in 10 comparisons. No major adverse drug effects were evident. AUTHORS' CONCLUSIONS: Alvimopan may prove to be beneficial but proper judgement needs adherence to reporting standards. Further trials are needed on intravenous lidocaine and neostigmine. The remaining drugs can not be recommended due to lack of evidence or absence of effect.

Reporting quality of abstracts of randomized controlled trials published in dental specialty journals

OBJECTIVES A widespread assessment of the reporting of RCT abstracts published in dental journals is lacking. Our aim was to investigate the quality of reporting of abstracts published in leading dental specialty journals using, as a guide, the CONSORT for abstracts checklist. METHODS Electronic and supplementary hand searching were undertaken to identify RCTs published in seven dental specialty journals. The quality of abstract reporting was evaluated using a modified checklist based on the CONSORT for abstracts checklist. Descriptive statistics followed by univariate and multivariate analyses were conducted. RESULTS 228 RCT abstracts were identified. Reporting of interventions, objectives and conclusions within abstracts were adequate. Inadequately reported items included: title, participants, outcomes, random number generation, numbers randomized and effect size estimate. Randomization restrictions, allocation concealment, blinding, numbers analyzed, confidence intervals, intention-to-treat analysis, harms, registration and funding were rarely described. CONCLUSIONS The mean overall reporting quality score was suboptimal at 62.5% (95% CI: 61.9, 63.0). Significantly better abstract reporting was noted in certain specialty journals and in multicenter trials.

Adjudication-related processes are underreported and lack standardization in clinical trials of venous thromboembolism: a systematic review

OBJECTIVES Although the use of an adjudication committee (AC) for outcomes is recommended in randomized controlled trials, there are limited data on the process of adjudication. We therefore aimed to assess whether the reporting of the adjudication process in venous thromboembolism (VTE) trials meets existing quality standards and which characteristics of trials influence the use of an AC. STUDY DESIGN AND SETTING We systematically searched MEDLINE and the Cochrane Library from January 1, 2003, to June 1, 2012, for randomized controlled trials on VTE. We abstracted information about characteristics and quality of trials and reporting of adjudication processes. We used stepwise backward logistic regression model to identify trial characteristics independently associated with the use of an AC. RESULTS We included 161 trials. Of these, 68.9% (111 of 161) reported the use of an AC. Overall, 99.1% (110 of 111) of trials with an AC used independent or blinded ACs, 14.4% (16 of 111) reported how the adjudication decision was reached within the AC, and 4.5% (5 of 111) reported on whether the reliability of adjudication was assessed. In multivariate analyses, multicenter trials [odds ratio (OR), 8.6; 95% confidence interval (CI): 2.7, 27.8], use of a data safety-monitoring board (OR, 3.7; 95% CI: 1.2, 11.6), and VTE as the primary outcome (OR, 5.7; 95% CI: 1.7, 19.4) were associated with the use of an AC. Trials without random allocation concealment (OR, 0.3; 95% CI: 0.1, 0.8) and open-label trials (OR, 0.3; 95% CI: 0.1, 1.0) were less likely to report an AC. CONCLUSION Recommended processes of adjudication are underreported and lack standardization in VTE-related clinical trials. The use of an AC varies substantially by trial characteristics.

Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study

BACKGROUND Empirical research has illustrated an association between study size and relative treatment effects, but conclusions have been inconsistent about the association of study size with the risk of bias items. Small studies give generally imprecisely estimated treatment effects, and study variance can serve as a surrogate for study size. METHODS We conducted a network meta-epidemiological study analyzing 32 networks including 613 randomized controlled trials, and used Bayesian network meta-analysis and meta-regression models to evaluate the impact of trial characteristics and study variance on the results of network meta-analysis. We examined changes in relative effects and between-studies variation in network meta-regression models as a function of the variance of the observed effect size and indicators for the adequacy of each risk of bias item. Adjustment was performed both within and across networks, allowing for between-networks variability. RESULTS Imprecise studies with large variances tended to exaggerate the effects of the active or new intervention in the majority of networks, with a ratio of odds ratios of 1.83 (95% CI: 1.09,3.32). Inappropriate or unclear conduct of random sequence generation and allocation concealment, as well as lack of blinding of patients and outcome assessors, did not materially impact on the summary results. Imprecise studies also appeared to be more prone to inadequate conduct. CONCLUSIONS Compared to more precise studies, studies with large variance may give substantially different answers that alter the results of network meta-analyses for dichotomous outcomes.

Assessing the reporting quality in abstracts of randomized controlled trials in leading journals of oral implantology.

AIM Abstracts of randomized clinical trials are extremely important as trial appraisal is often based on the information included here. The objective of this study was to assess the quality of the reporting of RCT abstracts in journals of Oral Implantology. MATERIAL AND METHODS Six leading Implantology journals were screened for RCTs between years 2008 and 2012. A 21-item modified CONSORT for abstracts checklist was used to examine the completeness of abstract reporting. Descriptive statistics and linear regression modeling were employed for data analysis. RESULTS One hundred and sixty three RCT abstracts were included in this study. The majority of the RCTs were published in the Clinical Oral Implants Research (42.9%). The mean overall reporting quality score was 58.6% (95% CI: 57.6-59.7). The highest score was noted in the European Journal of Oral Implantology (63.8%; 95% CI: 61.8-65.8). Multivariate analysis demonstrated that abstract quality score was related to publication journal and number of research centers involved. Most abstracts adequately reported interventions (89.0%), objectives (77.9%) and conclusions (74.8%) while failed to report randomization procedures, allocation concealment, effect estimate, confidence intervals, and funding. Registration of RCTs was not reported in any of the abstracts. CONCLUSIONS The reporting quality in abstracts of RCTs published in Oral Implantology journals needs to be improved. Editors and authors should be encouraged to endorse the CONSORT for abstracts guidelines in order to achieve optimal quality in abstract reporting.

An Assessment of the Risk of Bias in Randomized Controlled Trial Reports Published in Prosthodontic and Implant Dentistry Journals

PURPOSE The objective of this study was to assess the risk of bias of randomized controlled trials (RCTs) published in prosthodontic and implant dentistry journals. MATERIALS AND METHODS The last 30 issues of 9 journals in the field of prosthodontic and implant dentistry (Clinical Implant Dentistry and Related Research, Clinical Oral Implants Research, Implant Dentistry, International Journal of Oral & Maxillofacial Implants, International Journal of Periodontics and Restorative Dentistry, International Journal of Prosthodontics, Journal of Dentistry, Journal of Oral Rehabilitation, and Journal of Prosthetic Dentistry) were hand-searched for RCTs. Risk of bias was assessed using the Cochrane Collaboration's risk of bias tool and analyzed descriptively. RESULTS From the 3,667 articles screened, a total of 147 RCTs were identified and included. The number of published RCTs increased with time. The overall distribution of a high risk of bias assessment varied across the domains of the Cochrane risk of bias tool: 8% for random sequence generation, 18% for allocation concealment, 41% for masking, 47% for blinding of outcome assessment, 7% for incomplete outcome data, 12% for selective reporting, and 41% for other biases. CONCLUSION The distribution of high risk of bias for RCTs published in the selected prosthodontic and implant dentistry journals varied among journals and ranged from 8% to 47%, which can be considered as substantial.