Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials

Published evidence suggests that aspects of trial design lead to biased intervention effect estimates, but findings from different studies are inconsistent. This study combined data from 7 meta-epidemiologic studies and removed overlaps to derive a final data set of 234 unique meta-analyses containing 1973 trials. Outcome measures were classified as "mortality," "other objective," "or subjective," and Bayesian hierarchical models were used to estimate associations of trial characteristics with average bias and between-trial heterogeneity. Intervention effect estimates seemed to be exaggerated in trials with inadequate or unclear (vs. adequate) random-sequence generation (ratio of odds ratios, 0.89 [95% credible interval {CrI}, 0.82 to 0.96]) and with inadequate or unclear (vs. adequate) allocation concealment (ratio of odds ratios, 0.93 [CrI, 0.87 to 0.99]). Lack of or unclear double-blinding (vs. double-blinding) was associated with an average of 13% exaggeration of intervention effects (ratio of odds ratios, 0.87 [CrI, 0.79 to 0.96]), and between-trial heterogeneity was increased for such studies (SD increase in heterogeneity, 0.14 [CrI, 0.02 to 0.30]). For each characteristic, average bias and increases in between-trial heterogeneity were driven primarily by trials with subjective outcomes, with little evidence of bias in trials with objective and mortality outcomes. This study is limited by incomplete trial reporting, and findings may be confounded by other study design characteristics. Bias associated with study design characteristics may lead to exaggeration of intervention effect estimates and increases in between-trial heterogeneity in trials reporting subjectively assessed outcomes.

Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies

The design of randomised controlled trials (RCTs) should incorporate characteristics (such as concealment of randomised allocation and blinding of participants and personnel) that avoid biases resulting from lack of comparability of the intervention and control groups. Empirical evidence suggests that the absence of such characteristics leads to biased intervention effect estimates, but the findings of different studies are not consistent.

Integrated care in patients with schizophrenia: results of trials published between 2011 and 2013 focusing on effectiveness and efficiency

Purpose of review: Overview on integrated care trials focusing on effectiveness and efficiency published from 2011 to 2013. Recent findings: Eight randomized controlled trials (RCTs) and 21 non-RCT studies were published from 2011 to 2013. Studies differed in several methodological aspects such as study population, psychotherapeutic approaches used, outcome parameters, follow-up times, fidelities, and implementation of the integrated care model and the nation-specific healthcare context with different control conditions. This makes it difficult to draw firm conclusions. Most studies demonstrated relevant improvements regarding symptoms (P = 0.001) and functioning (P = 0.01), quality of life (P = 0.01), adherence (P <0.05) and patient's satisfaction (P = 0.01), and reduction of caregiver's stress (P < 0.05). Mean total costs were favoring or at least equalizing costs but with positive effects found on subjective health favoring integrated care models. Summary: There is an increasing interest in the effectiveness and efficiency of integrated care models in patients with mental disorders, specifically in those with severe and persistent mental illness. To increase generalizability, future trials should exactly describe rationales and content of integrated care model and control conditions.

Gonadal function and fertility in survivors after Hodgkin lymphoma treatment within the German Hodgkin Study Group HD13 to HD15 trials

To optimize fertility advice in patients with Hodgkin lymphoma (HL) before therapy and during survivorship, information on the impact of chemotherapy is needed. Therefore, we analyzed gonadal functions in survivors of HL.

[Modalities of acupuncture treatments in assisted reproductive technology--a comparison of treatment practice in Swiss, German, and Austrian fertility centers with findings from randomized controlled trials]

One in 5 couples is affected by infertility. To increase the effectiveness of assisted reproductive technology (ART) adjuvant acupuncture treatments are frequently administered. However, little is known about acupuncture treatment modalities employed in fertility centers. The aim of our study was to assess modalities of acupuncture treatments in fertility centers and compare them with investigated acupuncture treatments in randomized controlled trials (RCTs) related to ART.

Treating intermittent claudication with Tibetan medicine Padma 28: does it work?

Herbal drugs are being increasingly used in medical practice, often without appropriate scrutiny of their safety and efficacy. The medicinal product Padma 28 is a fixed combination with Tibetan origin, used in Europe since the 1960s for the symptomatic treatment of circulatory disorders, including those of peripheral arterial occlusive disease (PAOD). We have conducted an analysis of all available data on this herbal drug from published literature as well as from original data we obtained from contacting the authors of published papers, reports and the manufacturer. A total of 19 trials have reported on 2084 patients to date, 444 of whom were in six controlled clinical studies on PAOD. A meta-analysis of five trials showed Padma 28 to increase walking distance by >100m in 18.2% of the patients with verum, versus 2.1% with placebo (P<0.001; odds ratio: 10 [95% CI 3.03, 33.33]; RR: 0.12; number needed to treat=6.2). The safety profile appears to be favourable. Available evidence shows that Padma 28 provides significant relief from PAOD-related symptoms (i.e. walking distance), probably of the same order of magnitude as other employed medications. However, larger confirmatory RCTs are desirable.

A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints

Publication bias and related bias in meta-analysis is often examined by visually checking for asymmetry in funnel plots of treatment effect against its standard error. Formal statistical tests of funnel plot asymmetry have been proposed, but when applied to binary outcome data these can give false-positive rates that are higher than the nominal level in some situations (large treatment effects, or few events per trial, or all trials of similar sizes). We develop a modified linear regression test for funnel plot asymmetry based on the efficient score and its variance, Fisher's information. The performance of this test is compared to the other proposed tests in simulation analyses based on the characteristics of published controlled trials. When there is little or no between-trial heterogeneity, this modified test has a false-positive rate close to the nominal level while maintaining similar power to the original linear regression test ('Egger' test). When the degree of between-trial heterogeneity is large, none of the tests that have been proposed has uniformly good properties.

Herbal medicine in the treatment of liver diseases

Herbal drugs have become increasingly popular and their use is widespread. Licensing regulations and pharmacovigilance regarding herbal products are still incomplete and clearcut proof of their efficacy in liver diseases is sparse. Nevertheless, a number of herbals show promising activity including silymarin for antifibrotic treatment, phyllantus amarus in chronic hepatitis B, glycyrrhizin to treat chronic viral hepatitis, and a number of herbal combinations from China and Japan that deserve testing in appropriate studies. Apart from therapeutic properties, reports are accumulating about liver injury after the intake of herbals, including those advertised for liver diseases. Acute and/or chronic liver damage occurred after ingestion of some Chinese herbs, herbals that contain pyrrolizidine alkaloids, germander, greater celandine, kava, atractylis gummifera, callilepsis laureola, senna alkaloids, chaparral and many others. Since the evidence supporting the use of botanicals to treat chronic liver diseases is insufficient and only few of them are well standardised and free of potential serious side effects, most of these medications are not recommended outside clinical trials. Particularly with regard to the latter, adequately powered randomised-controlled clinical trials with well-selected end points are needed to assess the role of herbal therapy for liver diseases.

Peripheral endovascular revascularization trials: high noon for uniform reporting standards

In the era of evidence based medicine, proof of clinical benefits and cost-effectiveness need to be firmly based on transparent comparisons between various therapeutic alternatives. Standards for reports are a prerequisite for comparisons across reports and should reflect the patient's perspective. According standards have been successfully introduced in many fields of modern medicine, but not yet for peripheral endovascular interventions. Given the overwhelmingly increasing importance of endovascular revascularization in patients with chronic lower limb ischemia, this report provides an updated outline of the heterogeneity of current reporting practice and highlights the need for uniform reporting standards.

The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study

OBJECTIVE: To evaluate the association of adequate allocation concealment and patient blinding with estimates of treatment benefits in osteoarthritis trials. METHODS: We performed a meta-epidemiologic study of 16 meta-analyses with 175 trials that compared therapeutic interventions with placebo or nonintervention control in patients with hip or knee osteoarthritis. We calculated effect sizes from the differences in means of pain intensity between groups at the end of followup divided by the pooled SD and compared effect sizes between trials with and trials without adequate methodology. RESULTS: Effect sizes tended to be less beneficial in 46 trials with adequate allocation concealment compared with 112 trials with inadequate or unclear concealment of allocation (difference -0.15; 95% confidence interval [95% CI] -0.31, 0.02). Selection bias associated with inadequate or unclear concealment of allocation was most pronounced in meta-analyses with large estimated treatment benefits (P for interaction < 0.001), meta-analyses with high between-trial heterogeneity (P = 0.009), and meta-analyses of complementary medicine (P = 0.019). Effect sizes tended to be less beneficial in 64 trials with adequate blinding of patients compared with 58 trials without (difference -0.15; 95% CI -0.39, 0.09), but differences were less consistent and disappeared after accounting for allocation concealment. Detection bias associated with a lack of adequate patient blinding was most pronounced for nonpharmacologic interventions (P for interaction < 0.001). CONCLUSION: Results of osteoarthritis trials may be affected by selection and detection bias. Adequate concealment of allocation and attempts to blind patients will minimize these biases.

The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study

OBJECTIVE: To examine whether excluding patients from the analysis of randomised trials are associated with biased estimates of treatment effects and higher heterogeneity between trials. DESIGN: Meta-epidemiological study based on a collection of meta-analyses of randomised trials. DATA SOURCES: 14 meta-analyses including 167 trials that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patient reported pain as an outcome. METHODS: Effect sizes were calculated from differences in means of pain intensity between groups at the end of follow-up, divided by the pooled standard deviation. Trials were combined by using random effects meta-analysis. Estimates of treatment effects were compared between trials with and trials without exclusions from the analysis, and the impact of restricting meta-analyses to trials without exclusions was assessed. RESULTS: 39 trials (23%) had included all patients in the analysis. In 128 trials (77%) some patients were excluded from the analysis. Effect sizes from trials with exclusions tended to be more beneficial than those from trials without exclusions (difference -0.13, 95% confidence interval -0.29 to 0.04). However, estimates of bias between individual meta-analyses varied considerably (tau(2)=0.07). Tests of interaction between exclusions from the analysis and estimates of treatment effects were positive in five meta-analyses. Stratified analyses indicated that differences in effect sizes between trials with and trials without exclusions were more pronounced in meta-analyses with high between trial heterogeneity, in meta-analyses with large estimated treatment benefits, and in meta-analyses of complementary medicine. Restriction of meta-analyses to trials without exclusions resulted in smaller estimated treatment benefits, larger P values, and considerable decreases in between trial heterogeneity. CONCLUSION: Excluding patients from the analysis in randomised trials often results in biased estimates of treatment effects, but the extent and direction of bias is unpredictable. Results from intention to treat analyses should always be described in reports of randomised trials. In systematic reviews, the influence of exclusions from the analysis on estimated treatment effects should routinely be assessed.

Better conduct of clinical trials: the control group in critical care trials

OBJECTIVE: To review trial design issues related to control groups. DESIGN: Review of the literature with specific reference to critical care trials. MAIN RESULTS AND CONCLUSIONS: Performing randomized controlled trials in the critical care setting presents specific problems: studies include patients with rapidly lethal conditions, the majority of intensive care patients suffer from syndromes rather than from well-definable diseases, the severity of such syndromes cannot be precisely assessed, and the treatment consists of interacting therapies. Interactions between physiology, pathophysiology, and therapies are at best marginally understood and may have a major impact on study design and interpretation of results. Selection of the right control group is crucial for the interpretation and clinical implementation of results. Studies comparing new interventions with current ones or different levels of current treatments have the problem of the necessity of defining "usual care." Usual care controls without any constraints typically include substantial heterogeneity. Constraints in the usual therapy may help to reduce some variation. Inclusion of unrestricted usual care groups may help to enhance safety. Practice misalignment is a novel problem in which patients receive a treatment that is the direct opposite of usual care, and occurs when fixed-dose interventions are used in situations where care is normally titrated. Practice misalignment should be considered in the design and interpretation of studies on titrated therapies.

Cohort study of trials submitted to ethics committee identified discrepant reporting of outcomes in publications

OBJECTIVES To identify factors associated with discrepant outcome reporting in randomized drug trials. STUDY DESIGN AND SETTING Cohort study of protocols submitted to a Swiss ethics committee 1988-1998: 227 protocols and amendments were compared with 333 matching articles published during 1990-2008. Discrepant reporting was defined as addition, omission, or reclassification of outcomes. RESULTS Overall, 870 of 2,966 unique outcomes were reported discrepantly (29.3%). Among protocol-defined primary outcomes, 6.9% were not reported (19 of 274), whereas 10.4% of reported outcomes (30 of 288) were not defined in the protocol. Corresponding percentages for secondary outcomes were 19.0% (284 of 1,495) and 14.1% (334 of 2,375). Discrepant reporting was more likely if P values were <0.05 compared with P ≥ 0.05 [adjusted odds ratio (aOR): 1.38; 95% confidence interval (CI): 1.07, 1.78], more likely for efficacy compared with harm outcomes (aOR: 2.99; 95% CI: 2.08, 4.30) and more likely for composite than for single outcomes (aOR: 1.48; 95% CI: 1.00, 2.20). Cardiology (aOR: 2.34; 95% CI: 1.44, 3.79) and infectious diseases (aOR: 1.77; 95% CI: 1.01, 3.13) had more discrepancies compared with all specialties combined. CONCLUSION Discrepant reporting was associated with statistical significance of results, type of outcome, and specialty area. Trial protocols should be made freely available, and the publications should describe and justify any changes made to protocol-defined outcomes.