Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

OBJECTIVE: To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. DESIGN: Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses. DATA SOURCES: 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. MAIN OUTCOME MEASURES: Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates. RESULTS: In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. CONCLUSIONS: The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Multivariable adjustments counteract spectrum and test review bias in accuracy studies

OBJECTIVES: The STAndards for Reporting studies of Diagnostic accuracy (STARD) for investigators and editors and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) for reviewers and readers offer guidelines for the quality and reporting of test accuracy studies. These guidelines address and propose some solutions to two major threats to validity: spectrum bias and test review bias. STUDY DESIGN AND SETTING: Using a clinical example, we demonstrate that these solutions fail and propose an alternative solution that concomitantly addresses both sources of bias. We also derive formulas that prove the generality of our arguments. RESULTS: A logical extension of our ideas is to extend STARD item 23 by adding a requirement for multivariable statistical adjustment using information collected in QUADAS items 1, 2, and 12 and STARD items 3-5, 11, 15, and 18. CONCLUSION: We recommend reporting not only variation of diagnostic accuracy across subgroups (STARD item 23) but also the effects of the multivariable adjustments on test performance. We also suggest that the QUADAS be supplemented by an item addressing the appropriateness of statistical methods, in particular whether multivariable adjustments have been included in the analysis.

TH2 cytokines from malignant cells suppress TH1 responses and enforce a global TH2 bias in leukemic cutaneous T-cell lymphoma

PURPOSE In leukemic cutaneous T-cell lymphoma (L-CTCL), malignant T cells accumulate in the blood and give rise to widespread skin inflammation. Patients have intense pruritus, increased immunoglobulin E (IgE), and decreased T-helper (TH)-1 responses, and most die from infection. Depleting malignant T cells while preserving normal immunity is a clinical challenge. L-CTCL has been variably described as a malignancy of regulatory, TH2 and TH17 cells. EXPERIMENTAL DESIGN We analyzed phenotype and cytokine production in malignant and benign L-CTCL T cells, characterized the effects of malignant T cells on healthy T cells, and studied the immunomodulatory effects of treatment modalities in patients with L-CTCL. RESULTS Twelve out of 12 patients with L-CTCL overproduced TH2 cytokines. Remaining benign T cells were also strongly TH2 biased, suggesting a global TH2 skewing of the T-cell repertoire. Culture of benign T cells away from the malignant clone reduced TH2 and enhanced TH1 responses, but separate culture had no effect on malignant T cells. Coculture of healthy T cells with L-CTCL T cells reduced IFNγ production and neutralizing antibodies to interleukin (IL)-4 and IL-13 restored TH1 responses. In patients, enhanced TH1 responses were observed following a variety of treatment modalities that reduced malignant T-cell burden. CONCLUSIONS A global TH2 bias exists in both benign and malignant T cells in L-CTCL and may underlie the infectious susceptibility of patients. TH2 cytokines from malignant cells strongly inhibited TH1 responses. Our results suggest that therapies that inhibit TH2 cytokine activity, by virtue of their ability to improve TH1 responses, may have the potential to enhance both anticancer and antipathogen responses.

A test of genetic association among male nuptial coloration, female mating preference, and male aggression bias within a polymorphic population of cichlid fish

Both inter- and intrasexual selection have been implicated in the origin and maintenance of species-rich taxa with diverse sexual traits. Simultaneous disruptive selection by female mate choice and male-male competition can, in theory, lead to speciation without geographical isolation if both act on the same male trait. Female mate choice can generate discontinuities in gene flow, while male-male competition can generate negative frequency-dependent selection stabilizing the male trait polymorphism. Speciation may be facilitated when mating preference and/or aggression bias are physically linked to the trait they operate on. We tested for genetic associations among female mating preference, male aggression bias and male coloration in the Lake Victoria cichlid Pundamilia. We crossed females from a phenotypically variable population with males from both extreme ends of the phenotype distribution in the same population (blue or red). Male offspring of a red sire were significantly redder than males of a blue sire, indicating that intra-population variation in male coloration is heritable. We tested mating preferences of female offspring and aggression biases of male offspring using binary choice tests. There was no evidence for associations at the family level between female mating preferences and coloration of sires, but dam identity had a significant effect on female mate preference. Sons of the red sire directed significantly more aggression to red than blue males, whereas sons of the blue sire did not show any bias. There was a positive correlation among individuals between male aggression bias and body coloration, possibly due to pleiotropy or physical linkage, which could facilitate the maintenance of color polymorphism.