Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

OBJECTIVE: To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. DESIGN: Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses. DATA SOURCES: 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. MAIN OUTCOME MEASURES: Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates. RESULTS: In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. CONCLUSIONS: The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

Testing the allegiance bias hypothesis: a meta-analysis

This meta-analysis investigated whether the association between researcher allegiance (RA) and the relative effect of two psychotherapies can be explained through the methodological weaknesses of the treatment comparisons. Seventy-nine comparisons of psychotherapies for depression or posttraumatic stress disorder (PTSD) were included. Methodological quality (MQ) was investigated as both a moderator and a mediator of the RA-outcome association. MQ included balanced nonspecific factors, balanced specific factors, conceptual quality, patients-per-therapist ratio, randomization to conditions and outcome assessment. The RA-outcome association was stronger when the MQ was low, suggesting a buffering effect of MQ. In addition, differences in the conceptual quality of treatments mediated the effect of RA on outcome. The results support the view that RA acts as a bias in treatment comparisons.

Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials

Published evidence suggests that aspects of trial design lead to biased intervention effect estimates, but findings from different studies are inconsistent. This study combined data from 7 meta-epidemiologic studies and removed overlaps to derive a final data set of 234 unique meta-analyses containing 1973 trials. Outcome measures were classified as "mortality," "other objective," "or subjective," and Bayesian hierarchical models were used to estimate associations of trial characteristics with average bias and between-trial heterogeneity. Intervention effect estimates seemed to be exaggerated in trials with inadequate or unclear (vs. adequate) random-sequence generation (ratio of odds ratios, 0.89 [95% credible interval {CrI}, 0.82 to 0.96]) and with inadequate or unclear (vs. adequate) allocation concealment (ratio of odds ratios, 0.93 [CrI, 0.87 to 0.99]). Lack of or unclear double-blinding (vs. double-blinding) was associated with an average of 13% exaggeration of intervention effects (ratio of odds ratios, 0.87 [CrI, 0.79 to 0.96]), and between-trial heterogeneity was increased for such studies (SD increase in heterogeneity, 0.14 [CrI, 0.02 to 0.30]). For each characteristic, average bias and increases in between-trial heterogeneity were driven primarily by trials with subjective outcomes, with little evidence of bias in trials with objective and mortality outcomes. This study is limited by incomplete trial reporting, and findings may be confounded by other study design characteristics. Bias associated with study design characteristics may lead to exaggeration of intervention effect estimates and increases in between-trial heterogeneity in trials reporting subjectively assessed outcomes.

Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study

BACKGROUND Empirical research has illustrated an association between study size and relative treatment effects, but conclusions have been inconsistent about the association of study size with the risk of bias items. Small studies give generally imprecisely estimated treatment effects, and study variance can serve as a surrogate for study size. METHODS We conducted a network meta-epidemiological study analyzing 32 networks including 613 randomized controlled trials, and used Bayesian network meta-analysis and meta-regression models to evaluate the impact of trial characteristics and study variance on the results of network meta-analysis. We examined changes in relative effects and between-studies variation in network meta-regression models as a function of the variance of the observed effect size and indicators for the adequacy of each risk of bias item. Adjustment was performed both within and across networks, allowing for between-networks variability. RESULTS Imprecise studies with large variances tended to exaggerate the effects of the active or new intervention in the majority of networks, with a ratio of odds ratios of 1.83 (95% CI: 1.09,3.32). Inappropriate or unclear conduct of random sequence generation and allocation concealment, as well as lack of blinding of patients and outcome assessors, did not materially impact on the summary results. Imprecise studies also appeared to be more prone to inadequate conduct. CONCLUSIONS Compared to more precise studies, studies with large variance may give substantially different answers that alter the results of network meta-analyses for dichotomous outcomes.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers. METHODS/DESIGN This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals. DISCUSSION This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.

The effect of cumulating exposure to abacavir on the risk of cardiovascular disease events in patients from the Swiss HIV Cohort Study.

BACKGROUND Patients with HIV exposed to the antiretroviral drug abacavir may have an increased risk of cardiovascular disease (CVD). There is concern that this association arises because of a channelling bias. Even if exposure is a risk, it is not clear how that risk changes as exposure cumulates. METHODS We assess the effect of exposure to abacavir on the risk of CVD events in the Swiss HIV Cohort Study. We use a new marginal structural Cox model to estimate the effect of abacavir as a flexible function of past exposures while accounting for risk factors that potentially lie on a causal pathway between exposure to abacavir and CVD. RESULTS 11,856 patients were followed for a median of 6.6 years; 365 patients had a CVD event (4.6 events per 1000 patient years). In a conventional Cox model, recent - but not cumulative - exposure to abacavir increased the risk of a CVD event. In the new marginal structural Cox model, continued exposure to abacavir during the past four years increased the risk of a CVD event (hazard ratio 2.06, 95% confidence interval 1.43-2.98). The estimated function for the effect of past exposures suggests that exposure during the past 6 to 36 months caused the greatest increase in risk. CONCLUSIONS Abacavir increases the risk of a CVD event: the effect of exposure is not immediate, rather the risk increases as exposure cumulates over the past few years. This gradual increase in risk is not consistent with a rapidly acting mechanism, such as acute inflammation.

8p23 beta-defensin copy number determination by single-locus pseudogene-based paralog ratio tests risk bias due to low-frequency sequence variations.

BACKGROUND The copy number variation (CNV) in beta-defensin genes (DEFB) on human chromosome 8p23 has been proposed to contribute to the phenotypic differences in inflammatory diseases. However, determination of exact DEFB CN is a major challenge in association studies. Quantitative real-time PCR (qPCR), paralog ratio tests (PRT) and multiplex ligation-dependent probe amplification (MLPA) have been extensively used to determine DEFB CN in different laboratories, but inter-method inconsistencies were observed frequently. In this study we asked which one is superior among the three methods for DEFB CN determination. RESULTS We developed a clustering approach for MLPA and PRT to statistically correlate data from a single experiment. Then we compared qPCR, a newly designed PRT and MLPA for DEFB CN determination in 285 DNA samples. We found MLPA had the best convergence and clustering results of the raw data and the highest call rate. In addition, the concordance rates between MLPA or PRT and qPCR (32.12% and 37.99%, respectively) were unacceptably low with underestimated CN by qPCR. Concordance rate between MLPA and PRT (90.52%) was high but PRT systematically underestimated CN by one in a subset of samples. In these samples a sequence variant which caused complete PCR dropout of the respective DEFB cluster copies was found in one primer binding site of one of the targeted paralogous pseudogenes. CONCLUSION MLPA is superior to PRT and even more to qPCR for DEFB CN determination. Although the applied PRT provides in most cases reliable results, such a test is particularly sensitive to low-frequency sequence variations preferably accumulating in loci like pseudogenes which are most likely not under selective pressure. In the light of the superior performance of multiplex assays, the drawbacks of such single PRTs could be overcome by combining more test markers.

'Alternate-goal bias' in antisaccades and the influence of expectation

Saccadic performance depends on the requirements of the current trial, but also may be influenced by other trials in the same experiment. This effect of trial context has been investigated most for saccadic error rate and reaction time but seldom for the positional accuracy of saccadic landing points. We investigated whether the direction of saccades towards one goal is affected by the location of a second goal used in other trials in the same experimental block. In our first experiment, landing points ('endpoints') of antisaccades but not prosaccades were shifted towards the location of the alternate goal. This spatial bias decreased with increasing angular separation between the current and alternative goals. In a second experiment, we explored whether expectancy about the goal location was responsible for the biasing of the saccadic endpoint. For this, we used a condition where the saccadic goal randomly changed from one trial to the next between locations on, above or below the horizontal meridian. We modulated the prior probability of the alternate-goal location by showing cues prior to stimulus onset. The results showed that expectation about the possible positions of the saccadic goal is sufficient to bias saccadic endpoints and can account for at least part of this phenomenon of 'alternate-goal bias'.

Avoidance of Long Mononucleotide Repeats in Codon Pair Usage

Protein is an essential component for life, and its synthesis is mediated by codons in any organisms on earth. While some codons encode the same amino acid, their usage is often highly biased. There are many factors that can cause the bias, but a potential effect of mononucleotide repeats, which are known to be highly mutable, on codon usage and codon pair preference is largely unknown. In this study we performed a genomic survey on the relationship between mononucleotide repeats and codon pair bias in 53 bacteria, 68 archaea, and 13 eukaryotes. By distinguishing the codon pair bias from the codon usage bias, four general patterns were revealed: strong avoidance of five or six mononucleotide repeats in codon pairs; lower observed/expected (o/e) ratio for codon pairs with C or G repeats (C/G pairs) than that with A or T repeats (A/T pairs); a negative correlation between genomic GC contents and the o/e ratios, particularly for C/G pairs; and avoidance of C/G pairs in highly conserved genes. These results support natural selection against long mononucleotide repeats, which could induce frameshift mutations in coding sequences. The fact that these patterns are found in all kingdoms of life suggests that this is a general phenomenon in living organisms. Thus, long mononucleotide repeats may play an important role in base composition and genetic stability of a gene and gene functions.

The effect of 3 versus 6 years of zoledronic acid treatment of osteoporosis: a randomized extension to the HORIZON-Pivotal Fracture Trial (PFT)

Zoledronic acid 5 mg (ZOL) annually for 3 years reduces fracture risk in postmenopausal women with osteoporosis. To investigate long-term effects of ZOL on bone mineral density (BMD) and fracture risk, the Health Outcomes and Reduced Incidence with Zoledronic acid Once Yearly-Pivotal Fracture Trial (HORIZON-PFT) was extended to 6 years. In this international, multicenter, double-blind, placebo-controlled extension trial, 1233 postmenopausal women who received ZOL for 3 years in the core study were randomized to 3 additional years of ZOL (Z6, n = 616) or placebo (Z3P3, n = 617). The primary endpoint was femoral neck (FN) BMD percentage change from year 3 to 6 in the intent-to-treat (ITT) population. Secondary endpoints included other BMD sites, fractures, biochemical bone turnover markers, and safety. In years 3 to 6, FN-BMD remained constant in Z6 and dropped slightly in Z3P3 (between-treatment difference = 1.04%; 95% confidence interval 0.4 to 1.7; p = 0.0009) but remained above pretreatment levels. Other BMD sites showed similar differences. Biochemical markers remained constant in Z6 but rose slightly in Z3P3, remaining well below pretreatment levels in both. New morphometric vertebral fractures were lower in the Z6 (n = 14) versus Z3P3 (n = 30) group (odds ratio = 0.51; p = 0.035), whereas other fractures were not different. Significantly more Z6 patients had a transient increase in serum creatinine >0.5 mg/dL (0.65% versus 2.94% in Z3P3). Nonsignificant increases in Z6 of atrial fibrillation serious adverse events (2.0% versus 1.1% in Z3P3; p = 0.26) and stroke (3.1% versus 1.5% in Z3P3; p = 0.06) were seen. Postdose symptoms were similar in both groups. Reports of hypertension were significantly lower in Z6 versus Z3P3 (7.8% versus 15.1%, p < 0.001). Small differences in bone density and markers in those who continued versus those who stopped treatment suggest residual effects, and therefore, after 3 years of annual ZOL, many patients may discontinue therapy up to 3 years. However, vertebral fracture reductions suggest that those at high fracture risk, particularly vertebral fracture, may benefit by continued treatment.

Effect of Mutation and Genetic Background on Drug Resistance in Mycobacterium tuberculosis

Bacterial factors may contribute to the global emergence and spread of drug-resistant tuberculosis (TB). Only a few studies have reported on the interactions between different bacterial factors. We studied drug-resistant Mycobacterium tuberculosis isolates from a nationwide study conducted from 2000 to 2008 in Switzerland. We determined quantitative drug resistance levels of first-line drugs by using Bactec MGIT-960 and drug resistance genotypes by sequencing the hot-spot regions of the relevant genes. We determined recent transmission by molecular methods and collected clinical data. Overall, we analyzed 158 isolates that were resistant to isoniazid, rifampin, or ethambutol, 48 (30.4%) of which were multidrug resistant. Among 154 isoniazid-resistant strains, katG mutations were associated with high-level and inhA promoter mutations with low-level drug resistance. Only katG(S315T) (65.6% of all isoniazid-resistant strains) and inhA promoter -15C/T (22.7%) were found in molecular clusters. M. tuberculosis lineage 2 (includes Beijing genotype) was associated with any drug resistance (adjusted odds ratio [OR], 3.0; 95% confidence interval [CI], 1.7 to 5.6; P < 0.0001). Lineage 1 was associated with inhA promoter -15C/T mutations (OR, 6.4; 95% CI, 2.0 to 20.7; P = 0.002). We found that the genetic strain background influences the level of isoniazid resistance conveyed by particular mutations (interaction tests of drug resistance mutations across all lineages; P < 0.0001). In conclusion, M. tuberculosis drug resistance mutations were associated with various levels of drug resistance and transmission, and M. tuberculosis lineages were associated with particular drug resistance-conferring mutations and phenotypic drug resistance. Our study also supports a role for epistatic interactions between different drug resistance mutations and strain genetic backgrounds in M. tuberculosis drug resistance.