Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

OBJECTIVE: To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. DESIGN: Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses. DATA SOURCES: 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. MAIN OUTCOME MEASURES: Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates. RESULTS: In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. CONCLUSIONS: The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

Subjective wellbeing under quetiapine treatment: effect of diagnosis, mood state, and anxiety

To examine the effect of diagnosis, mood state, and anxiety on subjective wellbeing in patients with affective and non-affective psychotic disorders treated with quetiapine IR.

Is the sedimentation sign associated with spinal stenosis surgical treatment effect in SPORT?

STUDY DESIGN Subgroup analysis of the lumbar spinal stenosis (LSS) without degenerative spondylolisthesis diagnostic cohort of the Spine Patient Outcomes Research Trial multicenter randomized clinical trial with a concurrent observational cohort. OBJECTIVE To determine if sedimentation sign on magnetic resonance image can help with LSS treatment decisions. SUMMARY OF BACKGROUND DATA LSS is one of the most common reasons for surgery in the US elderly, but there is a dearth of reliable diagnostic tools that give a clear indication for surgery. Recent studies have suggested that positive sedimentation sign on magnetic resonance image may be a possible prognostic indicator. METHODS All patients with LSS in both the randomized and observational cohorts had imaging-confirmed stenosis, were surgical candidates, and had neurogenic claudication for at least 12 weeks prior to enrollment. Patients were categorized as "mild," "moderate," or "severe" according to stenosis severity. Of the 654 patients with LSS enrolled in Spine Patient Outcomes Research Trial, complete T2-weighted axial and sagittal digitized images of 115 patients were available for retrospective review. An independent orthopedic spine surgeon evaluated these deidentified Digital Imaging and Communications in Medicine files for the sedimentation sign. RESULTS Sixty-six percent (76/115) of patients were found to have a positive sedimentation sign. Those with a positive sedimentation sign were more likely to have stenosis at L2-L3 (33% vs. 10% P=0.016) or L3-L4 76% vs. 51%, P=0.012), and to have severe (72% vs. 33%, P<0.0001) central stenosis (93% vs. 67% P<0.001) at 2 or more concurrent levels (57% vs. 18%, P=0.01). In multivariate models, the surgical treatment effect was significantly larger in the positive sedimentation sign group for Oswestry Disability Index (-16 vs. -7; P=0.02). CONCLUSION A positive sedimentation sign was associated with a small but significantly greater surgical treatment effect for Oswestry Disability Index in patients with symptomatic LSS, after adjusting for other demographic and imaging features. These findings suggest that positive sedimentation sign may potentially be a useful adjunct to help guide an informed treatment choice regarding surgery for LSS. LEVEL OF EVIDENCE 2.

Clinical effectiveness of cognitive behavioral therapy for depression in routine care: A propensity score based comparison between randomized controlled trials and clinical practice

Background: The efficacy of cognitive behavioral therapy (CBT) for the treatment of depressive disorders has been demonstrated in many randomized controlled trials (RCTs). This study investigated whether for CBT similar effects can be expected under routine care conditions when the patients are comparable to those examined in RCTs. Method: N=574 CBT patients from an outpatient clinic were stepwise matched to the patients undergoing CBT in the National Institute of Mental Health Treatment of Depression Collaborative Research Program (TDCRP). First, the exclusion criteria of the RCT were applied to the naturalistic sample of the outpatient clinic. Second, propensity score matching (PSM) was used to adjust the remaining naturalistic sample on the basis of baseline covariate distributions. Matched samples were then compared regarding treatment effects using effect sizes, average treatment effect on the treated (ATT) and recovery rates. Results: CBT in the adjusted naturalistic subsample was as effective as in the RCT. However, treatments lasted significantly longer under routine care conditions. Limitations: The samples included only a limited amount of common predictor variables and stemmed from different countries. There might be additional covariates, which could potentially further improve the matching between the samples. Conclusions: CBT for depression in clinical practice might be equally effective as manual-based treatments in RCTs when they are applied to comparable patients. The fact that similar effects under routine conditions were reached with more sessions, however, points to the potential to optimize treatments in clinical practice with respect to their efficiency.

Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study

Objective To examine the presence and extent of small study effects in clinical osteoarthritis research. Design Meta-epidemiological study. Data sources 13 meta-analyses including 153 randomised trials (41 605 patients) that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patients’ reported pain as an outcome. Methods We compared estimated benefits of treatment between large trials (at least 100 patients per arm) and small trials, explored funnel plots supplemented with lines of predicted effects and contours of significance, and used three approaches to estimate treatment effects: meta-analyses including all trials irrespective of sample size, meta-analyses restricted to large trials, and treatment effects predicted for large trials. Results On average, treatment effects were more beneficial in small than in large trials (difference in effect sizes −0.21, 95% confidence interval −0.34 to −0.08, P=0.001). Depending on criteria used, six to eight funnel plots indicated small study effects. In six of 13 meta-analyses, the overall pooled estimate suggested a clinically relevant, significant benefit of treatment, whereas analyses restricted to large trials and predicted effects in large trials yielded smaller non-significant estimates. Conclusions Small study effects can often distort results of meta-analyses. The influence of small trials on estimated treatment effects should be routinely assessed.

Methods to convert continuous outcomes into odds ratios of treatment response and numbers needed to treat: meta-epidemiological study

Clinicians find standardized mean differences (SMDs) calculated from continuous outcomes difficult to interpret. Our objective was to determine the performance of methods in converting SMDs or means to odds ratios of treatment response and numbers needed to treat (NNTs) as more intuitive measures of treatment effect.

Grey literature in meta-analyses of randomized trials of health care interventions

BACKGROUND: The inclusion of grey literature (i.e. literature that has not been formally published) in systematic reviews may help to overcome some of the problems of publication bias, which can arise due to the selective availability of data. OBJECTIVES: To review systematically research studies, which have investigated the impact of grey literature in meta-analyses of randomized trials of health care interventions. SEARCH STRATEGY: We searched the Cochrane Methodology Register (The Cochrane Library Issue 3, 2005), MEDLINE (1966 to 20 May 2005), the Science Citation Index (June 2005) and contacted researchers who may have carried out relevant studies. SELECTION CRITERIA: A study was considered eligible for this review if it compared the effect of the inclusion and exclusion of grey literature on the results of a cohort of meta-analyses of randomized trials. DATA COLLECTION AND ANALYSIS: Data were extracted from each report independently by two reviewers. The main outcome measure was an estimate of the impact of trials from the grey literature on the pooled effect estimates of the meta-analyses. Information was also collected on the area of health care, the number of meta-analyses, the number of trials, the number of trial participants, the year of publication of the trials, the language and country of publication of the trials, the number and type of grey and published literature, and methodological quality. MAIN RESULTS: Five studies met the inclusion criteria. All five studies showed that published trials showed an overall greater treatment effect than grey trials. This difference was statistically significant in one of the five studies. Data could be combined for three of the five studies. This showed that, on average, published trials showed a 9% greater treatment effect than grey trials (ratio of odds ratios for grey versus published trials 1.09; 95% CI 1.03-1.16). Overall there were more published trials included in the meta-analyses than grey trials (median 224 (IQR 108-365) versus 45(IQR 40-102)). Published trials had more participants on average. The most common types of grey literature were abstracts (55%) and unpublished data (30%). There is limited evidence to show whether grey trials are of poorer methodological quality than published trials. AUTHORS' CONCLUSIONS: This review shows that published trials tend to be larger and show an overall greater treatment effect than grey trials. This has important implications for reviewers who need to ensure they identify grey trials, in order to minimise the risk of introducing bias into their review.

Native EEG and treatment effects in neuroleptic-naïve schizophrenic patients: time and frequency domain approaches

Time domain analysis of electroencephalography (EEG) can identify subsecond periods of quasi-stable brain states. These so-called microstates assumingly correspond to basic units of cognition and emotion. On the other hand, Global Field Synchronization (GFS) is a frequency domain measure to estimate functional synchronization of brain processes on a global level for each EEG frequency band [Koenig, T., Lehmann, D., Saito, N., Kuginuki, T., Kinoshita, T., Koukkou, M., 2001. Decreased functional connectivity of EEG theta-frequency activity in first-episode, neuroleptic-naive patients with schizophrenia: preliminary results. Schizophr Res. 50, 55-60.]. Using these time and frequency domain analyzes, several previous studies reported shortened microstate duration in specific microstate classes and decreased GFS in theta band in drug naïve schizophrenia compared to controls. The purpose of this study was to investigate changes of these EEG parameters after drug treatment in drug naïve schizophrenia. EEG analysis was performed in 21 drug-naive patients and 21 healthy controls. 14 patients were reevaluated 2-8 weeks (mean 4.3) after the initiation of drug administration. The results extended findings of treatment effect on brain functions in schizophrenia, and imply that shortened duration of specific microstate classes seems a state marker especially in patients with later neuroleptic responsive, while lower theta GFS seems a state-related phenomenon and that higher gamma GFS is a trait like phenomenon.

Randomised controlled trials of homeopathy in hyperactive children: treatment procedure leads to an unconventional study design. Experience with open-label homeopathic treatment preceding the Swiss ADHD placebo controlled, randomised, double-blind, cross-over trial

BACKGROUND: Treatment of patients with attention deficit hyperactivity disorder (ADHD) with homeopathy is difficult. The Swiss randomised, placebo controlled, cross-over trial in ADHD patients (Swiss ADHD trial) was designed with an open-label screening phase prior to the randomised controlled phase. During the screening phase, the response of each child to successive homeopathic medications was observed until the optimal medication was identified. Only children who reached a predefined level of improvement participated in the randomised, cross-over phase. Although the randomised phase revealed a significant beneficial effect of homeopathy, the cross-over caused a strong carryover effect diminishing the apparent difference between placebo and verum treatment. METHODS: This retrospective analysis explores the screening phase data with respect to the risk of failure to demonstrate a specific effect of a randomised controlled trial (RCT) with randomisation at the start of the treatment. RESULTS: During the screening phase, 84% (70/83) of the children responded to treatment and reached eligibility for the randomised trial after a median time of 5 months (range 1-18), with a median of 3 different medications (range 1-9). Thirteen children (16%) did not reach eligibility. Five months after treatment start, the difference in Conners Global Index (CGI) rating between responders and non-responders became highly significant (p = 0.0006). Improvement in CGI was much greater following the identification of the optimal medication than in the preceding suboptimal treatment period (p < 0.0001). CONCLUSIONS: Because of the necessity of identifying an optimal medication before response to treatment can be expected, randomisation at the start of treatment in an RCT of homeopathy in ADHD children has a high risk of failure to demonstrate a specific treatment effect, if the observation time is shorter than 12 months.

Differences across health care systems in outcome and cost-utility of surgical and conservative treatment of chronic low back pain: a study protocol

BACKGROUND: There is little evidence on differences across health care systems in choice and outcome of the treatment of chronic low back pain (CLBP) with spinal surgery and conservative treatment as the main options. At least six randomised controlled trials comparing these two options have been performed; they show conflicting results without clear-cut evidence for superior effectiveness of any of the evaluated interventions and could not address whether treatment effect varied across patient subgroups. Cost-utility analyses display inconsistent results when comparing surgical and conservative treatment of CLBP. Due to its higher feasibility, we chose to conduct a prospective observational cohort study. METHODS: This study aims to examine if1. Differences across health care systems result in different treatment outcomes of surgical and conservative treatment of CLBP2. Patient characteristics (work-related, psychological factors, etc.) and co-interventions (physiotherapy, cognitive behavioural therapy, return-to-work programs, etc.) modify the outcome of treatment for CLBP3. Cost-utility in terms of quality-adjusted life years differs between surgical and conservative treatment of CLBP.This study will recruit 1000 patients from orthopaedic spine units, rehabilitation centres, and pain clinics in Switzerland and New Zealand. Effectiveness will be measured by the Oswestry Disability Index (ODI) at baseline and after six months. The change in ODI will be the primary endpoint of this study.Multiple linear regression models will be used, with the change in ODI from baseline to six months as the dependent variable and the type of health care system, type of treatment, patient characteristics, and co-interventions as independent variables. Interactions will be incorporated between type of treatment and different co-interventions and patient characteristics. Cost-utility will be measured with an index based on EQol-5D in combination with cost data. CONCLUSION: This study will provide evidence if differences across health care systems in the outcome of treatment of CLBP exist. It will classify patients with CLBP into different clinical subgroups and help to identify specific target groups who might benefit from specific surgical or conservative interventions. Furthermore, cost-utility differences will be identified for different groups of patients with CLBP. Main results of this study should be replicated in future studies on CLBP.

Homeopathic treatment of Arabidopsis thaliana plants infected with Pseudomonas syringae

Homeopathic basic research is still in the screening phase to identify promising model systems that are adapted to the needs and peculiarities of homeopathic medicine and pharmacy. We investigated the potential of a common plant-pathogen system, Arabidopsis thaliana infected with the virulent bacteria Pseudomonas syringae, regarding its response towards a homeopathic treatment. A. thaliana plants were treated with homeopathic preparations before and after infection. Outcome measure was the number of P. syringae bacteria in the leaves of A. thaliana, assessed in randomized and blinded experiments. After a screening of 30 homeopathic preparations, we investigated the effect of Carbo vegetabilis 30x, Magnesium phosphoricum 30x, Nosode 30x, Biplantol (a homeopathic complex remedy), and Biplantol 30x on the infection rate in five or six independent experiments in total. The screening yielded significant effects for four out of 30 tested preparations. In the repeated experimental series, only the homeopathic complex remedy Biplantol induced a significant reduction of the infection rate (p = 0.01; effect size, d = 0.38). None of the other four repeatedly tested preparations (Carbo vegetabilis 30x, Magnesium phosphoricum 30x, Nosode 30x, Biplantol 30x) yielded significant effects in the overall evaluation. This phytopathological model yielded a small to medium effect size and thus might be of interest for homeopathic basic research after further improvement. Compared to Bion (a common SAR inducer used as positive control), the magnitude of the treatment effect of Biplantol was about 50%. Thus, homeopathic formulations might have a potential for the treatment of plant diseases after further optimization. However, the ecological impact should be investigated more closely before widespread application.

Antiplatelets versus Anticoagulants for the Treatment of Cervical Artery Dissection: Bayesian Meta-Analysis

OBJECTIVE To compare the effects of antiplatelets and anticoagulants on stroke and death in patients with acute cervical artery dissection. DESIGN Systematic review with Bayesian meta-analysis. DATA SOURCES The reviewers searched MEDLINE and EMBASE from inception to November 2012, checked reference lists, and contacted authors. STUDY SELECTION Studies were eligible if they were randomised, quasi-randomised or observational comparisons of antiplatelets and anticoagulants in patients with cervical artery dissection. DATA EXTRACTION Data were extracted by one reviewer and checked by another. Bayesian techniques were used to appropriately account for studies with scarce event data and imbalances in the size of comparison groups. DATA SYNTHESIS Thirty-seven studies (1991 patients) were included. We found no randomised trial. The primary analysis revealed a large treatment effect in favour of antiplatelets for preventing the primary composite outcome of ischaemic stroke, intracranial haemorrhage or death within the first 3 months after treatment initiation (relative risk 0.32, 95% credibility interval 0.12 to 0.63), while the degree of between-study heterogeneity was moderate (τ(2) = 0.18). In an analysis restricted to studies of higher methodological quality, the possible advantage of antiplatelets over anticoagulants was less obvious than in the main analysis (relative risk 0.73, 95% credibility interval 0.17 to 2.30). CONCLUSION In view of these results and the safety advantages, easier usage and lower cost of antiplatelets, we conclude that antiplatelets should be given precedence over anticoagulants as a first line treatment in patients with cervical artery dissection unless results of an adequately powered randomised trial suggest the opposite.

Combining follow-up and change data is valid in meta-analyses of continuous outcomes: a meta-epidemiological study

OBJECTIVE To investigate whether it is valid to combine follow-up and change data when conducting meta-analyses of continuous outcomes. STUDY DESIGN AND SETTING Meta-epidemiological study of randomized controlled trials in patients with osteoarthritis of the knee/hip, which assessed patient-reported pain. We calculated standardized mean differences (SMDs) based on follow-up and change data, and pooled within-trial differences in SMDs. We also derived pooled SMDs indicating the largest treatment effect within a trial (optimistic selection of SMDs) and derived pooled SMDs from the estimate indicating the smallest treatment effect within a trial (pessimistic selection of SMDs). RESULTS A total of 21 meta-analyses with 189 trials with 292 randomized comparisons in 41,256 patients were included. On average, SMDs were 0.04 standard deviation units more beneficial when follow-up values were used (difference in SMDs: -0.04; 95% confidence interval: -0.13, 0.06; P=0.44). In 13 meta-analyses (62%), there was a relevant difference in clinical and/or significance level between optimistic and pessimistic pooled SMDs. CONCLUSION On average, there is no relevant difference between follow-up and change data SMDs, and combining these estimates in meta-analysis is generally valid. Decision on which type of data to use when both follow-up and change data are available should be prespecified in the meta-analysis protocol.