Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

OBJECTIVE: To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. DESIGN: Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses. DATA SOURCES: 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. MAIN OUTCOME MEASURES: Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates. RESULTS: In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. CONCLUSIONS: The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Multivariable adjustments counteract spectrum and test review bias in accuracy studies

OBJECTIVES: The STAndards for Reporting studies of Diagnostic accuracy (STARD) for investigators and editors and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) for reviewers and readers offer guidelines for the quality and reporting of test accuracy studies. These guidelines address and propose some solutions to two major threats to validity: spectrum bias and test review bias. STUDY DESIGN AND SETTING: Using a clinical example, we demonstrate that these solutions fail and propose an alternative solution that concomitantly addresses both sources of bias. We also derive formulas that prove the generality of our arguments. RESULTS: A logical extension of our ideas is to extend STARD item 23 by adding a requirement for multivariable statistical adjustment using information collected in QUADAS items 1, 2, and 12 and STARD items 3-5, 11, 15, and 18. CONCLUSION: We recommend reporting not only variation of diagnostic accuracy across subgroups (STARD item 23) but also the effects of the multivariable adjustments on test performance. We also suggest that the QUADAS be supplemented by an item addressing the appropriateness of statistical methods, in particular whether multivariable adjustments have been included in the analysis.

Reverse total shoulder arthroplasty

The radical changes in prosthetic design made in the mid 1980s transformed the historically poorly performing reverse ball-and-socket total shoulder prosthesis into a highly successful salvage implant for pseudoparalytic, severely rotator cuff-deficient shoulders. Moving the center of rotation more medial and distal as well as implanting a large glenoid hemisphere that articulates with a humeral cup in 155 degrees of valgus are the biomechanical keys to sometimes spectacular short- to mid-term results. Use of the reverse total shoulder arthroplasty device allows salvage of injuries that previously were beyond surgical treatment. However, this technique has a complication rate approximately three times that of conventional arthroplasty. Radiographic and clinical results appear to deteriorate over time. Proper patient selection and attention to technical details are needed to reduce the currently high complication rate.

Novel functional profiling approach combining reverse phase protein microarrays and human 3-D ex vivo tissue cultures: expression of apoptosis-related proteins in human colon cancer

Cancer is caused by a complex pattern of molecular perturbations. To understand the biology of cancer, it is thus important to look at the activation state of key proteins and signaling networks. The limited amount of available sample material from patients and the complexity of protein expression patterns make the use of traditional protein analysis methods particularly difficult. In addition, the only approach that is currently available for performing functional studies is the use of serial biopsies, which is limited by ethical constraints and patient acceptance. The goal of this work was to establish a 3-D ex vivo culture technique in combination with reverse-phase protein microarrays (RPPM) as a novel experimental tool for use in cancer research. The RPPM platform allows the parallel profiling of large numbers of protein analytes to determine their relative abundance and activation level. Cancer tissue and the respective corresponding normal tissue controls from patients with colorectal cancer were cultured ex vivo. At various time points, the cultured samples were processed into lysates and analyzed on RPPM to assess the expression of carcinoembryonic antigen (CEA) and 24 proteins involved in the regulation of apoptosis. The methodology displayed good robustness and low system noise. As a proof of concept, CEA expression was significantly higher in tumor compared with normal tissue (p<0.0001). The caspase 9 expression signal was lower in tumor tissue than in normal tissue (p<0.001). Cleaved Caspase 8 (p=0.014), Bad (p=0.007), Bim (p=0.007), p73 (p=0.005), PARP (p<0.001), and cleaved PARP (p=0.007) were differentially expressed in normal liver and normal colon tissue. We demonstrate here the feasibility of using RPPM technology with 3-D ex vivo cultured samples. This approach is useful for investigating complex patterns of protein expression and modification over time. It should allow functional proteomics in patient samples with various applications such as pharmacodynamic analyses in drug development.

Enriching Reverse Engineering with Annotations

Much of the knowledge about software systems is implicit, and therefore difficult to recover by purely automated techniques. Architectural layers and the externally visible features of software systems are two examples of information that can be difficult to detect from source code alone, and that would benefit from additional human knowledge. Typical approaches to reasoning about data involve encoding an explicit meta-model and expressing analyses at that level. Due to its informal nature, however, human knowledge can be difficult to characterize up-front and integrate into such a meta-model. We propose a generic, annotation-based approach to capture such knowledge during the reverse engineering process. Annotation types can be iteratively defined, refined and transformed, without requiring a fixed meta-model to be defined in advance. We show how our approach supports reverse engineering by implementing it in a tool called Metanool and by applying it to (i) analyzing architectural layering, (ii) tracking reengineering tasks, (iii) detecting design flaws, and (iv) analyzing features.

Enabling the evolution of J2EE applications through reverse engineering and quality assurance

Enterprise Applications are complex software systems that manipulate much persistent data and interact with the user through a vast and complex user interface. In particular applications written for the Java 2 Platform, Enterprise Edition (J2EE) are composed using various technologies such as Enterprise Java Beans (EJB) or Java Server Pages (JSP) that in turn rely on languages other than Java, such as XML or SQL. In this heterogeneous context applying existing reverse engineering and quality assurance techniques developed for object-oriented systems is not enough. Because those techniques have been created to measure quality or provide information about one aspect of J2EE applications, they cannot properly measure the quality of the entire system. We intend to devise techniques and metrics to measure quality in J2EE applications considering all their aspects and to aid their evolution. Using software visualization we also intend to inspect to structure of J2EE applications and all other aspects that can be investigate through this technique. In order to do that we also need to create a unified meta-model including all elements composing a J2EE application.