Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study

Objectives To examine the extent of multiplicity of data in trial reports and to assess the impact of multiplicity on meta-analysis results. Design Empirical study on a cohort of Cochrane systematic reviews. Data sources All Cochrane systematic reviews published from issue 3 in 2006 to issue 2 in 2007 that presented a result as a standardised mean difference (SMD). We retrieved trial reports contributing to the first SMD result in each review, and downloaded review protocols. We used these SMDs to identify a specific outcome for each meta-analysis from its protocol. Review methods Reviews were eligible if SMD results were based on two to ten randomised trials and if protocols described the outcome. We excluded reviews if they only presented results of subgroup analyses. Based on review protocols and index outcomes, two observers independently extracted the data necessary to calculate SMDs from the original trial reports for any intervention group, time point, or outcome measure compatible with the protocol. From the extracted data, we used Monte Carlo simulations to calculate all possible SMDs for every meta-analysis. Results We identified 19 eligible meta-analyses (including 83 trials). Published review protocols often lacked information about which data to choose. Twenty-four (29%) trials reported data for multiple intervention groups, 30 (36%) reported data for multiple time points, and 29 (35%) reported the index outcome measured on multiple scales. In 18 meta-analyses, we found multiplicity of data in at least one trial report; the median difference between the smallest and largest SMD results within a meta-analysis was 0.40 standard deviation units (range 0.04 to 0.91). Conclusions Multiplicity of data can affect the findings of systematic reviews and meta-analyses. To reduce the risk of bias, reviews and meta-analyses should comply with prespecified protocols that clearly identify time points, intervention groups, and scales of interest.

How valuable are multiple treatment comparison methods in evidence-based health-care evaluation?

Objectives To compare the use of pair-wise meta-analysis methods to multiple treatment comparison (MTC) methods for evidence-based health-care evaluation to estimate the effectiveness and cost-effectiveness of alternative health-care interventions based on the available evidence. Methods Pair-wise meta-analysis and more complex evidence syntheses, incorporating an MTC component, are applied to three examples: 1) clinical effectiveness of interventions for preventing strokes in people with atrial fibrillation; 2) clinical and cost-effectiveness of using drug-eluting stents in percutaneous coronary intervention in patients with coronary artery disease; and 3) clinical and cost-effectiveness of using neuraminidase inhibitors in the treatment of influenza. We compare the two synthesis approaches with respect to the assumptions made, empirical estimates produced, and conclusions drawn. Results The difference between point estimates of effectiveness produced by the pair-wise and MTC approaches was generally unpredictable—sometimes agreeing closely whereas in other instances differing considerably. In all three examples, the MTC approach allowed the inclusion of randomized controlled trial evidence ignored in the pair-wise meta-analysis approach. This generally increased the precision of the effectiveness estimates from the MTC model. Conclusions The MTC approach to synthesis allows the evidence base on clinical effectiveness to be treated as a coherent whole, include more data, and sometimes relax the assumptions made in the pair-wise approaches. However, MTC models are necessarily more complex than those developed for pair-wise meta-analysis and thus could be seen as less transparent. Therefore, it is important that model details and the assumptions made are carefully reported alongside the results.

Protective and risk factors in amateur equestrians and description of injury patterns: A retrospective data analysis and a case - control survey

Background In Switzerland there are about 150,000 equestrians. Horse related injuries, including head and spinal injuries, are frequently treated at our level I trauma centre. Objectives To analyse injury patterns, protective factors, and risk factors related to horse riding, and to define groups of safer riders and those at greater risk Methods We present a retrospective and a case-control survey at conducted a tertiary trauma centre in Bern, Switzerland. Injured equestrians from July 2000 - June 2006 were retrospectively classified by injury pattern and neurological symptoms. Injured equestrians from July-December 2008 were prospectively collected using a questionnaire with 17 variables. The same questionnaire was applied in non-injured controls. Multiple logistic regression was performed, and combined risk factors were calculated using inference trees. Results Retrospective survey A total of 528 injuries occured in 365 patients. The injury pattern revealed as follows: extremities (32%: upper 17%, lower 15%), head (24%), spine (14%), thorax (9%), face (9%), pelvis (7%) and abdomen (2%). Two injuries were fatal. One case resulted in quadriplegia, one in paraplegia. Case-control survey 61 patients and 102 controls (patients: 72% female, 28% male; controls: 63% female, 37% male) were included. Falls were most frequent (65%), followed by horse kicks (19%) and horse bites (2%). Variables statistically significant for the controls were: Older age (p = 0.015), male gender (p = 0.04) and holding a diploma in horse riding (p = 0.004). Inference trees revealed typical groups less and more likely to suffer injury. Conclusions Experience with riding and having passed a diploma in horse riding seem to be protective factors. Educational levels and injury risk should be graded within an educational level-injury risk index.

Does published orthodontic research account for clustering effects during statistical data analysis?

In orthodontics, multiple site observations within patients or multiple observations collected at consecutive time points are often encountered. Clustered designs require larger sample sizes compared to individual randomized trials and special statistical analyses that account for the fact that observations within clusters are correlated. It is the purpose of this study to assess to what degree clustering effects are considered during design and data analysis in the three major orthodontic journals. The contents of the most recent 24 issues of the American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), Angle Orthodontist (AO), and European Journal of Orthodontics (EJO) from December 2010 backwards were hand searched. Articles with clustering effects and whether the authors accounted for clustering effects were identified. Additionally, information was collected on: involvement of a statistician, single or multicenter study, number of authors in the publication, geographical area, and statistical significance. From the 1584 articles, after exclusions, 1062 were assessed for clustering effects from which 250 (23.5 per cent) were considered to have clustering effects in the design (kappa = 0.92, 95 per cent CI: 0.67-0.99 for inter rater agreement). From the studies with clustering effects only, 63 (25.20 per cent) had indicated accounting for clustering effects. There was evidence that the studies published in the AO have higher odds of accounting for clustering effects [AO versus AJODO: odds ratio (OR) = 2.17, 95 per cent confidence interval (CI): 1.06-4.43, P = 0.03; EJO versus AJODO: OR = 1.90, 95 per cent CI: 0.84-4.24, non-significant; and EJO versus AO: OR = 1.15, 95 per cent CI: 0.57-2.33, non-significant). The results of this study indicate that only about a quarter of the studies with clustering effects account for this in statistical data analysis.

Cathepsin D primes caspase-8 activation by multiple intra-chain proteolysis

During the resolution of inflammatory responses, neutrophils rapidly undergo apoptosis. A direct and fast activation of caspase-8 by cathepsin D was shown to be crucial in the initial steps of neutrophil apoptosis. Nevertheless, the activation mechanism of caspase-8 remains unclear. Here, by using site-specific mutants of caspase-8, we show that both cathepsin D-mediated proteolysis and homodimerization of caspase-8 are necessary to generate an active caspase-8. At acidic pH, cathepsin D specifically cleaved caspase-8 but not the initiator caspase-9 or -10 and significantly increased caspase-8 activity in dimerizing conditions. These events were completely abolished by pepstatin A, a pharmacological inhibitor of cathepsin D. The cathepsin D intra-chain proteolysis greatly stabilized the active site of caspase-8. Moreover, the main caspase-8 fragment generated by cathepsin D cleavage could be affinity-labeled with the active site probe biotin-VAD-fluoromethyl ketone, suggesting that this fragment is enzymatically active. Importantly, in an in vitro cell-free assay, the addition of recombinant human caspase-8 protein, pre-cleaved by cathepsin D, was followed by caspase-3 activation. Our data therefore indicate that cathepsin D is able to initiate the caspase cascade by direct activation of caspase-8. As cathepsin D is ubiquitously expressed, this may represent a general mechanism to induce apoptosis in a variety of immune and nonimmune cells.

Multiple small "imaging" branch-duct type intraductal papillary mucinous neoplasms (IPMNs) in familial pancreatic cancer: indicator for concomitant high grade pancreatic intraepithelial neoplasia?

Most screening programs for familial pancreatic cancer are currently based on endoscopic ultrasonography and/or magnetic resonance imaging (MRI). Cystic lesions, especially those suspicious for small intraductal pancreatic mucinous neoplasms (IPMNs) of the branch ducts, can be visualized in up to 40 % of individuals at risk, but their pathological importance in the setting of FPC is yet not well established. Individuals at risk from a prospective screening program for familial pancreatic cancer with small "imaging" IPMNs of the branch-duct type (BD-IPMN) who underwent pancreatic resection were analysed regarding clinico-pathological data and the locations of pancreatic lesions. Five of 125 individuals at risk who underwent screening had multiple small (size 2-10 mm) unicystic lesions and/or multicystic single lesions in the pancreatic body and tail suspicious for BD-IPMNs upon MRI imaging and decided to undergo surgical resection after interdisciplinary counselling, although none fulfilled the consensus criteria for IPMN resection. Histological examination revealed BD-IPMNs with low or moderate dysplasia of the gastric type in combination with multifocal PanIN2 and PanIN3 lesions in 4 individuals. The remaining patient had only tiny ductectasias in the pancreatic tail with multifocal PanIN 2 lesions in the entire gland and one PanIN3 lesion in the pancreatic head. Intriguingly, the location of the most dysplastic histological lesions (PanIN3) did not correspond to the preoperatively detected lesions and were not visible in preoperative imaging. In the setting of FPC, the presence of multiple small "imaging" BD-IPMNs may indicate the presence of high-grade PanIN lesions elsewhere in the pancreas.

Response Bias Using Two-Stage Data Collection: A study of elderly participants in a program.

Researchers examining the effects of programs, in this case a state-level pharmaceutical assistance program for the elderly, sometimes must rely on multiple methods of data collection. Two-stage data collection (e.g., a telephone interview followed by a mail questionnaire) was used to obtain a full range of information. Older age groups were found to participate less frequently in the telephone interview, while certain demographic factors characterized mail questionnaire nonparticipants, all of which supports past research. Results also show that those in the poorest health are less likely to participate in the mail survey. Combining the two methods did not result in high attrition, suggesting that innovation can be successfully employed. Knowledge of the bias associated with each method will aid in targeting special groups.

P-Technique Factor Analyses of the Multiple Affect Adjective Checklist (MAACL)

The Multiple Affect Adjective Check List (MAACL) has been found to have five first-order factors representing Anxiety, Depression, Hostility, Positive Affect, and Sensation Seeking and two second-order factors representing Positive Affect and Sensation Seeking (PASS) and Dysphoria. The present study examines whether these first- and second-order conceptions of affect (based on R-technique factor analysis) can also account for patterns of intraindividual variability in affect (based on P-technique factor analysis) in eight elderly women. Although the hypothesized five-factor model of affect was not testable in all of the present P-technique datasets, the results were consistent with this interindividual model of affect. Moreover, evidence of second-order (PASS and Dysphoria) and third-order (generalized distress) factors was found in one data set. Sufficient convergence in findings between the present P-technique research and prior R-technique research suggests that the MAACL is robust in describing both inter- and intraindividual components of affect in elderly women.

Spinal Cord Diffusion-Tensor Imaging and Motor-evoked Potentials in Multiple Sclerosis Patients: Microstructural and Functional Asymmetry

Purpose: To assess possible association between intrinsic structural damage and clinical disability by correlating spinal cord diffusion-tensor (DT) imaging data with electrophysiological parameters in patients with a diagnosis of multiple sclerosis (MS). Materials and Methods: This study was approved by the local ethical committee according to the declaration of Helsinki and written informed consent was obtained. DT images and T1- and T2-weighted images of the spinal cord were acquired in 28 healthy volunteers and 41 MS patients. Fractional anisotropy (FA) and apparent diffusion coefficients were evaluated in normal-appearing white matter (NAWM) at the cervical level and were correlated with motor-evoked potentials (n = 34). Asymmetry index was calculated for FA values with corresponding left and right regions of interest as percentage of the absolute difference between these values relative to the sum of the respective FA values. Statistical analysis included Spearman rank correlations, Mann-Whitney test, and reliability analysis. Results: Healthy volunteers had low asymmetry index (1.5%-2.2%). In MS patients, structural abnormalities were reflected by asymmetric decrease of FA (asymmetry index: 3.6%; P = .15). Frequently asymmetrically affected among MS patients was left and right central motor conduction time (CMCT) to abductor digiti minimi muscle (ADMM) (asymmetry index, 15%-16%) and tibialis anterior muscle (TAM) (asymmetry index, 9.5%-14.1%). Statistically significant correlations of functional (ie, electrophysiological) and structural (ie, DT imaging) asymmetries were found (P = .005 for CMCT to ADMM; P = .007 for CMCT to TAM) for the cervical lateral funiculi, which comprise the crossed pyramidal tract. Interobserver reliability for DT imaging measurements was excellent (78%-87%). Conclusion: DT imaging revealed asymmetric anatomic changes in spinal cord NAWM, which corresponded to asymmetric electrophysiological deficits for both arms and legs, and reflected a specific structure-function relationship in the human spinal cord. © RSNA, 2013.

Accuracy of magnetic resonance imaging for the diagnosis of multiple sclerosis: systematic review

OBJECTIVE: To determine the accuracy of magnetic resonance imaging criteria for the early diagnosis of multiple sclerosis in patients with suspected disease. DESIGN: Systematic review. DATA SOURCES: 12 electronic databases, citation searches, and reference lists of included studies. Review methods Studies on accuracy of diagnosis that compared magnetic resonance imaging, or diagnostic criteria incorporating such imaging, to a reference standard for the diagnosis of multiple sclerosis. RESULTS: 29 studies (18 cohort studies, 11 other designs) were included. On average, studies of other designs (mainly diagnostic case-control studies) produced higher estimated diagnostic odds ratios than did cohort studies. Among 15 studies of higher methodological quality (cohort design, clinical follow-up as reference standard), those with longer follow-up produced higher estimates of specificity and lower estimates of sensitivity. Only two such studies followed patients for more than 10 years. Even in the presence of many lesions (> 10 or > 8), magnetic resonance imaging could not accurately rule multiple sclerosis in (likelihood ratio of a positive test result 3.0 and 2.0, respectively). Similarly, the absence of lesions was of limited utility in ruling out a diagnosis of multiple sclerosis (likelihood ratio of a negative test result 0.1 and 0.5). CONCLUSIONS: Many evaluations of the accuracy of magnetic resonance imaging for the early detection of multiple sclerosis have produced inflated estimates of test performance owing to methodological weaknesses. Use of magnetic resonance imaging to confirm multiple sclerosis on the basis of a single attack of neurological dysfunction may lead to over-diagnosis and over-treatment.

An introduction to low-level analysis methods of DNA microarray data

This article gives an overview over the methods used in the low--level analysis of gene expression data generated using DNA microarrays. This type of experiment allows to determine relative levels of nucleic acid abundance in a set of tissues or cell populations for thousands of transcripts or loci simultaneously. Careful statistical design and analysis are essential to improve the efficiency and reliability of microarray experiments throughout the data acquisition and analysis process. This includes the design of probes, the experimental design, the image analysis of microarray scanned images, the normalization of fluorescence intensities, the assessment of the quality of microarray data and incorporation of quality information in subsequent analyses, the combination of information across arrays and across sets of experiments, the discovery and recognition of patterns in expression at the single gene and multiple gene levels, and the assessment of significance of these findings, considering the fact that there is a lot of noise and thus random features in the data. For all of these components, access to a flexible and efficient statistical computing environment is an essential aspect.

Multiple B-cell clones producing antibodies directed to the spacer and disintegrin/thrombospondin type-1 repeat 1 (TSP1) of ADAMTS13 in a patient with acquired thrombotic thrombocytopenic purpura

BACKGROUND: The cysteine-rich/spacer domains of ADAMTS13 contain a major binding site for antibodies in patients with acquired thrombotic thrombocytopenic purpura (TTP). OBJECTIVE: To study the heterogeneity of the antibody response towards these domains an immunoglobulin V-gene phage-display library was constructed to isolate monoclonal anti-ADAMTS13 antibodies from the immunoglobulin repertoire of a patient with acquired TTP. METHODS: Combined variable heavy chain (VH) and variable light chain (VL) segments, expressed as single-chain Fv fragments (scFv), were selected for binding to an ADAMTS13 fragment consisting of the disintegrin/thrombospondin type-1 repeat 1 (TSP1)/cysteine-rich/spacer domains. RESULTS: Seven different scFv antibody clones were identified that were assigned to four groups based on their homology to VH germline gene segments. Epitope-mapping revealed that scFv I-9 (VH1-69), I-26 (VH1-02), and I-41 (VH3-09) bind to an overlapping binding site in the ADAMTS13 spacer domain, whereas scFv I-16 (VH3-07) binds to the disintegrin/TSP1 domains. The affinity of scFv for the disintegrin/TSP1/cysteine-rich/spacer domain was determined by surface plasmon resonance analysis and the dissociation constants ranged from 3 to 254 nM. The scFv partially inhibited ADAMTS13 activity. However, full-length IgG prepared from the variable domains of scFv I-9 inhibited ADAMTS13 activity more profoundly. Plasma of six patients with acquired TTP competed for binding of scFv I-9 to ADAMTS13. CONCLUSION: Our data indicate that multiple B-cell clones producing antibodies directed against the spacer domain are present in the patient analyzed in this study. Our findings also suggest that antibodies with a similar epitope specificity as scFv I-9 are present in plasma of other patients with acquired TTP.

A graph theoretic approach to testing associations between disparate sources of functional genomic data

The last few years have seen the advent of high-throughput technologies to analyze various properties of the transcriptome and proteome of several organisms. The congruency of these different data sources, or lack thereof, can shed light on the mechanisms that govern cellular function. A central challenge for bioinformatics research is to develop a unified framework for combining the multiple sources of functional genomics information and testing associations between them, thus obtaining a robust and integrated view of the underlying biology. We present a graph theoretic approach to test the significance of the association between multiple disparate sources of functional genomics data by proposing two statistical tests, namely edge permutation and node label permutation tests. We demonstrate the use of the proposed tests by finding significant association between a Gene Ontology-derived "predictome" and data obtained from mRNA expression and phenotypic experiments for Saccharomyces cerevisiae. Moreover, we employ the graph theoretic framework to recast a surprising discrepancy presented in Giaever et al. (2002) between gene expression and knockout phenotype, using expression data from a different set of experiments.

Multiple Testing With an Empirical Alternative Hypothesis

An optimal multiple testing procedure is identified for linear hypotheses under the general linear model, maximizing the expected number of false null hypotheses rejected at any significance level. The optimal procedure depends on the unknown data-generating distribution, but can be consistently estimated. Drawing information together across many hypotheses, the estimated optimal procedure provides an empirical alternative hypothesis by adapting to underlying patterns of departure from the null. Proposed multiple testing procedures based on the empirical alternative are evaluated through simulations and an application to gene expression microarray data. Compared to a standard multiple testing procedure, it is not unusual for use of an empirical alternative hypothesis to increase by 50% or more the number of true positives identified at a given significance level.

A Method for Visualizing Multivariate Time Series Data

Visualization and exploratory analysis is an important part of any data analysis and is made more challenging when the data are voluminous and high-dimensional. One such example is environmental monitoring data, which are often collected over time and at multiple locations, resulting in a geographically indexed multivariate time series. Financial data, although not necessarily containing a geographic component, present another source of high-volume multivariate time series data. We present the mvtsplot function which provides a method for visualizing multivariate time series data. We outline the basic design concepts and provide some examples of its usage by applying it to a database of ambient air pollution measurements in the United States and to a hypothetical portfolio of stocks.