The potential for bias in reporting of industry-sponsored clinical trials

Concerns about potentially misleading reporting of pharmaceutical industry research have surfaced many times. The potential for duality (and thereby conflict) of interest is only too clear when you consider the sums of money required for the discovery, development and commercialization of new medicines. As the ability of major, mid-size and small pharmaceutical companies to innovate has waned, as evidenced by the seemingly relentless decline in the numbers of new medicines approved by Food and Drug Administration and European Medicines Agency year-on-year, not only has the cost per new approved medicine risen: so too has the public and media concern about the extent to which the pharmaceutical industry is open and honest about the efficacy, safety and quality of the drugs we manufacture and sell. In 2005 an Editorial in Journal of the American Medical Association made clear that, so great was their concern about misleading reporting of industry-sponsored studies, henceforth no article would be published that was not also guaranteed by independent statistical analysis. We examine the precursors to this Editorial, as well as its immediate and lasting effects for statisticians, for the manner in which statistical analysis is carried out, and for the industry more generally.

Quality of Reporting of Observational Longitudinal Research

Observational longitudinal research is particularly useful for assessing etiology and prognosis and for providing evidence for clinical decision making. However, there are no structured reporting requirements for studies of this design to assist authors, editors, and readers. The authors developed and tested a checklist of criteria related to threats to the internal and external validity of observational longitudinal studies. The checklist criteria concerned recruitment, data collection, biases, and data analysis and descriptive issues relevant to study rationale, study population, and generalizability. Two raters independently assessed 49 randomly selected articles describing stroke research published from 1999 to 2003 in six journals: American Journal of Epidemiology, Journal of Epidemiology and Community Health, Stroke, Annals of Neurology, Archives of Physical Medicine and Rehabilitation, and American Journal of Physical Medicine and Rehabilitation. On average, 17 of the 33 checklist criteria were reported. Criteria describing the study design were better reported than those related to internal validity. No relation was found between study type (etiologic or prognostic) or word count and quality of reporting. A flow diagram for summarizing participant flow through a study was developed. Editors and authors should consider using a checklist and flow diagram when reporting on observational longitudinal research.

Mothers' mental illness and child behavior problems: Cause-effect association or observation bias

Objective: A number, of studies have consistently found that a mother's mental health (particularly her level of depression) is a strong predictor of mental health problems experienced by her child(ren). However, the validity of this finding is in doubt because the majority of these studies have relied on maternal reports as indicators of children's behavior. Method: This prospective, longitudinal study examines data an the mental health of the mother from prior to the birth of her child to when the child reaches 14 years of age. Child behavior is measured at 14 years of age using reports from mother and child. Mother and child responses are compared to provide an indication of the possible magnitude of maternal observation bias in the reporting of child behavior problems. Results: Anxious and/or depressed mothers tend to report more cases of child behavior problems than do their mentally healthy counterparts or children themselves. Differences between mothers and youths In reporting behavior problems appear to be related to the mothers' mental health. Conclusions: Current maternal mental health impairment appears to have a substantial effect on the reporting of child behavior problems by the mother, thereby raising questions about the validity of reports of child behavior by persons who are currently emotionally distressed.

Evaluation of Bias in the Hamburg Wheel Tracking Device, RB00-010, 2013

As the list of states adopting the HWTD continues to grow, there is a need to evaluate how results are utilized. AASHTO T 324 does not standardize the analysis and reporting of test results. Furthermore, processing and reporting of the results among manufacturers is not uniform. This is partly due to the variation among agency reporting requirements. Some include only the midpoint rut depth, while others include the average across the entire length of the wheel track. To eliminate bias in reporting, statistical analysis was performed on over 150 test runs on gyratory specimens. Measurement location was found to be a source of significant variation in the HWTD. This is likely due to the nonuniform wheel speed across the specimen, geometry of the specimen, and air void profile. Eliminating this source of bias when reporting results is feasible though is dependent upon the average rut depth at the final pass. When reporting rut depth at the final pass, it is suggested for poor performing samples to average measurement locations near the interface of the adjoining gyratory specimens. This is necessary due to the wheel lipping on the mold. For all other samples it is reasonable to only eliminate the 3 locations furthest from the gear house. For multi‐wheel units, wheel side was also found to be significant for poor and good performing samples. After eliminating the suggested measurements from the analysis, the wheel was no longer a significant source of variation.

Publication bias in animal research: a systematic review protocol.

BACKGROUND: Systematic reviews and meta-analyses of pre-clinical studies, in vivo animal experiments in particular, can influence clinical care. Publication bias is one of the major threats of validity in systematic reviews and meta-analyses. Previous empirical studies suggested that systematic reviews and meta-analyses have become more prevalent until 2010 and found evidence for compromised methodological rigor with a trend towards improvement. We aim to comprehensively summarize and update the evidence base on systematic reviews and meta-analyses of animal studies, their methodological quality and assessment of publication bias in particular. METHODS/DESIGN: The objectives of this systematic review are as follows: âeuro¢To investigate the epidemiology of published systematic reviews of animal studies until present. âeuro¢To examine methodological features of systematic reviews and meta-analyses of animal studies with special attention to the assessment of publication bias. âeuro¢To investigate the influence of systematic reviews of animal studies on clinical research by examining citations of the systematic reviews by clinical studies. Eligible studies for this systematic review constitute systematic reviews and meta-analyses that summarize in vivo animal experiments with the purpose of reviewing animal evidence to inform human health. We will exclude genome-wide association studies and animal experiments with the main purpose to learn more about fundamental biology, physical functioning or behavior. In addition to the inclusion of systematic reviews and meta-analyses identified by other empirical studies, we will systematically search Ovid Medline, Embase, ToxNet, and ScienceDirect from 2009 to January 2013 for further eligible studies without language restrictions. Two reviewers working independently will assess titles, abstracts, and full texts for eligibility and extract relevant data from included studies. Data reporting will involve a descriptive summary of meta-analyses and systematic reviews. DISCUSSION: Results are expected to be publicly available later in 2013 and may form the basis for recommendations to improve the quality of systematic reviews and meta-analyses of animal studies and their use with respect to clinical care.

Detecting, quantifying and adjusting for publication bias in meta-analyses: protocol of a systematic review on methods.

BACKGROUND: Health professionals and policymakers aspire to make healthcare decisions based on the entire relevant research evidence. This, however, can rarely be achieved because a considerable amount of research findings are not published, especially in case of 'negative' results - a phenomenon widely recognized as publication bias. Different methods of detecting, quantifying and adjusting for publication bias in meta-analyses have been described in the literature, such as graphical approaches and formal statistical tests to detect publication bias, and statistical approaches to modify effect sizes to adjust a pooled estimate when the presence of publication bias is suspected. An up-to-date systematic review of the existing methods is lacking. METHODS/DESIGN: The objectives of this systematic review are as follows:âeuro¢ To systematically review methodological articles which focus on non-publication of studies and to describe methods of detecting and/or quantifying and/or adjusting for publication bias in meta-analyses.âeuro¢ To appraise strengths and weaknesses of methods, the resources they require, and the conditions under which the method could be used, based on findings of included studies.We will systematically search Web of Science, Medline, and the Cochrane Library for methodological articles that describe at least one method of detecting and/or quantifying and/or adjusting for publication bias in meta-analyses. A dedicated data extraction form is developed and pilot-tested. Working in teams of two, we will independently extract relevant information from each eligible article. As this will be a qualitative systematic review, data reporting will involve a descriptive summary. DISCUSSION: Results are expected to be publicly available in mid 2013. This systematic review together with the results of other systematic reviews of the OPEN project (To Overcome Failure to Publish Negative Findings) will serve as a basis for the development of future policies and guidelines regarding the assessment and handling of publication bias in meta-analyses.

Reporting of methodologic information on trial registries for quality assessment: a study of trial records retrieved from the WHO search portal

Background: Although randomized clinical trials (RCTs) are considered the gold standard of evidence, their reporting is often suboptimal. Trial registries have the potential to contribute important methodologic information for critical appraisal of study results. Methods and Findings: The objective of the study was to evaluate the reporting of key methodologic study characteristics in trial registries. We identified a random sample (n = 265) of actively recruiting RCTs using the World Health Organization International Clinical Trials Registry Platform (ICTRP) search portal in 2008. We assessed the reporting of relevant domains from the Cochrane Collaboration’s ‘Risk of bias’ tool and other key methodological aspects. Our primary outcomes were the proportion of registry records with adequate reporting of random sequence generation, allocation concealment, blinding, and trial outcomes. Two reviewers independently assessed each record. Weighted overall proportions in the ICTRP search portal for adequate reporting of sequence generation, allocation concealment, blinding (including and excluding open label RCT) and primary outcomes were 5.7% (95% CI 3.0–8.4%), 1.4% (0–2.8%), 41% (35–47%), 8.4% (4.1–13%), and 66% (60–72%), respectively. The proportion of adequately reported RCTs was higher for registries that used specific methodological fields for describing methods of randomization and allocation concealment compared to registries that did not. Concerning other key methodological aspects, weighted overall proportions of RCTs with adequately reported items were as follows: eligibility criteria (81%), secondary outcomes (46%), harm (5%) follow-up duration (62%), description of the interventions (53%) and sample size calculation (1%). Conclusions: Trial registries currently contain limited methodologic information about registered RCTs. In order to permit adequate critical appraisal of trial results reported in journals and registries, trial registries should consider requesting details on key RCT methods to complement journal publications. Full protocols remain the most comprehensive source of methodologic information and should be made publicly available.

Medication incidents in primary care medicine: protocol of a study by the Swiss Federal Sentinel Reporting System.

BACKGROUND/RATIONALE: Patient safety is a major concern in healthcare systems worldwide. Although most safety research has been conducted in the inpatient setting, evidence indicates that medical errors and adverse events are a threat to patients in the primary care setting as well. Since information about the frequency and outcomes of safety incidents in primary care is required, the goals of this study are to describe the type, frequency, seasonal and regional distribution of medication incidents in primary care in Switzerland and to elucidate possible risk factors for medication incidents. Label="METHODS AND ANALYSIS" ="METHODS"/> <AbstractText STUDY DESIGN AND SETTING: We will conduct a prospective surveillance study to identify cases of medication incidents among primary care patients in Switzerland over the course of the year 2015. PARTICIPANTS: Patients undergoing drug treatment by 167 general practitioners or paediatricians reporting to the Swiss Federal Sentinel Reporting System. INCLUSION CRITERIA: Any erroneous event, as defined by the physician, related to the medication process and interfering with normal treatment course. EXCLUSION CRITERIA: Lack of treatment effect, adverse drug reactions or drug-drug or drug-disease interactions without detectable treatment error. PRIMARY OUTCOME: Medication incidents. RISK FACTORS: Age, gender, polymedication, morbidity, care dependency, hospitalisation. STATISTICAL ANALYSIS: Descriptive statistics to assess type, frequency, seasonal and regional distribution of medication incidents and logistic regression to assess their association with potential risk factors. Estimated sample size: 500 medication incidents. LIMITATIONS: We will take into account under-reporting and selective reporting among others as potential sources of bias or imprecision when interpreting the results. ETHICS AND DISSEMINATION: No formal request was necessary because of fully anonymised data. The results will be published in a peer-reviewed journal. TRIAL REGISTRATION NUMBER: NCT0229537.

Media bias and central bank response. Evidence from the nominal exchange rate behavior in Colombia

Since 1991 Colombia has had a market-determined Peso - US Dollar Nominal Exchange Rate (NER), after more than 20 years of controlled and multiple exchange rates. The behavior (revaluation / devaluation) of the NER is constantly reported in news, editorials and op-eds of major newspapers of the nation with particular attention to revaluation. The uneven reporting of revaluation episodes can be explained by the existence of an interest group particulary affected by revaluation, looking to increase awareness and sympathy for help from public institutions. Using the number of news and op-eds from a major Colombian newspaper, it is shown that there is an over-reporting of revaluation episodes in contrast to devaluation ones. Secondly, using text analysis upon the content of the news, it is also shown that the words devaluation and revaluation are far apart in the distribution of words within the news; and revaluation is highly correlated with words related to: public institutions, exporters and the need of assistance. Finally it is also shown that the probability of the central bank buying US dollars to lessen revaluation effects increases with the number of news; even though the central bank allegedly intervenes in the exchange rate market only to tame volatility or accumulate international reserves.

An investigation of the shortcomings of the CONSORT 2010 statement for the reporting of group sequential randomised controlled trials: a methodological systematic review

Background It can be argued that adaptive designs are underused in clinical research. We have explored concerns related to inadequate reporting of such trials, which may influence their uptake. Through a careful examination of the literature, we evaluated the standards of reporting of group sequential (GS) randomised controlled trials, one form of a confirmatory adaptive design. Methods We undertook a systematic review, by searching Ovid MEDLINE from the 1st January 2001 to 23rd September 2014, supplemented with trials from an audit study. We included parallel group, confirmatory, GS trials that were prospectively designed using a Frequentist approach. Eligible trials were examined for compliance in their reporting against the CONSORT 2010 checklist. In addition, as part of our evaluation, we developed a supplementary checklist to explicitly capture group sequential specific reporting aspects, and investigated how these are currently being reported. Results Of the 284 screened trials, 68(24%) were eligible. Most trials were published in “high impact” peer-reviewed journals. Examination of trials established that 46(68%) were stopped early, predominantly either for futility or efficacy. Suboptimal reporting compliance was found in general items relating to: access to full trials protocols; methods to generate randomisation list(s); details of randomisation concealment, and its implementation. Benchmarking against the supplementary checklist, GS aspects were largely inadequately reported. Only 3(7%) trials which stopped early reported use of statistical bias correction. Moreover, 52(76%) trials failed to disclose methods used to minimise the risk of operational bias, due to the knowledge or leakage of interim results. Occurrence of changes to trial methods and outcomes could not be determined in most trials, due to inaccessible protocols and amendments. Discussion and Conclusions There are issues with the reporting of GS trials, particularly those specific to the conduct of interim analyses. Suboptimal reporting of bias correction methods could potentially imply most GS trials stopping early are giving biased results of treatment effects. As a result, research consumers may question credibility of findings to change practice when trials are stopped early. These issues could be alleviated through a CONSORT extension. Assurance of scientific rigour through transparent adequate reporting is paramount to the credibility of findings from adaptive trials. Our systematic literature search was restricted to one database due to resource constraints.

Publication Bias in Five Dental Implant Journals: An Observation from 2005 to 2009

Purpose: This study evaluated possible publication bias and its related factors in implant-related research over time. Materials and Methods: Articles published in Clinical Implant Dentistry and Related Research, Clinical Oral Implants Research, Implant Dentistry, Journal of Oral Implantology, and The International Journal of Oral & Maxillofacial Implants between 2005 and 2009 were reviewed. Nonoriginal articles were excluded. For each article included, study outcome, extramural funding source, type of study, and geographic origin were recorded. Descriptive and analytic statistics (alpha = .05), including the chi-square test and logistic regression analysis, were performed where appropriate. Results: From a total of 2,085 articles, 1,503 met the inclusion criteria. of the articles analyzed, 1,226 (81.6%), 160 (10.6%), and 117 (7.8%) articles reported positive, negative, and neutral outcomes, respectively. In vitro studies, studies from Asia, and funded animal studies were more likely to report positive outcomes compared to others (P = .02, P < .0001, and P = .009, respectively). Industry-funded studies represented the lowest frequency of positive outcomes versus studies funded by other sources. Conclusions: There were a high number of implant-related studies reporting positive outcomes in the five selected journals. Some selected factors were associated with positive outcome bias. In general, funding was not associated with a positive outcome, except for animal studies. Industry-supported research did not show any association with the publication of positive outcomes. INT J ORAL MAXILLOFAC IMPLANTS 2011;26:1024-1032

Multivariable adjustments counteract spectrum and test review bias in accuracy studies

OBJECTIVES: The STAndards for Reporting studies of Diagnostic accuracy (STARD) for investigators and editors and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) for reviewers and readers offer guidelines for the quality and reporting of test accuracy studies. These guidelines address and propose some solutions to two major threats to validity: spectrum bias and test review bias. STUDY DESIGN AND SETTING: Using a clinical example, we demonstrate that these solutions fail and propose an alternative solution that concomitantly addresses both sources of bias. We also derive formulas that prove the generality of our arguments. RESULTS: A logical extension of our ideas is to extend STARD item 23 by adding a requirement for multivariable statistical adjustment using information collected in QUADAS items 1, 2, and 12 and STARD items 3-5, 11, 15, and 18. CONCLUSION: We recommend reporting not only variation of diagnostic accuracy across subgroups (STARD item 23) but also the effects of the multivariable adjustments on test performance. We also suggest that the QUADAS be supplemented by an item addressing the appropriateness of statistical methods, in particular whether multivariable adjustments have been included in the analysis.

Cancer, meta-analysis and reporting biases: the case of erythropoiesis-stimulating agents.

Reporting and publication bias is a well-known problem in meta-analysis and healthcare research. In 2002 we conducted a meta-analysis on the effects of erythropoiesis-stimulating agents (ESAs) on overall survival in cancer patients, which suggested some evidence for improved survival in patients receiving ESAs compared with controls. However, a meta-analysis of individual patient data conducted several years later showed the opposite of our first meta-analysis, that is, evidence for increased on-study mortality and reduced overall survival in cancer patients receiving ESAs. We aimed to determine whether the results of our first meta-analysis could have been affected by publication and reporting biases and, if so, whether timely access to clinical study reports and individual patient data could have prevented this. We conducted a hypothetical meta-analysis for overall survival including all studies and study data that could have been available in 2002, at the time when we conducted our first meta-analysis. Compared with our original meta-analysis, which suggested an overall survival benefit for cancer patients receiving ESAs [hazard ratio (HR) 0.81, 95% confidence interval (CI) 0.67‒0.99], our hypothetical meta-analysis based on the results of all studies conducted at the time of the first analysis did not show evidence for a beneficial effect of ESAs on overall survival (HR 0.97, 95% CI 0.83‒1.12). Thus we have to conclude that our first meta-analysis showed misleading overall survival benefits due to publication and reporting biases, which could have been prevented by timely access to clinical study reports and individual patient data. Unrestricted access to clinical study protocols including amendments, clinical study reports and individual patient data is needed to ensure timely detection of both beneficial and harmful effects of healthcare interventions.

Strengthening the Reporting of Molecular Epidemiology for Infectious Diseases (STROME-ID): an extension of the STROBE statement

Molecular data are now widely used in epidemiological studies to investigate the transmission, distribution, biology, and diversity of pathogens. Our objective was to establish recommendations to support good scientific reporting of molecular epidemiological studies to encourage authors to consider specific threats to valid inference. The statement Strengthening the Reporting of Molecular Epidemiology for Infectious Diseases (STROME-ID) builds upon the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) initiative. The STROME-ID statement was developed by a working group of epidemiologists, statisticians, bioinformaticians, virologists, and microbiologists with expertise in control of infection and communicable diseases. The statement focuses on issues relating to the reporting of epidemiological studies of infectious diseases using molecular data that were not addressed by STROBE. STROME-ID addresses terminology, measures of genetic diversity within pathogen populations, laboratory methods, sample collection, use of molecular markers, molecular clocks, timeframe, multiple-strain infections, non-independence of infectious-disease data, missing data, ascertainment bias, consistency between molecular and epidemiological data, and ethical considerations with respect to infectious-disease research. In total, 20 items were added to the 22 item STROBE checklist. When used, the STROME-ID recommendations should advance the quality and transparency of scientific reporting, with clear benefits for evidence reviews and health-policy decision making.

An Assessment of the Risk of Bias in Randomized Controlled Trial Reports Published in Prosthodontic and Implant Dentistry Journals

PURPOSE The objective of this study was to assess the risk of bias of randomized controlled trials (RCTs) published in prosthodontic and implant dentistry journals. MATERIALS AND METHODS The last 30 issues of 9 journals in the field of prosthodontic and implant dentistry (Clinical Implant Dentistry and Related Research, Clinical Oral Implants Research, Implant Dentistry, International Journal of Oral & Maxillofacial Implants, International Journal of Periodontics and Restorative Dentistry, International Journal of Prosthodontics, Journal of Dentistry, Journal of Oral Rehabilitation, and Journal of Prosthetic Dentistry) were hand-searched for RCTs. Risk of bias was assessed using the Cochrane Collaboration's risk of bias tool and analyzed descriptively. RESULTS From the 3,667 articles screened, a total of 147 RCTs were identified and included. The number of published RCTs increased with time. The overall distribution of a high risk of bias assessment varied across the domains of the Cochrane risk of bias tool: 8% for random sequence generation, 18% for allocation concealment, 41% for masking, 47% for blinding of outcome assessment, 7% for incomplete outcome data, 12% for selective reporting, and 41% for other biases. CONCLUSION The distribution of high risk of bias for RCTs published in the selected prosthodontic and implant dentistry journals varied among journals and ranged from 8% to 47%, which can be considered as substantial.