ASSESSING THE UNRELIABILITY OF THE MEDICAL LITERATURE: A RESPONSE TO "WHY MOST PUBLISHED RESEARCH FINDINGS ARE FALSE"

A recent article in this journal (Ioannidis JP (2005) Why most published research findings are false. PLoS Med 2: e124) argued that more than half of published research findings in the medical literature are false. In this commentary, we examine the structure of that argument, and show that it has three basic components: 1)An assumption that the prior probability of most hypotheses explored in medical research is below 50%. 2)Dichotomization of P-values at the 0.05 level and introduction of a “bias” factor (produced by significance-seeking), the combination of which severely weakens the evidence provided by every design. 3)Use of Bayes theorem to show that, in the face of weak evidence, hypotheses with low prior probabilities cannot have posterior probabilities over 50%. Thus, the claim is based on a priori assumptions that most tested hypotheses are likely to be false, and then the inferential model used makes it impossible for evidence from any study to overcome this handicap. We focus largely on step (2), explaining how the combination of dichotomization and “bias” dilutes experimental evidence, and showing how this dilution leads inevitably to the stated conclusion. We also demonstrate a fallacy in another important component of the argument –that papers in “hot” fields are more likely to produce false findings. We agree with the paper’s conclusions and recommendations that many medical research findings are less definitive than readers suspect, that P-values are widely misinterpreted, that bias of various forms is widespread, that multiple approaches are needed to prevent the literature from being systematically biased and the need for more data on the prevalence of false claims. But calculating the unreliability of the medical research literature, in whole or in part, requires more empirical evidence and different inferential models than were used. The claim that “most research findings are false for most research designs and for most fields” must be considered as yet unproven.

Incidence and outcome of chronic graft-versus-host disease using National Institutes of Health consensus criteria

Chronic graft-versus-host disease (cGVHD), a common complication after stem cell transplant (SCT), has an impact on morbidity and survival. Previous classification of cGVHD has not been reproducible or prognostic for nonrelapse mortality (NRM). Recently the National Institutes of Health (NIH) consensus criteria were proposed, but the ability of this classification to predict outcome of various subtypes of cGVHD is unknown. Patients (N = 110) undergoing an SCT for a hematologic malignancy and surviving until day 100 posttransplant from 2001 to 2003 were studied. The overall survival (OS) using a landmark analysis at day 100 was 44% versus 66% (no GVHD vs. GVHD, P = .026). The OS of patients with various types of GVHD as proposed by the NIH criteria were significantly different (P < .0001). In a univariate analyses, this was more apparent when patients with any acute features of GVHD were compared to classic cGVHD (3-year OS 46% vs. 68%, P = .033). The 3-year NRM for the entire cohort was 21%, and was not affected by presence or absence of GVHD or subtypes of GVHD. In a multivariable analysis, extensive cGVHD (hazard ratio [HR] 0.35, P = .015) and having any acute feature of GVHD after day 100 (HR 3.36, P = .0144) were significant independent predictors of survival. The OS with different NIH subtypes of GVHD after day 100 from SCT varies, and is superior for patients with classic cGVHD.

High-dose chemotherapy followed by autologous stem cell transplantation for relapsed or refractory Hodgkin lymphoma: prognostic features and outcomes

Between January 1990 and April 2001, 115 patients received high-dose chemotherapy (HDT) followed by autologous stem cell transplantation (ASCT) for relapsed or refractory Hodgkin lymphoma (HL). With a median follow-up of 58 months (range, 1 - 175 months), 5-year progression-free survival (PFS) and overall survival (OS) were 46% and 58%, respectively. Twelve patients with primary refractory disease had a 5-year PFS of 41% and OS of 58%, not significantly different from those of the remaining cohort. Early and overall regimen related mortality were 7% and 16%, respectively. Male gender (P = 0.04) and a time to relapse (TTR) < 12 months (P = 0.03) were associated with decreased OS by univariate analysis. In multivariate analysis, TTR < 12 months remained statistically significant (P = 0.04). We have confirmed that HDT and ASCT result in long-term survival for a proportion of patients with relapsed or refractory HL. All patients, including those with primary refractory disease, benefited from HDT and ASCT.

Inhibition of integrin alphavbeta6 on cholangiocytes blocks transforming growth factor-beta activation and retards biliary fibrosis progression

BACKGROUND ; AIMS: Integrin alphavbeta6 is highly expressed on certain activated epithelia, where it mediates attachment to fibronectin and serves as coreceptor for the activation of latent transforming growth factor (TGF)-beta1. Because its role in liver fibrosis is unknown, we studied alphavbeta6 function in vitro and explored the antifibrotic potential of the specific alphavbeta6 antagonist EMD527040. METHODS: Experimental liver fibrosis was studied in rats after bile duct ligation (BDL) and in Mdr2(abcb4)(-/-) mice. Different doses of EMD527040 were given to rats from week 2 to 6 after BDL and to Mdr2(-/-) mice from week 4 to 8. Liver collagen was quantified, and expression of alphavbeta6 and fibrosis-related transcripts was determined by quantitative reverse-transcription polymerase chain reaction. alphavbeta6-expressing cells, bile duct proliferation, and apoptosis were assessed histologically. The effect of EMD527040 on cholangiocyte adhesion, proliferation, apoptosis, and TGF-beta1 activation was studied in vitro. RESULTS: alphavbeta6 was highly expressed on proliferating bile duct epithelia in fibrosis, with 100-fold increased transcript levels in advanced fibrosis. EMD527040 attenuated bile ductular proliferation and peribiliary collagen deposition by 40%-50%, induced down-regulation of fibrogenic and up-regulation of fibrolytic genes, and improved liver architecture and function. In vitro alphavbeta6 inhibition reduced activated cholangiocyte proliferation, their adhesion to fibronectin, and endogenous activation of TGF-beta1 by 50% but did not affect bile duct apoptosis. CONCLUSIONS: Integrin alphavbeta6 is strongly up-regulated in proliferating bile duct epithelia and drives fibrogenesis via adhesion to fibronectin and auto/paracrine TGF-beta1 activation. Pharmacologic inhibition of alphavbeta6 potently inhibits the progression of primary and secondary biliary fibrosis.

A Preliminary Investigation of the Hydrometallurgy of Philipsburg Manganese

In the past the steel industry of the United States has depended almost wholly on imports for its supplies of manganese. Although it is well known that there are enormous deposits of low grade manganese ore in the United States the production of substantial amounts of ferro-grade mater­ial from domestic sources presents a field for con­structive and practical research.

Pharmacological inhibition of integrin alphavbeta3 aggravates experimental liver fibrosis and suppresses hepatic angiogenesis

The vitronectin receptor integrin alphavbeta3 promotes angiogenesis by mediating migration and proliferation of endothelial cells, but also drives fibrogenic activation of hepatic stellate cells (HSCs) in vitro. Expecting antifibrotic synergism, we studied the effect of alphavbeta3 inhibition in two in vivo models of liver fibrogenesis. Liver fibrosis was induced in rats by way of bile duct ligation (BDL) for 6 weeks or thioacetamide (TAA) injections for 12 weeks. A specific alphavbeta3 (alphavbeta5) inhibitor (Cilengitide) was given intraperitoneally twice daily at 15 mg/kg during BDL or after TAA administration. Liver collagen was determined as hydroxyproline, and gene expression was quantified by way of quantitative polymerase chain reaction. Liver angiogenesis, macrophage infiltration, and hypoxia were assessed by way of CD31, CD68 and hypoxia-inducible factor-1alpha immunostaining. Cilengitide decreased overall vessel formation. This was significant in portal areas of BDL and septal areas of TAA fibrotic rats and was associated with a significant increase of liver collagen by 31% (BDL) and 27% (TAA), and up-regulation of profibrogenic genes and matrix metalloproteinase-13. Treatment increased gamma glutamyl transpeptidase in both models, while other serum markers remained unchanged. alphavbeta3 inhibition resulted in mild liver hypoxia, as evidenced by up-regulation of hypoxia-inducible genes. Liver infiltration by macrophages/Kupffer cells was not affected, although increases in tumor necrosis factor alpha, interleukin-18, and cyclooxygenase-2 messenger RNA indicated modest macrophage activation. CONCLUSION: Specific inhibition of integrin alphavbeta3 (alphavbeta5) in vivo decreased angiogenesis but worsened biliary (BDL) and septal (TAA) fibrosis, despite its antifibrogenic effect on HSCs in vitro. Angiogenesis inhibitors should be used with caution in patients with hepatic fibrosis.

Classic and overlap chronic graft-versus-host disease (cGVHD) is associated with superior outcome after extracorporeal photopheresis (ECP)

The National Institutes of Health (NIH) classification of graft-versus-host disease (GVHD) is a significant improvement over prior classifications, and has prognostic implications. We hypothesized that the NIH classification of GVHD would predict the survival of patients with GVHD treated with extracorporeal photopheresis (ECP). Sixty-four patients with steroid refractory/dependent GVHD treated with ECP were studied. The 3-year overall survival (OS) was 36% (95% confidence interval [CI] 13-59). Progressive GVHD was seen in 39% of patients with any acute GVHD (aGVHD) (classic acute, recurrent acute, overlap) compared to 3% of patients with classic chronic GVHD (cGVHD) (P=.002). OS was superior for patients with classic cGVHD (median survival, not reached) compared to overlap GVHD (median survival, 395 days, 95% CI 101 to not reached) and aGVHD (delayed, recurrent or persistent) (median survival, 72 days, 95% CI 39-152). In univariate analyses, significant predictors of survival after ECP included GVHD subtype, bilirubin, platelet count, and steroid dose. In multivariate analyses overlap plus classic cGVHD was an independent prognostic feature predictive of superior survival (hazard ratio [HR] 0.34, 95% CI 0.14-0.8, p=.014). This study suggests that NIH classification can predict outcome after ECP for steroid refractory/dependent GVHD.

Correlating the Ancient Maya and Modern European Calendars with High-Precision AMS 14C Dating

The reasons for the development and collapse of Maya civilization remain controversial and historical events carved on stone monuments throughout this region provide a remarkable source of data about the rise and fall of these complex polities. Use of these records depends on correlating the Maya and European calendars so that they can be compared with climate and environmental datasets. Correlation constants can vary up to 1000 years and remain controversial. We report a series of high-resolution AMS C-14 dates on a wooden lintel collected from the Classic Period city of Tikal bearing Maya calendar dates. The radiocarbon dates were calibrated using a Bayesian statistical model and indicate that the dates were carved on the lintel between AD 658-696. This strongly supports the Goodman-Martinez-Thompson (GMT) correlation and the hypothesis that climate change played an important role in the development and demise of this complex civilization.

CD8+ Pagetoid Reticulosis Presenting as a Solitary Foot Plaque in a Young Woman.

Background: Pagetoid reticulosis is a rare variant of mycosis fungoides. This rare condition typically presents as a solitary plaque located on the extremities with an indolent clinical course (Woringer-Kolopp disease) or as a more generalized presentation with diffuse cutaneous involvement and a more aggressive clinical course (Ketron-Goodman disease). Purpose: To review the cutaneous manifestations, pathology, and treatment of localized pagetoid reticulosis. Methods: The authors describe a 24-year-old woman with a slowly enlarging, localized plaque of seven months duration, representing the localized form of pagetoid reticulosis with CD8+ immunophenotype. Results: The histological, immunohistochemical, and clinical features of the patient's skin lesion were characteristic for a diagnosis of Woringer-Kolopp disease. Systemic work-up for lymphoma was negative. Conclusion: Woringer-Kolopp disease is most commonly seen in middle-aged men as a solitary lesion of the extremities, and it should always be considered in the differential diagnosis when a patient presents with such a lesion. A histological analysis demonstrated atypical lymphocytes preferentially localized to the epidermis with a CD4+, CD8+, or CD4-/CD8- phenotype. The treatment of choice for a solitary lesion may be localized radiation therapy, but newer therapies, such as bexarotene, may warrant further investigation.

Multiple GABA receptors in rabbit retina

The task of encoding and processing complex sensory input requires many types of transsynaptic signals. This requirement is served in part by an extensive group of neurotransmitter substances which may include thirty or more different compounds. At the next level of information processing, the existence of multiple receptors for a given neurotransmitter appears to be a widely used mechanism to generate multiple responses to a given first messenger (Snyder and Goodman, 1980). Despite the wealth of published data on GABA receptors, the existence of more than one GABA receptor was in doubt until the mid 1980's. Presently there is still disagreement on the number of types of GABA receptors, estimates for which range from two to four (DeFeudis, 1983; Johnston, 1985). Part of the problem in evaluating data concerning multiple receptor types is the lack of information on the number of gene products and their subsequent supramolecular organization in different neurons. In order to evaluate the question concerning the diversity of GABA receptors in the nervous system, we must rely on indirect information derived from a wide variety of experimental techniques. These include pharmacological binding studies to membrane fractions, electrophysiological studies, localization studies, purification studies, and functional assays. Almost all parts of the central and peripheral nervous system use GABA as a neurotransmitter, and these experimental techniques have therefore been applied to many different parts of the nervous system for the analysis of GABA receptor characteristics. We are left with a large amount of data from a wide variety of techniques derived from many parts of the nervous system. When this project was initiated in 1983, there were only a handful of pharmacological tools to assess the question of multiple GABA receptors. The approach adopted was to focus on a single model system, using a variety of experimental techniques, in order to evaluate the existence of multiple forms of GABA receptors. Using the in vitro rabbit retina, a combination of pharmacological binding studies, functional release studies and partial purification studies were undertaken to examine the GABA receptor composition of this tissue. Three types of GABA receptors were observed: Al receptors coupled to benzodiazepine and barbiturate modulation, and A2 or uncoupled GABA-A receptors, and GABA-B receptors. These results are evaluated and discussed in light of recent findings by others concerning the number and subtypes of GABA receptors in the nervous system. ^

Managing the risks of medical technology: Do hazard surveillance systems provide an early warning of medical device problems?

The Food and Drug Administration (FDA) is responsible for risk assessment and risk management in the post-market surveillance of the U.S. medical device industry. One of the FDA regulatory mechanisms, the Medical Device Reporting System (MDR) is an adverse event reporting system intended to provide the FDA with advance warning of device problems. It includes voluntary reporting for individuals, and mandatory reporting for device manufacturers. ^ In a study of alleged breast implant safety problems, this research examines the organizational processes by which the FDA gathers data on adverse events and uses adverse event reporting systems to assess and manage risk. The research reviews the literature on problem recognition, risk perception, and organizational learning to understand the influence highly publicized events may have on adverse event reporting. Understanding the influence of an environmental factor, such as publicity, on adverse event reporting can provide insight into the question of whether the FDA's adverse event reporting system operates as an early warning system for medical device problems. ^ The research focuses on two main questions. The first question addresses the relationship between publicity and the voluntary and mandatory reporting of adverse events. The second question examines whether government agencies make use of these adverse event reports. ^ Using quantitative and qualitative methods, a longitudinal study was conducted of the number and content of adverse event reports regarding breast implants filed with the FDA's medical device reporting system during 1985–1991. To assess variation in publicity over time, the print media were analyzed to identify articles related to breast implant failures. ^ The exploratory findings suggest that an increase in media activity is related to an increase in voluntary reporting, especially following periods of intense media coverage of the FDA. However, a similar relationship was not found between media activity and manufacturers' mandatory adverse event reporting. A review of government committee and agency reports on the FDA published during 1976–1996 produced little evidence to suggest that publicity or MDR information contributed to problem recognition, agenda setting, or the formulation of policy recommendations. ^ The research findings suggest that the reporting of breast implant problems to FDA may reflect the perceptions and concerns of the reporting groups, a barometer of the volume and content of media attention. ^

The synagogue and the church : being a contribution to the apologetics of Judaism

by Paul Goodman