ASSESSING THE UNRELIABILITY OF THE MEDICAL LITERATURE: A RESPONSE TO "WHY MOST PUBLISHED RESEARCH FINDINGS ARE FALSE"

A recent article in this journal (Ioannidis JP (2005) Why most published research findings are false. PLoS Med 2: e124) argued that more than half of published research findings in the medical literature are false. In this commentary, we examine the structure of that argument, and show that it has three basic components: 1)An assumption that the prior probability of most hypotheses explored in medical research is below 50%. 2)Dichotomization of P-values at the 0.05 level and introduction of a “bias” factor (produced by significance-seeking), the combination of which severely weakens the evidence provided by every design. 3)Use of Bayes theorem to show that, in the face of weak evidence, hypotheses with low prior probabilities cannot have posterior probabilities over 50%. Thus, the claim is based on a priori assumptions that most tested hypotheses are likely to be false, and then the inferential model used makes it impossible for evidence from any study to overcome this handicap. We focus largely on step (2), explaining how the combination of dichotomization and “bias” dilutes experimental evidence, and showing how this dilution leads inevitably to the stated conclusion. We also demonstrate a fallacy in another important component of the argument –that papers in “hot” fields are more likely to produce false findings. We agree with the paper’s conclusions and recommendations that many medical research findings are less definitive than readers suspect, that P-values are widely misinterpreted, that bias of various forms is widespread, that multiple approaches are needed to prevent the literature from being systematically biased and the need for more data on the prevalence of false claims. But calculating the unreliability of the medical research literature, in whole or in part, requires more empirical evidence and different inferential models than were used. The claim that “most research findings are false for most research designs and for most fields” must be considered as yet unproven.

Low prevalence of SV40 in Swiss mesothelioma patients after elimination of false-positive PCR results

The association of simian virus 40 (SV40) with malignant pleural mesothelioma is currently under debate. In some malignancies of viral aetiology, viral DNA can be detected in the patients' serum or plasma. To characterize the prevalence of SV40 in Swiss mesothelioma patients, we optimized a real-time PCR for quantitative detection of SV40 DNA in plasma, and used a monoclonal antibody for immunohistochemical detection of SV40 in mesothelioma tissue microarrays. Real-time PCR was linear over five orders of magnitude, and sensitive to a single gene copy. Repeat PCR determinations showed excellent reproducibility. However, SV40 status varied for independent DNA isolates of single samples. We noted that SV40 detection rates by PCR were drastically reduced by the implementation of strict room compartmentalization and decontamination procedures. Therefore, we systematically addressed common sources of contamination and found no cross-reactivity with DNA of other polyomaviruses. Contamination during PCR was rare and plasmid contamination was infrequent. SV40 DNA was reproducibly detected in only 4 of 78 (5.1%) plasma samples. SV40 DNA levels were low and not consistently observed in paired plasma and tumour samples from the same patient. Immunohistochemical analysis revealed a weak but reproducible SV40 staining in 16 of 341 (4.7%) mesotheliomas. Our data support the occurrence of non-reproducible SV40 PCR amplifications and underscore the importance of proper sample handling and analysis. SV40 DNA and protein were found at low prevalence (5%) in plasma and tumour tissue, respectively. This suggests that SV40 does not appear to play a major role in the development of mesothelioma.

Screening for carriage of methicillin-resistant Staphylococcus aureus shortly after exposure may lead to false-negative results

We evaluated a double screening strategy for carriage of methicillin-resistant Staphylococcus aureus (MRSA) in patients exposed to a newly detected MRSA carrier. If the first screening of the exposed patient yielded negative results, screening was repeated 4 days later. This strategy detected 12 (28%) of the 43 new MRSA carriers identified during the study period. The results suggest that there is an incubation period before MRSA carriage is detectable.

The Critical Friends Group: An Innovative Way to Build Intercultural Competence Among Student and Faculty Groups

Introduction Few physicians involved in medical education are likely to have had formal training in teaching. One pedagogical method that can enhance relationships, thus improve teaching and learning is the Critical Friends Group (CFG). The CFG is a collegial support team that offers improved understanding of others. Unconditional high regard for team members frames the interactions in the CFG. These teams could be used to reduce bias and enhance intercultural competence among student CFGs and faculty CFGs. [See PDF for complete abstract]

Friends of the library groups in health sciences libraries.

The Houston Academy of Medicine--Texas Medical Center (HAM--TMC) Library collected data on friends of the library groups from 103 health sciences libraries, using a mail questionnaire. Sixteen of the responding libraries had independent friends groups; seven had friends groups that were subordinate to a university group. The sixteen independent groups gave as their major purposes (1) to raise money for their associated library and (2) to develop support for their library. These groups contributed an average of $4,870 a year to their libraries, the money being used primarily to purchase rare books and working-collection books and to sponsor social events. The subordinate groups contributed relatively little money to the health sciences libraries responding to the survey.