The statistical significance of randomized controlled trial results is frequently fragile: A case for a Fragility Index

Objectives

A P-value <0.05 is one metric used to evaluate the results of a randomized controlled trial (RCT). We wondered how often statistically significant results in RCTs may be lost with small changes in the numbers of outcomes.

Study Design and Setting

A review of RCTs in high-impact medical journals that reported a statistically significant result for at least one dichotomous or time-to-event outcome in the abstract. In the group with the smallest number of events, we changed the status of patients without an event to an event until the P-value exceeded 0.05. We labeled this number the Fragility Index; smaller numbers indicated a more fragile result.

Results

The 399 eligible trials had a median sample size of 682 patients (range: 15-112,604) and a median of 112 events (range: 8-5,142); 53% reported a P-value <0.01. The median Fragility Index was 8 (range: 0-109); 25% had a Fragility Index of 3 or less. In 53% of trials, the Fragility Index was less than the number of patients lost to follow-up.

Conclusion

The statistically significant results of many RCTs hinge on small numbers of events. The Fragility Index complements the P-value and helps identify less robust results. 

Diagnostic Performance and Additional Value of Elastosonography in Focal Breast Lesions: Statistical Correlation between Size-Dependant Strain Index Measurements, Multimodality-BI-RADS Score, and Histopathology in a Clinical Routine Setting

Objective. To evaluate the diagnostic benefit of real-time elastography (RTE) in clinical routine. Strain indices (SI) for benign and malignant tumors were assessed. Methods. 100 patients with 110 focal breast lesions were retrieved. Patients had mammography (MG), ultrasound (US), and, if necessary, MRI. RTE was conducted after ultrasound. Lesions were assessed with BI-RADS for mammography and ultrasound. Diagnosis was established with histology or follow-up. Results. SI for BI-RADS 2 was 1.71 ± 0.86. Higher SI (2.21 ± 1.96) was observed for BI-RADS 3 lesions. SI of BI-RADS 4 and 5 lesions were significantly higher (16.92 ± 20.89) and (19.54 ± 10.41). 31 malignant tumors exhibited an average SI of 16.13 ± 14.67; SI of benign lesions was 5.29 ± 11.87 (P value <0.0001). ROC analysis threshold was >3.8 for malignant disease. Sensitivity of sonography was 90.3% (specificity 78.5%). RTE showed a sensitivity of 87.1% (specificity 79.7%). Accuracy of all modalities combined was 96.8%. In BI-RADS 3 lesions RTE was able to detect all malignant lesions (sensitivity 100%, specificity 92.9%, and accuracy 93.9%). Conclusions. RTE increased sensitivity and specificity for breast cancer detection when used in combination with ultrasound.