The statistical significance of randomized controlled trial results is frequently fragile: A case for a Fragility Index

Objectives

A P-value <0.05 is one metric used to evaluate the results of a randomized controlled trial (RCT). We wondered how often statistically significant results in RCTs may be lost with small changes in the numbers of outcomes.

Study Design and Setting

A review of RCTs in high-impact medical journals that reported a statistically significant result for at least one dichotomous or time-to-event outcome in the abstract. In the group with the smallest number of events, we changed the status of patients without an event to an event until the P-value exceeded 0.05. We labeled this number the Fragility Index; smaller numbers indicated a more fragile result.

Results

The 399 eligible trials had a median sample size of 682 patients (range: 15-112,604) and a median of 112 events (range: 8-5,142); 53% reported a P-value <0.01. The median Fragility Index was 8 (range: 0-109); 25% had a Fragility Index of 3 or less. In 53% of trials, the Fragility Index was less than the number of patients lost to follow-up.

Conclusion

The statistically significant results of many RCTs hinge on small numbers of events. The Fragility Index complements the P-value and helps identify less robust results. 

An efficacy and mechanism evaluation study of Levosimendan for the Prevention of Acute oRgan Dysfunction in Sepsis (LeoPARDS): Protocol for a randomized controlled trial

Background

Organ dysfunction consequent to infection (‘severe sepsis’) is the leading cause of admission to an intensive care unit (ICU). In both animal models and early clinical studies the calcium channel sensitizer levosimendan has been demonstrated to have potentially beneficial effects on organ function. The aims of the Levosimendan for the Prevention of Acute oRgan Dysfunction in Sepsis (LeoPARDS) trial are to identify whether a 24-hour infusion of levosimendan will improve organ dysfunction in adults who have septic shock and to establish the safety profile of levosimendan in this group of patients.

This is a multicenter, randomized, double-blind, parallel group, placebo-controlled trial. Adults fulfilling the criteria for systemic inflammatory response syndrome due to infection, and requiring vasopressor therapy, will be eligible for inclusion in the trial. Within 24 hours of meeting these inclusion criteria, patients will be randomized in a 1:1 ratio stratified by the ICU to receive either levosimendan (0.05 to 0.2 μg.kg^-1.min^-1 or placebo for 24 hours in addition to standard care. The primary outcome measure is the mean Sequential Organ Failure Assessment (SOFA) score while in the ICU. Secondary outcomes include: central venous oxygen saturations and cardiac output; incidence and severity of renal failure using the Acute Kidney Injury Network criteria; duration of renal replacement therapy; serum bilirubin; time to liberation from mechanical ventilation; 28-day, hospital, 3 and 6 month survival; ICU and hospital length-of-stay; and days free from catecholamine therapy. Blood and urine samples will be collected on the day of inclusion, at 24 hours, and on days 4 and 6 post-inclusion for investigation of the mechanisms by which levosimendan might improve organ function. Eighty patients will have additional blood samples taken to measure levels of levosimendan and its active metabolites OR-1896 and OR-1855. A total of 516 patients will be recruited from approximately 25 ICUs in the United Kingdom.

This trial will test the efficacy of levosimendan to reduce acute organ dysfunction in adult patients who have septic shock and evaluate its biological mechanisms of action.

Total colonic dye-spray increases the detection of diminutive adenomas during routine colonoscopy:a randomized controlled trial

Background: Small adenomas may be missed during colonoscopy, but chromoscopy has been reported to enhance detection. The aim of this randomized-controlled trial was to determine the effect of total colonic dye spray on adenoma detection during routine colonoscopy.

Methods: Consecutive outpatients undergoing routine colonoscopy were randomized to a dye-spray group (0.1% indigo carmine used to coat the entire colon during withdrawal from the cecum) or control group (no dye).

Results: Two hundred fifty-nine patients were randomized, 124 to the dye-spray and 135 to the control group; demographics, indication for colonoscopy, and quality of the preparation were similar between the groups. Extubation from the cecum took a median of 9:05 minutes (range: 2:4824:44 min) in the dye-spray group versus 4:52 minutes (range: 1:42-15:21 min] in the control group (p <0.0001). The proportion of patients with at least 1 adenoma and the total number of adenomas were not different between groups. However, in the dye-spray group significantly more diminutive adenomas (

Conclusions: Dye-spray increases the detection of small adenomas in the proximal colon and patients with multiple adenomas, but long-term outcomes should be studied to determine the clinical value of these findings.

A randomized clinical trial of ascorbic acid in open abdominal aortic aneurysm repair

BACKGROUND: Open AAA repair is associated with ischaemia-reperfusion injury where systemic inflammation and endothelial dysfunction can lead to multiple organ injury including acute lung injury. Oxidative stress plays a role that may be inhibited by ascorbic acid.

METHODS: A double blind, allocation concealed, randomized placebo-controlled trial was performed to test the hypothesis that a single bolus dose (2g) of intra-operative parenteral ascorbic acid would attenuate biomarkers of ischaemia-reperfusion injury in patients undergoing elective open AAA repair.

RESULTS: Thirty one patients completed the study; 18 received placebo and 13 ascorbic acid. Groups were comparable demographically. Open AAA repair caused an increase in urinary Albumin:Creatinine Ratio (ACR) as well as plasma IL-6 and IL-8. There was a decrease in exhaled breath pH and oxygenation. Lipid hydroperoxides were significantly higher in the ascorbic acid group following open AAA repair. There were no other differences between the ascorbic acid or placebo groups up to 4 hours after removal of the aortic clamping.

CONCLUSIONS: Open AAA repair caused an increase in markers of systemic endothelial damage and systemic inflammation. Administration of 2g parenteral ascorbic acid did not attenuate this response and with higher levels of lipid hydroperoxides post-operatively a pro-oxidant effect could not be excluded.

TRIAL REGISTRATION: ISRCTN27369400.

Multicenter Randomized Controlled Trial of Withdrawal of Inhaled Corticosteroids in Cystic Fibrosis

Rationale: Lung inflammation and injury is critical in cystic fibrosis. An ideal antiinflammatory agent has not been identified but inhaled corticosteroids are widely used despite lack of evidence.

Objectives: To test the safety of withdrawal of inhaled corticosteroids with the hypothesis this would not be associated with an earlier onset of acute chest exacerbations.

Methods: Multicenter randomized double-blind placebo-controlled trial in 18 pediatric and adult UK centers. Eligibility criteria included age > 6.0 yr, FEV1 ? 40% predicted, and corticosteroid use > 3 mo. During the 2-mo run-in period, all patients received fluticasone; they then took either fluticasone or placebo for 6 mo.

Measurements and Main Results: Fluticasone group: n = 84, median age 14.6 yr, mean (SD) FEV1 76% (18); placebo group: n = 87, median age 15.8 yr, mean (SD) FEV1 76% (18). There was no difference in time to first exacerbation (primary outcome) with hazard ratio (95% confidence interval) of 1.07 (0.68 to 1.70) for fluticasone versus placebo. There was no effect of age, atopy, corticosteroid dose, FEV1, or Pseudomonas aeruginosa status. There was no change in lung function or differences in antibiotic or rescue bronchodilator use. Fewer patients in the fluticasone group withdrew from the study due to lung-related adverse events (9 vs. 15%); with a relative risk (95% confidence interval) of 0.59 (0.23–1.48) fluticasone versus placebo.

Conclusions: In this study population (applicable to 40% of patients with cystic fibrosis in the UK), it appears safe to consider stopping inhaled corticosteroids. Potential advantages will be to reduce the drug burden on patients, reduce adverse effects, and make financial savings.

Effect of eucaloric high- and low-sucrose diets with identical macronutrient profile on insulin resistance and vascular risk - A randomized controlled trial

The long-term impact of dietary carbohydrate type, in particular sucrose, on insulin resistance and the development of diabetes and atherosclerosis is not established. Current guidelines for the healthy population advise restriction of sucrose intake. We investigated the effect of high- versus low-sucrose diet (25 vs. 10%, respectively, of total energy intake) in 13 healthy subjects aged 33 +/- 3 years (mean +/- SE), BMI 26.6 +/- 0.9 kg/m(2), in a randomized crossover design with sequential 6-week dietary interventions separated by a 4-week washout. Weight maintenance, eucaloric diets with identical macronutrient profiles and fiber content were designed. All food was weighed and distributed. Insulin action was assessed using a two-step euglycemic clamp; glycemic profiles were assessed by the continuous glucose monitoring system and vascular compliance by pulse-wave analysis. There was no change in weight across the study. Peripheral glucose uptake and suppression of endogenous glucose production were similar after each diet. Glycemic profiles and measures of vascular compliance did not change. A rise in total and LDL cholesterol was observed. In this study, a high-sucrose intake as part of an eucaloric, weight-maintaining diet had no detrimental effect on insulin sensitivity, glycemic profiles, or measures of vascular compliance in healthy nondiabetic subjects.

Training of Reaching in Stroke Survivors With Severe and Chronic Upper Limb Paresis Using a Novel Nonrobotic Device: A Randomized Clinical Trial

Background and Purpose—Severe upper limb paresis is a major contributor to disability after stroke. This study investigated the efficacy of a new nonrobotic training device, the Sensorimotor Active Rehabilitation Training (SMART) Arm, that was used with or without electromyography-triggered electrical stimulation of triceps brachii to augment elbow extension, permitting stroke survivors with severe paresis to practice a constrained reaching task.

Methods—A single-blind, randomized clinical trial was conducted with 42 stroke survivors with severe and chronic paresis. Thirty-three participants completed the study, of whom 10 received training using the SMART Arm with electromyography-triggered electrical stimulation, 13 received training using the SMART Arm alone, and 10 received no intervention (control). Training consisted of 12 1-hour sessions over 4 weeks. The primary outcome measure was “upper arm function,” item 6 of the Motor Assessment Scale. Secondary outcome measures included impairment measures; triceps muscle strength, reaching force, modified Ashworth scale; and activity measures: reaching distance and Motor Assessment Scale. Assessments were administered before (0 weeks) and after training (4 weeks) and at 2 months follow-up (12 weeks).

Results—Both SMART Arm groups demonstrated significant improvements in all impairment and activity measures after training and at follow-up. There was no significant difference between these 2 groups. There was no change in the control group.

Conclusions—Our findings indicate that training of reaching using the SMART Arm can reduce impairment and improve activity in stroke survivors with severe and chronic upper limb paresis, highlighting the benefits of intensive task-oriented practice, even in the context of severe paresis.