The statistical significance of randomized controlled trial results is frequently fragile: A case for a Fragility Index

Objectives

A P-value <0.05 is one metric used to evaluate the results of a randomized controlled trial (RCT). We wondered how often statistically significant results in RCTs may be lost with small changes in the numbers of outcomes.

Study Design and Setting

A review of RCTs in high-impact medical journals that reported a statistically significant result for at least one dichotomous or time-to-event outcome in the abstract. In the group with the smallest number of events, we changed the status of patients without an event to an event until the P-value exceeded 0.05. We labeled this number the Fragility Index; smaller numbers indicated a more fragile result.

Results

The 399 eligible trials had a median sample size of 682 patients (range: 15-112,604) and a median of 112 events (range: 8-5,142); 53% reported a P-value <0.01. The median Fragility Index was 8 (range: 0-109); 25% had a Fragility Index of 3 or less. In 53% of trials, the Fragility Index was less than the number of patients lost to follow-up.

Conclusion

The statistically significant results of many RCTs hinge on small numbers of events. The Fragility Index complements the P-value and helps identify less robust results. 

An efficacy and mechanism evaluation study of Levosimendan for the Prevention of Acute oRgan Dysfunction in Sepsis (LeoPARDS): Protocol for a randomized controlled trial

Background

Organ dysfunction consequent to infection (‘severe sepsis’) is the leading cause of admission to an intensive care unit (ICU). In both animal models and early clinical studies the calcium channel sensitizer levosimendan has been demonstrated to have potentially beneficial effects on organ function. The aims of the Levosimendan for the Prevention of Acute oRgan Dysfunction in Sepsis (LeoPARDS) trial are to identify whether a 24-hour infusion of levosimendan will improve organ dysfunction in adults who have septic shock and to establish the safety profile of levosimendan in this group of patients.

This is a multicenter, randomized, double-blind, parallel group, placebo-controlled trial. Adults fulfilling the criteria for systemic inflammatory response syndrome due to infection, and requiring vasopressor therapy, will be eligible for inclusion in the trial. Within 24 hours of meeting these inclusion criteria, patients will be randomized in a 1:1 ratio stratified by the ICU to receive either levosimendan (0.05 to 0.2 μg.kg^-1.min^-1 or placebo for 24 hours in addition to standard care. The primary outcome measure is the mean Sequential Organ Failure Assessment (SOFA) score while in the ICU. Secondary outcomes include: central venous oxygen saturations and cardiac output; incidence and severity of renal failure using the Acute Kidney Injury Network criteria; duration of renal replacement therapy; serum bilirubin; time to liberation from mechanical ventilation; 28-day, hospital, 3 and 6 month survival; ICU and hospital length-of-stay; and days free from catecholamine therapy. Blood and urine samples will be collected on the day of inclusion, at 24 hours, and on days 4 and 6 post-inclusion for investigation of the mechanisms by which levosimendan might improve organ function. Eighty patients will have additional blood samples taken to measure levels of levosimendan and its active metabolites OR-1896 and OR-1855. A total of 516 patients will be recruited from approximately 25 ICUs in the United Kingdom.

This trial will test the efficacy of levosimendan to reduce acute organ dysfunction in adult patients who have septic shock and evaluate its biological mechanisms of action.

Clinical trial designs for rare diseases: Studies developed and discussed by the International Rare Cancers Initiative

BACKGROUND: The past three decades have seen rapid improvements in the diagnosis and treatment of most cancers and the most important contributor has been research. Progress in rare cancers has been slower, not least because of the challenges of undertaking research.

SETTINGS: The International Rare Cancers Initiative (IRCI) is a partnership which aims to stimulate and facilitate the development of international clinical trials for patients with rare cancers. It is focused on interventional--usually randomized--clinical trials with the clear goal of improving outcomes for patients. The key challenges are organisational and methodological. A multi-disciplinary workshop to review the methods used in ICRI portfolio trials was held in Amsterdam in September 2013. Other as-yet unrealised methods were also discussed.

RESULTS: The IRCI trials are each presented to exemplify possible approaches to designing credible trials in rare cancers. Researchers may consider these for use in future trials and understand the choices made for each design.

INTERPRETATION: Trials can be designed using a wide array of possibilities. There is no 'one size fits all' solution. In order to make progress in the rare diseases, decisions to change practice will have to be based on less direct evidence from clinical trials than in more common diseases.

Total colonic dye-spray increases the detection of diminutive adenomas during routine colonoscopy:a randomized controlled trial

Background: Small adenomas may be missed during colonoscopy, but chromoscopy has been reported to enhance detection. The aim of this randomized-controlled trial was to determine the effect of total colonic dye spray on adenoma detection during routine colonoscopy.

Methods: Consecutive outpatients undergoing routine colonoscopy were randomized to a dye-spray group (0.1% indigo carmine used to coat the entire colon during withdrawal from the cecum) or control group (no dye).

Results: Two hundred fifty-nine patients were randomized, 124 to the dye-spray and 135 to the control group; demographics, indication for colonoscopy, and quality of the preparation were similar between the groups. Extubation from the cecum took a median of 9:05 minutes (range: 2:4824:44 min) in the dye-spray group versus 4:52 minutes (range: 1:42-15:21 min] in the control group (p <0.0001). The proportion of patients with at least 1 adenoma and the total number of adenomas were not different between groups. However, in the dye-spray group significantly more diminutive adenomas (

Conclusions: Dye-spray increases the detection of small adenomas in the proximal colon and patients with multiple adenomas, but long-term outcomes should be studied to determine the clinical value of these findings.

Multicenter Randomized Controlled Trial of Withdrawal of Inhaled Corticosteroids in Cystic Fibrosis

Rationale: Lung inflammation and injury is critical in cystic fibrosis. An ideal antiinflammatory agent has not been identified but inhaled corticosteroids are widely used despite lack of evidence.

Objectives: To test the safety of withdrawal of inhaled corticosteroids with the hypothesis this would not be associated with an earlier onset of acute chest exacerbations.

Methods: Multicenter randomized double-blind placebo-controlled trial in 18 pediatric and adult UK centers. Eligibility criteria included age > 6.0 yr, FEV1 ? 40% predicted, and corticosteroid use > 3 mo. During the 2-mo run-in period, all patients received fluticasone; they then took either fluticasone or placebo for 6 mo.

Measurements and Main Results: Fluticasone group: n = 84, median age 14.6 yr, mean (SD) FEV1 76% (18); placebo group: n = 87, median age 15.8 yr, mean (SD) FEV1 76% (18). There was no difference in time to first exacerbation (primary outcome) with hazard ratio (95% confidence interval) of 1.07 (0.68 to 1.70) for fluticasone versus placebo. There was no effect of age, atopy, corticosteroid dose, FEV1, or Pseudomonas aeruginosa status. There was no change in lung function or differences in antibiotic or rescue bronchodilator use. Fewer patients in the fluticasone group withdrew from the study due to lung-related adverse events (9 vs. 15%); with a relative risk (95% confidence interval) of 0.59 (0.23–1.48) fluticasone versus placebo.

Conclusions: In this study population (applicable to 40% of patients with cystic fibrosis in the UK), it appears safe to consider stopping inhaled corticosteroids. Potential advantages will be to reduce the drug burden on patients, reduce adverse effects, and make financial savings.

Effect of eucaloric high- and low-sucrose diets with identical macronutrient profile on insulin resistance and vascular risk - A randomized controlled trial

The long-term impact of dietary carbohydrate type, in particular sucrose, on insulin resistance and the development of diabetes and atherosclerosis is not established. Current guidelines for the healthy population advise restriction of sucrose intake. We investigated the effect of high- versus low-sucrose diet (25 vs. 10%, respectively, of total energy intake) in 13 healthy subjects aged 33 +/- 3 years (mean +/- SE), BMI 26.6 +/- 0.9 kg/m(2), in a randomized crossover design with sequential 6-week dietary interventions separated by a 4-week washout. Weight maintenance, eucaloric diets with identical macronutrient profiles and fiber content were designed. All food was weighed and distributed. Insulin action was assessed using a two-step euglycemic clamp; glycemic profiles were assessed by the continuous glucose monitoring system and vascular compliance by pulse-wave analysis. There was no change in weight across the study. Peripheral glucose uptake and suppression of endogenous glucose production were similar after each diet. Glycemic profiles and measures of vascular compliance did not change. A rise in total and LDL cholesterol was observed. In this study, a high-sucrose intake as part of an eucaloric, weight-maintaining diet had no detrimental effect on insulin sensitivity, glycemic profiles, or measures of vascular compliance in healthy nondiabetic subjects.

A prospective clinical trial of a real-time polymerase chain reaction assay for the diagnosis of candidemia in nonneutropenic, critically ill adults

Background. Invasive Candida infection among nonneutropenic, critically ill adults is a clinical problem that has received increasing attention in recent years. Poor performance of extant diagnostic modalities has promoted risk-based, preemptive prescribing in view of the poor outcomes associated with inadequate or delayed antifungal therapy; this risks unnecessary overtreatment. A rapid, reliable diagnostic test could have a substantial impact on therapeutic practice in this patient population.

Methods. Three TaqMan-based real-time polymerase chain reaction assays were developed that are capable of detecting the main medically important Candida species, categorized according to the likelihood of fluconazole susceptibility. Assay 1 detected Candida albicans, Candida parapsilosis, Candida tropicalis, and Candida dubliniensis. Assays 2 and 3 detected Candida glabrata and Candida krusei, respectively. The clinical performance of these assays, applied to serum, was evaluated in a prospective trial of nonneutropenic adults in a single intensive care unit.

Results. In all, 527 specimens were obtained from 157 participants. All 3 assays were run in parallel for each specimen; they could be completed within 1 working day. Of these, 23 specimens were obtained from 23 participants categorized as having proven Candida infection at the time of sampling. If a single episode of Candida famata candidemia was excluded, the estimated clinical sensitivity, specificity, and positive and negative predictive values of the assays in this trial were 90.9%, 100%, 100% and 99.8%, respectively.

Conclusions. These data suggest that the described assays perform well in this population for enhancing the diagnosis of candidemia. The extent to which they may affect clinical outcomes, prescribing practice, and cost-effectiveness of care remains to be ascertained.