Fragility of Evidence Supporting Anti-Fracture Treatment Efficacy in RCTs

Overview

This analysis of 27 phase 3/4 randomized controlled trials (RCTs) reveals that the evidence supporting anti-fracture efficacy of pharmacological interventions is often fragile. The median Fragility Index (FI) was 9, indicating that changing outcomes in as few as 9 patients could negate statistical significance, with many trials having more participants lost to follow-up than their FI.

Background

Randomized controlled trials are the gold standard for assessing intervention efficacy, typically relying on P values to determine statistical significance. However, P values can be misleading, especially with small sample sizes or low event rates, as minor data changes may shift results from significant to nonsignificant. The Fragility Index (FI) and Fragility Quotient (FQ) have been proposed to quantify the robustness of trial results beyond P values. This study evaluates the fragility of evidence for anti-fracture efficacy in pharmacological treatments for osteoporosis.

Data Highlights

Key Findings

• The median Fragility Index across 27 RCTs was 9, meaning reversing outcomes in 9 patients could eliminate statistical significance.
• Approximately 0.51% of study participants corresponded to the median FI, indicating high sensitivity of results to small changes.
• In 60% of analyses, the number of participants lost to follow-up exceeded the FI, raising concerns about result robustness.
• Romosozumab showed the most robust anti-fracture evidence (median FI 19.5), while denosumab and calcium/vitamin D supplementation had lower FI values (4 and 7, respectively).
• Trials with fractures as the primary endpoint or with P values less than .001 demonstrated higher FI values, indicating more robust evidence.

Clinical Implications

Clinicians should interpret anti-fracture efficacy data with caution, considering the fragility of statistical significance in many RCTs. Incorporating the Fragility Index and Fragility Quotient into guideline development and patient discussions can improve risk communication and decision-making. Awareness of loss to follow-up exceeding FI highlights the need for rigorous trial conduct and data completeness.

Conclusion

The current RCT evidence supporting anti-fracture efficacy of pharmacological treatments is frequently fragile, underscoring the importance of supplementing P value interpretation with fragility metrics to better inform clinical practice.

References

1. Assessing the Vulnerability of Evidence Supporting Anti-Fracture Treatment Efficacy, 2024