Oral Presentation 33rd ASM of the Australian & New Zealand Bone & Mineral Society 2023

Assessing the robustness of evidence for the efficacy of anti-fracture  medications (#49)

Nick Tran 1 2 , Thach Tran 1 3 , Tuan V Nguyen 1
  1. School of Biomedical Engineering, University of Technology Sydney, Sydney, NSW, Australia
  2. Macquarie University, Sydney, NSW, Australia
  3. Garvan Institute of Medical Research, Darlinghurst, NSW, Australia

The evidence of the efficacy of anti-fracture medications is conventionally derived from randomized controlled trials (RCTs) with P-values< 0.05 being the gold standard. This study sought to quantify the fragility of the RCT evidence for anti-fracture medications.

This analysis included 22 RCTs in high-impact medical journals which generated 110 statistically significant results (P< 0.05) for at least one fracture site. The fragility of the evidence was assessed by the 'Fragility Index' (FI), which is defined as the minimum number of patients whose condition would need to change from non-fracture to fracture to invalidate a statistically significant outcome.

The overall median FI was 9 (IQR: 3, 11), indicating that adding as few as 9 fracture patients (~0.4% of the study size) to the intervention group would eliminate the previously documented evidence of fracture prevention efficacy. Notably, in 65% of these analyses, the number of participants lost to follow-up exceeded the corresponding FI. Among the 34 results with P< 0.001, the median FI was 25 (17, 47) which was lower than the number of participants lost to follow-up in 25 results. Specifically, the evidence of anti-fracture efficacy of denosumab and calcium/vitamin D supplementation would be lost if only additional 4 (3, 17) and 6 (2, 16) patients respectively, sustained a fracture during the follow-up period (Figure). Among 37 positive results (~34%) that used fracture as the primary outcome measure, the evidence for anti-fracture efficacy remained fragile, with a median FI being only 15 (8, 25). In approximately 90% of these positive results, the number of participants lost to follow-up was higher than that required to render the results statistically non-significant.

These data indicate that the existing RCT evidence of anti-fracture efficacy is fragile. In order to increase the robustness of the anti-fracture evidence, the P-value threshold should be lower than 0.001.

 

64b9c20ded5de-FI+in+bone+research.jpg