Statistical Fragility of Surgical and Procedural Clinical Trials in Orthopaedic Oncology

Forrester, Lynn Ann; Jang, Eugene; Lawson, Michelle; Capi, Ana; Tyler, Wakenda

doi:10.5435/jaaosglobal-d-19-00152

Cited by 17 publications

(28 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[49][50][51] The FI and FQ have also been utilized for orthopedic research, often illustrating a body of literature whose results are far from robust. A recent article by Forrester et al 15 of 23 clinical trials in orthopedic oncology reported a median FI of 2, which the authors attributed to small sample sizes and large losses to follow-up. In a similar study examining 48 RCTs in sports medicine, Khan et al 13 found results to be fragile with a median FI of 2 and loss to follow-up exceeding a study's FI for nearly half (48%) of outcomes.…”

Section: Discussionmentioning

confidence: 99%

“…[8][9][10][11] Within the orthopedic literature, FI and FQ have been used to evaluate the findings of a multitude of subspecialties, from pediatric trauma to orthopedic oncology. [12][13][14][15][16][17][18] The majority of these studies have demonstrated significant statistical fragility, thus calling into question the robustness of study conclusions. However, the fragility of studies pertaining to articular cartilage defects of the knee has yet to be explored in the literature.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Fragility of Statistical Significance in Cartilage Restoration of the Knee: A Systematic Review of Randomized Controlled Trials

et al. 2021

View full text Add to dashboard Cite

Objective The purpose of this study was to utilize fragility analysis to assess the robustness of randomized controlled trials (RCTs) evaluating the management of articular cartilage defects of the knee. We hypothesize that the cartilage restorative literature will be fragile with the reversal of only a few outcome events required to change statistical significance. Design RCTs from 11 orthopedic journals indexed on PubMed from 2000 to 2020 reporting dichotomous outcome measures relating to the management of articular cartilage defects of the knee were included. The Fragility Index (FI) for each outcome was calculated through the iterative reversal of a single outcome event until significance was reversed. The Fragility Quotient (FQ) was calculated by dividing each FI by study sample size. Additional statistical analysis was performed to provide median FI and FQ across subgroups. Results Nineteen RCTs containing 60 dichotomous outcomes were included for analysis. The FI and FQ of all outcomes was 4 (IQR 2-7) and 0.067 (IQR 0.034-0.096), respectively. The average number of patients lost to follow-up (LTF) was 3.9 patients with 15.8% of the included studies reporting LTF greater than or equal to 4, the FI of all included outcomes. Conclusions The orthopedic literature evaluating articular cartilage defects of the knee is fragile as the reversal of relatively few outcome events may alter the significance of statistical findings. We therefore recommend comprehensive fragility analysis and triple reporting of the P value, FI, and FQ to aid in the interpretation and contextualization of clinical findings reported in the cartilage restoration literature.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The Fragility of Statistical Significance in Cartilage Restoration of the Knee: A Systematic Review of Randomized Controlled Trials

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Similarly, in an evaluation of statistical fragility in both the spine and orthopedic oncology literature, a FI of just two was reported. 18 , 27 Regarding the orthopaedic trauma literature, Parisien et al 25 analyzed 198 studies consisting of 775 total outcome events and demonstrated a FI of five and associated FQ of 0.046, representing just 3.8% of the total study population. Furthermore, the evaluation of RCTs in the pediatric orthopaedic literature were found to have a FI of only three.…”

Section: Discussionmentioning

confidence: 99%

The Fragility of Statistical Findings in Achilles Tendon Injury Research: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

Introduction: Probability analysis with the reporting of P values is often used to determine the statistical significance of study findings in the Achilles literature. The purpose of this study was to determine the utility of applying a fragility analysis to comparative trials evaluating Achilles tendon injuries. Methods: We identified all dichotomous outcome data for comparative studies of Achilles tendon injuries published in 11 orthopaedic journals from 2000 to 2020. The fragility index (FI) was determined by the number of event reversals required to change a P value from less than 0.05 to greater than 0.05, or vice-versa. The associated fragility quotient was determined by dividing the FI by the sample size. Results: Of the 51,357 studies screened, 1,487 met the search criteria, with 51 comparative studies and 177 total outcome events included for analysis. The overall FI was only 4 with an associated fragility quotient of 0.048. One-half of the studies failed to report lost to follow-up data, with an additional 21.6% reporting loss to follow-up of greater than or equal to 4. Conclusion: Our fragility analysis suggests that Achilles tendon injury outcomes are not as statistically stable as previously thought and should be interpreted with caution.

show abstract

“…Additionally, in an evaluation of the sports medicine literature, Parisien et al 26 reported that the average LTF value (7.9) was greater than the overall FI of 5. Furthermore, in an evaluation of the orthopaedic oncology literature, Forrester et al found that 60% of the outcomes had an FI value that was less than or equal to the LTF value 23 . Parisien et al, in a recent fragility analysis evaluating the cartilage restoration literature, found that 15.8% of studies either did not report LTF data or reported an LTF value that was greater than the FI 29 .…”

Section: Discussionmentioning

confidence: 99%

The Fragility of Significance in the Hip Arthroscopy Literature

et al. 2021

View full text Add to dashboard Cite

Background: The purpose of the present study was to perform the first examination of the utility of p values and the degree of statistical fragility in the hip arthroscopy literature by applying both the Fragility Index (FI) and the Fragility Quotient (FQ) to dichotomous comparative trials. We hypothesized that dichotomous comparative trials evaluating categorical outcomes in the hip arthroscopy literature are statistically fragile. Methods: The PubMed and MEDLINE databases were queried from 2008-2018 for comparative studies evaluating dichotomous data in the hip arthroscopy literature. The present analysis included both randomized controlled trials (RCTs) and non-RCTs in which dichotomous data and associated p values were reported. Fragility analysis was performed with use of the Fisher exact test until an alteration of significance was determined. Results: Of the 5,836 studies screened, 4,156 met the search criteria, with 52 comparative studies included for analysis. One hundred and fifty total outcome events with 33 significant (p < 0.05) outcomes and 117 nonsignificant (p ≥ 0.05) outcomes were identified. The final FI incorporating all 150 outcome events from 52 comparative studies was only 3.5 (interquartile range, 2 to 6), with an associated FQ of 0.032 (interquartile range, 0.017 to 0.063). Twenty-two studies (42.3%) either failed to report loss to follow-up (LTF) data or reported LTF greater than the overall FI of 3.5. Conclusions: The peer-reviewed hip arthroscopy literature may not be as stable as previously thought, as the sole reliance on a threshold p value has proven misleading. We therefore recommend reporting of the FI and FQ, in conjunction with p values, to aid in the evaluation and interpretation of statistical robustness and quantitative significance in future comparative hip arthroscopy studies.

show abstract

Statistical Fragility of Surgical and Procedural Clinical Trials in Orthopaedic Oncology

Cited by 17 publications

References 18 publications

The Fragility of Statistical Significance in Cartilage Restoration of the Knee: A Systematic Review of Randomized Controlled Trials

The Fragility of Statistical Significance in Cartilage Restoration of the Knee: A Systematic Review of Randomized Controlled Trials

The Fragility of Statistical Findings in Achilles Tendon Injury Research: A Systematic Review

The Fragility of Significance in the Hip Arthroscopy Literature

Contact Info

Product

Resources

About