The fragility of significant results underscores the need of larger randomized controlled trials in nephrology

Shochet, Lani; Kerr, Peter G.; Polkinghorne, Kevan R.

doi:10.1016/j.kint.2017.05.011

Cited by 47 publications

(31 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Collectively, these findings suggest that the evidence base supporting the severe TBI management guidelines may be considered "fragile", in that the interpretation of study findings can be swayed by a small number of events. The median positive FI F was 3, which is similar to other studies that have evaluated FI in different fields [6][7][8][9]11]. Thus our findings are not unique to the severe TBI management guidelines, but appear consistent with the view that much of the biomedical literature is founded on weak evidence [21].…”

Section: Fragility Index Analysis Of the Tbi Literature And Comparisosupporting

confidence: 89%

“…On face value, FI offers a straightforward interpretation of the susceptibility of a trial to change in the number of events, and this has led to a large number of studies applying FI analysis to various topics in recent years [2][3][4][5][6][7][8][9][10][11]24]. However, such an interpretation overlooks other shortcomings associated with FI.…”

Section: Plos Onementioning

confidence: 99%

“…The fragility index (FI) represents the minimum number of events required to change the statistical significance of a study result from significant to nonsignificant [2], and has largely been promoted as a measure to assess the robustness of randomised controlled trial (RCT) results with a dichotomous endpoint. Evaluations of FI have been performed in diverse areas ranging from heart failure [4], diabetes [5], nephrology [6], anaesthesiology [7], oncology [8], ischaemic stroke [9], intracranial haemorrhage [10], to paediatrics [11]. These studies have demonstrated that many RCTs have a low FI, with median FI ranging from 2 to 26 (median of the median 4), suggesting that the interpretation of RCT results commonly hinge on a small number of results.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The weakness of fragility index exposed in an analysis of the traumatic brain injury management guidelines: A meta-epidemiological and simulation study

et al. 2020

View full text Add to dashboard Cite

Objectives To perform fragility index (FI) analysis on the evidence that forms the basis of the guidelines for the management of severe traumatic brain injury (TBI), and develop a deeper understanding of the pitfalls associated with FI. Design Meta-epidemiological analysis and numerical simulations. Methods The Brain Trauma Foundation guidelines (4 th edition) for management of severe TBI were used to identify relevant randomised controlled trials (RCTs). FI based on Fisher's exact test and relative risk was performed on eligible RCTs. The relationship between FI, event counts and P values was explored by exhaustively considering different combinations of outcomes for studies of total size ranging from 80 to 10000. Sample size calculations were also performed for a range of power, baseline risk and relative risk, to determine the influence of study design on FI. Results FI analysis of the severe TBI management guidelines revealed that most studies were associated with a low FI. In the majority of studies, FI was of a similar magnitude to the number lost to follow-up. The simulations revealed that while FI was inversely related to P value, a wide range of FI may be associated with a given P value. FI is also affected by sample size, baseline risk and effect size. Sample size calculations suggest that aside from very high-PLOS ONE

show abstract

Section: Fragility Index Analysis Of the Tbi Literature And Comparisosupporting

confidence: 89%

Section: Plos Onementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The weakness of fragility index exposed in an analysis of the traumatic brain injury management guidelines: A meta-epidemiological and simulation study

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Our findings are consistent with previous FI studies that reported a range of FIs from 2 to 7. [14][15][16][17][18][19][20] First, in general medicine, Walsh et al reported an FI of 8 in trials published in general medicine journals 15 and most recently, Shochet et al reported an FI of 2 in nephrology trials. 16 All of the previous studies compared the percentage of trials with loss to follow-up greater than the endpoint FI.…”

Section: Discussionmentioning

confidence: 99%

The Fragility Index in a Cohort of HIV/AIDS Randomized Controlled Trials

et al. 2019

View full text Add to dashboard Cite

HIV/AIDS is associated with significant morbidity, mortality, and financial burden. For these reasons, robust clinical evidence is critical. We aim to investigate the fragility index, fragility quotient, and risk of bias of clinical trial endpoints in HIV medicine. The fragility index represents the minimum amount of trial endpoint Bnoneventsĉ hanged to Bevents^in one trial arm required to nullify statistical significance. The fragility quotient contextualized the fragility index by dividing the index by the total trial sample size. We selected eligible trials from the Department of Health and Human Services guideline for the use of antiretroviral agents in HIV-1-infected adults and adolescents. We calculated the fragility index and fragility quotient for all included trials. The Cochrane Brisk of biasT ool 2.0 was used to evaluate the likelihood and sources of bias in the included trials. Thirty-nine RCTs were included for our analysis of fragility. Thirty-six were included for our analysis of the risk of bias. The median fragility index was 5. Three RCTs were at high risk of bias, all due to the selection of the endpoint or statistical test. Twenty had some concerns for risk of bias. The analyzed HIV medicine RCT endpoints were fragile, overall. This indicates that a median of 5 patients across all included studies would nullify the statistical significance of the endpoints. Furthermore, we found evidence that concerns for bias are present at a high rate.

show abstract

“…18 Additionally, deficits in the quality of reporting of such trials remain, 18 and the trials are often small and therefore underpowered. [19][20][21] It is clear that survival for patients with ESKD is poor and only modestly improving, and the work of Naylor et al puts this into sharp focus through comparison with patients with the most common forms of solid-organ cancer. While it is appropriate that the prevention of ESKD is a top priority for the nephrology community, improving survival outcomes for patients receiving dialysis is also imperative.…”

mentioning

confidence: 99%