Fragility index of network meta-analysis with application to smoking cessation data

Xing, Aiwen; Chu, Haitao; Lin, Lifeng

doi:10.1016/j.jclinepi.2020.07.003

Cited by 11 publications

(10 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following this first report, many medical specialties examined the results of "their" randomised-controlled trials for fragility and found similar results: trial results are often fragile [2][3][4][5]. Consequently, the fragility index has been extended to evaluate metaanalyses [8,14,15], and already been expanded to network meta-analyses [16]. Fragility quotient of the included meta-analyses with a cumulative sum to better visualise the distribution.…”

Section: Discussionmentioning

confidence: 99%

Meta-analyses in paediatric surgery are often fragile: implications and consequences

2021

View full text Add to dashboard Cite

Purpose Meta-analyses occupy the highest level of evidence and thereby guide clinical decision-making. Recently, randomised-controlled trials were evaluated for the robustness of their findings by calculating the fragility index. The fragility index is the number of events that needs to be added to one treatment arm until the statistical significance collapses. We, therefore, aimed to evaluate the robustness of paediatric surgical meta-analyses. Methods We searched MEDLINE for paediatric surgical meta-analyses in the last decade. All meta-analyses on a paediatric surgical condition were eligible for analysis if they based their conclusion on a statistically significant meta-analysis. Results We screened 303 records and conducted a full-text evaluation of 60 manuscripts. Of them, 39 were included in our analysis that conducted 79 individual meta-analyses with significant results. Median fragility index was 5 (Q25–Q75% 2–11). Median fragility in relation to included patients was 0.77% (Q25–Q75% 0.29–1.87%). Conclusion Paediatric surgical meta-analyses are often fragile. In almost 60% of results, the statistical significance depends on less than 1% of the included population. However, as the fragility index is just a transformation of the P value, it basically conveys the same information in a different format. It therefore should be avoided.

show abstract

Section: Discussionmentioning

confidence: 99%

Meta-analyses in paediatric surgery are often fragile: implications and consequences

2021

View full text Add to dashboard Cite

show abstract

“…Various methods have been developed to perform NMA under both the frequentist and Bayesian frameworks [ 78 – 87 ]. To assess the fragility of an NMA, similar iterative procedures for a pairwise MA can be used [ 42 ]. We focus on the frequentist method by Rücker [ 79 ] to produce the CIs of treatment comparisons in the NMA.…”

Section: Methodsmentioning

confidence: 99%

“…Similar to pre-registering clinical trials, pre-registered prospective MAs have been recommended [ 39 , 40 ]. Recently, the FI was extended to assess the fragility of conventional pairwise MAs as well as network meta-analyses (NMAs) of multiple treatment comparisons [ 41 , 42 ]. The FI of an MA is defined similarly to that in a clinical trial; however, its estimation is more complicated, because the modifications of event status may occur in different studies within the MA.…”

Section: Introductionmentioning

confidence: 99%

Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package

Lin

Chu

2022

PLoS ONE

Self Cite

View full text Add to dashboard Cite

With the growing concerns about research reproducibility and replicability, the assessment of scientific results’ fragility (or robustness) has been of increasing interest. The fragility index was proposed to quantify the robustness of statistical significance of clinical studies with binary outcomes. It is defined as the minimal event status modifications that can alter statistical significance. It helps clinicians evaluate the reliability of the conclusions. Many factors may affect the fragility index, including the treatment groups in which event status is modified, the statistical methods used for testing for the association between treatments and outcomes, and the pre-specified significance level. In addition to assessing the fragility of individual studies, the fragility index was recently extended to both conventional pairwise meta-analyses and network meta-analyses of multiple treatment comparisons. It is not straightforward for clinicians to calculate these measures and visualize the results. We have developed an R package called “fragility” to offer user-friendly functions for such purposes. This article provides an overview of methods for assessing and visualizing the fragility of individual studies as well as pairwise and network meta-analyses, introduces the usage of the “fragility” package, and illustrates the implementations with several worked examples.

show abstract

“…Thereby, it reports the number of events that would have transformed statistically insignificant results into statistically significant ones [ 8 ]. (3) The fragility index for meta-analyses[ 9 ] and network meta-analyses [ 10 ] that allow a similar assessment, not only of trials, but their synthesis in meta-analyses.…”

Section: Introductionmentioning

confidence: 99%

Paediatric surgical trials, their fragility index, and why to avoid using it to evaluate results

2022

View full text Add to dashboard Cite

Background The fragility index has been gaining ground in the evaluation of comparative clinical studies. Many scientists evaluated trials in their fields and deemed them to be fragile, although there is no consensus on the definition of fragility. We aimed to calculate the fragility index and its permutations for paediatric surgical trials. Methods We searched pubmed for prospectively conducted paediatric surgical trials with intervention and control group without limitations and calculated their (reverse) fragility indices and respective quotients along with posthoc-power. Relationships between variables were evaluated using Spearman’s ρ. We also calculated S values by negative log transformation base-2 of P values. Results Of 516 retrieved records, we included 87. The median fragility index was 1.5 (interquartile range: 0–4) and the median reverse fragility index was 3 (interquartile range: 2–4), although they were statistically not different (Mood’s test: χ2 = 0.557, df = 1, P = 0.4556). P values and fragility indices were strongly inversely correlated (ρ = − 0.71, 95% confidence interval: − 0.53 to − 0.85, P < 0.0001), while reverse fragility indices were moderately correlated to P values (ρ = 0.5, 95% confidence interval: 0.37–0.62, P < 0.0001). A fragility index of 1 resulted from P values between 0.039 and 0.003, which resulted in S values between 4 and 8. Conclusions Fragility indices, reverse fragility indices, and their respective fragility quotients of paediatric surgical trials are low. The fragility index can be viewed as no more than a transformed P value with even more substantial limitations. Its inherent penalisation of small studies irrespective of their clinical relevance is particularly harmful for paediatric surgery. Consequently, the fragility index should be avoided.

show abstract

Fragility index of network meta-analysis with application to smoking cessation data

Cited by 11 publications

References 77 publications

Meta-analyses in paediatric surgery are often fragile: implications and consequences

Meta-analyses in paediatric surgery are often fragile: implications and consequences

Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package

Paediatric surgical trials, their fragility index, and why to avoid using it to evaluate results

Contact Info

Product

Resources

About