Meta-analyses in paediatric surgery are often fragile: implications and consequences

Abstract

Purpose Meta-analyses occupy the highest level of evidence and thereby guide clinical decision-making. Recently, randomised-controlled trials were evaluated for the robustness of their findings by calculating the fragility index. The fragility index is the number of events that needs to be added to one treatment arm until the statistical significance collapses. We, therefore, aimed to evaluate the robustness of paediatric surgical meta-analyses. Methods We searched MEDLINE for paediatric surgical meta-analyses in the last decade. All meta-analyses on a paediatric surgical condition were eligible for analysis if they based their conclusion on a statistically significant meta-analysis. Results We screened 303 records and conducted a full-text evaluation of 60 manuscripts. Of them, 39 were included in our analysis that conducted 79 individual meta-analyses with significant results. Median fragility index was 5 (Q25–Q75% 2–11). Median fragility in relation to included patients was 0.77% (Q25–Q75% 0.29–1.87%). Conclusion Paediatric surgical meta-analyses are often fragile. In almost 60% of results, the statistical significance depends on less than 1% of the included population. However, as the fragility index is just a transformation of the P value, it basically conveys the same information in a different format. It therefore should be avoided.

Fragility Index of Recently Published Meta-Analyses in Pediatric Urology: A Striking Observation

“…Consistent results were observed by Schröder et al [5], depicting a median (Q1-Q3; range) FI of 5 (2-11; 1-50) among the meta-analyses related to pediatric surgery. The authors have very aptly discussed the rarity of the pediatric surgical ailments as the possible reason behind the low FI among these meta-analyses.…”

“…Later, Atal et al demonstrated its application for meta-analyses of RCTs [4]. A recent study by Schröder et al [5] has shown its wider applicability and has supported its use for meta-analyses of observational studies also. Although the application of FI for meta-analyses of non-RCTs is a point of debate, the technical challenges of conducting good-quality RCTs in surgical specialties are even greater.…”

The fragility index may not be ideal for paediatric surgical conditions: the example of foetal endoscopic tracheal occlusion

“…We truly appreciate that Saikia & Thakuria have found the time during this devastating pandemic to respond to our manuscript. It is a pleasure for us to respond to their comments to the editor in this issue of Paediatric Surgery International [ 1 ], which provides us with the valuable opportunity to point out some issues that may not have become clear enough in our initial contribution [ 2 ].…”

“…The fragility index has been extended beyond this limitation before [ 8 – 10 ], and we have thus extended its applicability as well. Let us assume that we would have limited our analysis [ 2 ] to meta-analyses with these primary study properties: the underlying numbers would have possibly changed slightly, but it would not have changed our conclusion.…”

Fragility index of meta-analyses in paediatric surgery

“…To the editor, We are doing a few projects on fragility index of randomized controlled trials (RCTs) and meta-analyses. Therefore, we read the work by Arne Schröder and colleagues with great enthusiasm [1]. Ignacio Atal and colleagues described the fragility index of meta-analyses that include only RCTs reporting dichotomous outcome [2].…”