Trial Sequential Analysis in Critical Care Medicine: The Way Forward

Abstract

We congratulate Sharif et al (1) for their interesting systematic review and meta-analysis, published in a recent issue of Critical Care Medicine, of randomized clinical trials (RCTs) on probiotics in critical illness. They included 65 RCTs among adults and pediatric critically ill patients (n = 8,483) and concluded that probiotics may reduce ventilatorassociated pneumonia (VAP) with low certainty evidence.In their review, the authors used a technique: trial sequential analysis (TSA). TSA is a valuable technique that can assist in conventional meta-analysis as it controls type-I and type-II errors. Specifically, conventional meta-analyses are regularly updated with new trials, but this inevitably inflates type-I errors due to multiple retesting of data (2). TSA considers the latter, estimates the diversity-adjusted required information size (DARIS) for statistical inference, and constructs thresholds for statistical significance and futility (2, 3). This provides more reliable results than conventional meta-analysis (3).For the TSA on VAP, the authors used the following variables: event proportion in the control group 24.8%, relative risk reduction (RRR) 10%, alpha 5%; beta 20%, and heterogeneity adjusted by the diversity estimate. They concluded that the DARIS (n = 2212 participants) was achieved as a total of 4,738 participants were accrued in the RCTs.The relatively small RRR of 10% and the high diversity of 75% usually results in a large DARIS in most TSA. Therefore, we used the TSA software (0.9.5.10 Beta, The Copenhagen Trial Unit, Copenhagen, Denmark) and executed a TSA according to the variables outlined in their supplementary material in (1) to replicate their results. However, we are unable to derive a DARIS of 2212. Instead, we derived a DARIS of 36,329 (Supplementary Fig. 1, Supplemental Digital Content, http://links.lww.com/CCM/H184). The corollary of this large DARIS is that the conclusion of the systematic review may be further weakened as the DARIS was not reached (only 13% was achieved). The cumulative Z-curve did not cross the TSA boundaries for benefit, futility, or harm. Accordingly, the effect reported should likely be assessed at "very low certainty" according to the Grading Recommendations Assessment, Development, and Evaluation approach.It is commendable that the systematic review exhibits some best practices that future TSAs for dichotomous outcomes should take after. The following vital variables were outlined: 1) proportion of events in the control group, 2) the

Intravenous vitamin C monotherapy in critically ill patients: a systematic review and meta-analysis of randomized controlled trials with trial sequential analysis

“…To control for type-I [31] and type-II errors and further confirming the results of our meta-analysis, TSA for overall mortality was performed with the following parameters [32]: alpha 5%, beta 10%, and the DerSimonian-Laird random effect model. Between-trial heterogeneity was adjusted by the diversity-estimate (D 2 ).…”

The authors reply

“…We thank Lew et al (1) for their feedback on our recently published systematic review and meta-analysis in Critical Care Medicine comparing the safety and efficacy of probiotics versus placebo in critically ill patients (2). After careful review, there is an error in the published trial sequential analysis (TSA) for the ventilator-associated pneumonia (VAP) outcome.…”