Viewpoint: taking into account risks of random errors when analysing multiple outcomes in systematic reviews

Abstract

The clinical significance and positive or negative implication of an e ect estimate should only be emphasised if the statistical assessment is conclusive beyond reasonable doubt.[1] Confidence intervals not containing 1.0 for a binary outcome or 0.0 for a continuous outcome, as well as the corresponding P values, are o en used as thresholds for statistical significance. Based on the fact that the 'true' multiplicity-adjusted thresholds for significance lie somewhere between the unadjusted threshold Viewpoint: taking into account risks of random errors when analysing multiple outcomes in systematic reviews (Editorial) 1

Lower vs. higher fluid volumes in sepsis—protocol for a systematic review with meta‐analysis

“…51,52 The required information size 52 will take into account the event proportion in the control group, the assumption of a plausible RRR, and the heterogeneity variance of the meta-analysis. 36,54,55 If there is only data available for one or two of the co-primary outcomes, we will consider a P < 0.05 or P < 0.033 as statistically significant, respectively. 36 In brief, as we have three co-primary outcomes and eight secondary outcomes, we will consider a P < 0.025 and P < 0.01 as statistically significant, respectively to limit the family wise error rate (FWER) to < 0.05.…”

Pharmacological interventions for delirium in intensive care patients: a protocol for an overview of reviews

“…We use three primary outcomes and, therefore, we will consider a P value of 0.025 or less as statistically significant analysing the primary outcomes [70, 81]. We use three secondary outcomes and, therefore, we will consider a P value of 0.025 or less as statistically significant analysing the secondary outcomes [70].…”

Network meta-analysis: the highest level of medical evidence?

“…All systematic reviews should rely on future updates to account for novel evidence. Moreover, such updates involve multiple tests as new evidence becomes available, which also lead to inflated type I errors 26. Recent attention toward concepts like ‘living systematic reviews’, or reviews that are updated as soon as new and relevant data present,27 makes it only more urgent to address multiplicity problems in network meta-analyses.…”