Effect Size Measures and Their Relationships in Stroke Studies

Abstract

Many articles in Stroke have considered good statistical practice for adequate planning and high-power analysis for stroke trials. They have discussed which test may be adequate and powerful, proposals for an effect size measure, and proposals for defining number needed to treat (NNT) based on an ordinal scale (see online-only Data Supplement for citations).The clinical problem is straightforward. We read results of trials that fall into 3 categories: unequivocally neutral or even negative; overwhelmingly positive; or encouraging but open to various interpretations according to the approach taken to statistical analysis and presentation of findings. This dependence on methodology for the third group undermines our confidence in effective treatments and can prompt unjustified repetition of trials of ineffective treatments. A robust, powerful, and universal statistical approach is required.The statistical problem is more complex. From dozens of available statistical tests, each may be uniquely powerful in certain circumstances. However, trials intended as confirmatory for regulatory approval or for the use in clinical guidelines demand that the analysis plan be prespecified so that the test of choice should minimize assumptions yet maximize power for the anticipated difference between treatment groups. Furthermore, it is not sufficient to indicate that 1 treatment is significantly different from another: clinical research guidelines require that the magnitude of the treatment effect should be declared using the so-called effect size measure accompanied by its measure of precision, the confidence interval.Thus, there is need for a robust test, preferably for all data types-binary, ordinal, or continuous-and a test-related effect size measure, with a confidence interval that matches the test-related P value. This requirement for an adequate analysis of study data fortunately restricts the plethora of available tests to a small number of useful candidates.Here, we describe and explain the relationships between 2 tests, of which 1 is well-known and the other is less familiar. Both are suitable for the analysis of binary, ordinal, and continuous data, and both offer associated confidence intervals. Using intravenous thrombolysis trial data, we can illustrate the relative merits of these tests when compared with other approaches. We will demonstrate why dichotomizing ordinal scales are undesirable. Finally, we can show that in principle each well-known effect size measure can be reexpressed as the popular NNT. Family of Mann-Whitney MeasuresOne family of effect size measures fulfills both desirable criteria: robustness, that is minimizing assumptions, and relevance for clinical research in which medicinal products are tested against a reference treatment. This family is based on the concept of proversions. Two groups are contrasted by comparing each patient of 1 group with each member of the other and counting the number of cases for which there is superiority for 1 or the other group. The number of proversions for the ...

A collaborative sequential meta-analysis of individual patient data from randomized trials of endovascular therapy and tPA vs. tPA alone for acute ischemic stroke: ThRombEctomy And tPA (TREAT) analysis: statistical analysis plan for a sequential meta-analysis performed within the VISTA-Endovascular collaboration

“…Certain preplanned secondary analyses of outcome measured by the mRS will be considered. An analysis of the distribution of mRS using the ‘sliding dichotomy’ approach . An analysis of the distribution of mRS using the approach presented by Rahlfs et al . based on the Mann–Whitney U ‐test will be considered. This will be stratified by trial to account for between‐trial heterogeneity. …”

Serial Alberta Stroke Program Early Computed Tomography Score From Baseline to 24 Hours in SWIFT

“…With known wide variation in baseline status of patients with stroke and among outcomes, it may be more powerful statistically and more relevant clinically to consider the full range of disability, using ordinal approaches to both measures rather than arbitrary dichotomizations. [10][11][12][13] The authors were surprised that infarct progression was seen in patients with reperfusion, despite absence of hemorrhagic transformation. Two factors may contribute to this finding.…”

Confounding adjustment performance of ordinal analysis methods in stroke studies

“…For comparison purposes, we converted the odds ratios (ORs) generated by the OLR to Mann-Whitney measures of superiority, with the following approximation formula: MWS = (OR/ (OR-1) 2 ) x ((OR-1)-ln(OR)). [6] Outcome parameters…”

“…Because OLR is based on several assumptions, such as a linear and proportional effect of the independent variables on the outcome variable, [5] the Mann-Whitney measure of superiority (MWS) was recently proposed as a more robust analysis method of an ordinal outcome scale. [6,7] In contrast to regression methods, the MWS is a nonparametric rank-based test based on proversions, which are one to one comparisons of outcome between observations. In short, each observation in one group (A) is compared to each observation in the other group (B), and the following three complementary probabilities (Ps) are derived: P(A>B), P(A = B), and P(B>A).…”