An evaluation of methods for testing hypotheses relating to two endpoints in a single clinical trial

Abstract

The issues and dangers involved in testing multiple hypotheses are well recognised within the pharmaceutical industry. In reporting clinical trials, strenuous efforts are taken to avoid the inflation of type I error, with procedures such as the Bonferroni adjustment and its many elaborations and refinements being widely employed. Typically, such methods are conservative. They tend to be accurate if the multiple test statistics involved are mutually independent and achieve less than the type I error rate specified if these statistics are positively correlated. An alternative approach is to estimate the correlations between the test statistics and to perform a test that is conditional on those estimates being the true correlations. In this paper, we begin by assuming that test statistics are normally distributed and that their correlations are known. Under these circumstances, we explore several approaches to multiple testing, adapt them so that type I error is preserved exactly and then compare their powers over a range of true parameter values. For simplicity, the explorations are confined to the bivariate case. Having described the relative strengths and weaknesses of the approaches under study, we use simulation to assess the accuracy of the approximate theory developed when the correlations are estimated from the study data rather than being known in advance and when data are binary so that test statistics are only approximately normally distributed.

Optimal Multiple Testing and Design in Clinical Trials

“…They used Stouffer's local test (Stouffer et al, 1949). For the case of two endpoints, a comprehensive comparison of the suggestion in Bittman et al (2009) with the Bonferroni and Simes local tests is given by Su et al (2012). Rosenblum et al (2014), and Rosset et al (2022) considered finding the optimal procedure for a power objective of interest while controlling in the strong sense a desired error rate, without restrictions to a selected set of allowed procedures.…”

Decision-making with multiple correlated binary outcomes in clinical trials

“…When these models rely on large sample approximations for decision-making (such as methods presented by Whitehead et al., 17 Sozu et al., 5 , 16 and Su et al. 18 ; see for an exception Murray et al. 3 ), their applicability is limited, since the validity of z-tests for small samples may be inaccurate.…”

“…The majority of these alternatives assume a (latent) normally distributed continuous variable. When these models rely on large sample approximations for decision-making (such as methods presented by Whitehead et al, 17 Sozu et al, 5,16 and Su et al 18 ; see for an exception Murray et al 3 ), their applicability is limited, since the validity of z-tests for small samples may be inaccurate. A second class of alternatives uses copula models, which is a flexible approach to model dependencies between multiple univariate marginal distributions.…”

Bayesian multilevel multivariate logistic regression for superiority decision-making under observable treatment heterogeneity

“…Secondary outcome variables often contribute to treatment evaluation as well, but are given a co-primary status in the All and Any rules or are not formally included in the statistical decision procedure when the Single rule is used [ 17 , 45 ]. To handle outcomes that differ in relative importance, linear combinations of dependent variables with pre-assigned (importance) weights have been proposed as a flexible alternative [ 14 , 16 , 18 , 46 , 47 ]. We refer to a linear combination as a Compensatory rule, referring to its inherent mechanism that allows (weighted) positive and negative effects to compensate each other.…”