Consonance and the Closure Method in Multiple Testing

Abstract

Consider the problem of testing s null hypotheses simultaneously. In order to deal with the multiplicity problem, the classical approach is to restrict attention to multiple testing procedures that control the familywise error rate (FWE). The closure method of Marcus et al. (1976) reduces the problem of constructing such procedures to one of constructing single tests that control the usual probability of a Type 1 error. It was shown by Sonnemann ( , 2008) that any coherent multiple testing procedure can be constructed using the closure method. Moreover, it was shown by Sonnemann and Finner (1988) that any incoherent multiple testing procedure can be replaced by a coherent multiple testing procedure which is at least as good. In this paper, we first show an analogous result for dissonant and consonant multiple testing procedures. We show further that, in many cases, the improvement of the consonant multiple testing procedure over the dissonant multiple testing procedure may in fact be strict in the sense that it has strictly greater probability of detecting a false null hypothesis while still maintaining control of the FWE. Finally, we show how consonance can be used in the construction of some optimal maximin multiple testing procedures. This last result is especially of interest because there are very few results on optimality in the multiple testing literature.

Closed Testing in Pharmaceutical Research: Historical and Recent Developments

“…Their global maximin consonant test is $T={\sum }_{i=1}^m\cosh (\epsilon \mid X_i\mid )$, where cosh(·) is the hyperbolic cosine function. Interestingly, Romano et al (2011) note that for large ε , their test nearly reduces to the ordinary Bonferroni global test, which supports our general conclusion that MINP tests are far better suited to closure than AC methods.…”

“…Our simulation study enhances the theory in Romano et al (2011) by showing empirically how dramatic and pervasive the loss of power can be when using dissonant (i.e., non-consonant) tests in the closure framework. The only situation that allows the dissonant AC tests to perform reasonably well is the 100% alternative case, which is quite restrictive.…”

“…Romano et al (2011) show that power inheritance is possible but not automatic. More precisely, if an optimal global test of an intersection hypothesis H 1 exists and results in a consonant multiple testing procedure, then that multiple testing procedure will have maximum power to reject at least one false null hypothesis among all procedures controlling the FWER at α .…”

“…Birnbaum, 1954; Bhattacharya, 1961; Berk and Cohen, 1979; Marden, 1982, 1985; Westberg, 1985; Loughin, 2004), but optimality results for these tests when employed in closed testing are less well known (Romano et al, 2011). We fill this gap by evaluating the power of several PVCTs in the closure setting and comparing the results to the power of the tests when they are used to test "global" (intersection) null hypotheses, which has been their traditional application.…”

“…Romano et al (2011) go on to define a test that has this desirable power inheritance property for the problem of testing several normal means, that is, $${\mathrm{boldH}}_0:\theta =0_{m\times 1}\mathrm{vs}.{\mathrm{boldH}}_A:\theta \ne 0_{m\times 1}.$$…”