Testing One Hypothesis Multiple Times: The Multidimensional Case

Abstract

The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models, are among the most challenging problems in statistical practice. These challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative, and a computationally efficient solution can be obtained by the "Testing One Hypothesis Multiple times" (TOHM) method. In the one-dimensional setting, a fine discretization of the space of the non-identifiable parameter is specified, and a global p-value is obtained by approximating the distribution of the supremum of the resulting stochastic process. In this paper, we propose a computationally efficient inferential tool to perform TOHM in the multidimensional setting. Here, the approximations of interest typically involve the expected Euler Characteristics (EC) of the excursion set of the underlying random field. We introduce a simple algorithm to compute the EC in multiple dimensions and for arbitrarily large significance levels. This leads to an highly generalizable computational tool to perform hypothesis testing under non-standard regularity conditions.

Combined collider constraints on neutralinos and charginos

“…Approximate procedures can be applied in lower dimensions, e.g. [145][146][147], however, they are are mainly aimed at reducing the number of pseudodata realisations that are required to perform the trial correction, which is not the issue here. Our problem is instead that obtaining sufficiently good sampling of the possible signal predictions in the EWMSSM is hard.…”

Testing One Hypothesis Multiple Times

“…Finally, we investigate the relationship between the TOHM and Bonferroni bounds and we implement both procedures on our running examples. Extensions of our results to the case where the nuisance parameter specified only under the alternative, θ, is multi-dimensional are the subject of a forthcoming paper (Algeri and van Dyk, 2018).…”

“…Since both tests are of the form in (1.2), the test is on the boundary of the parameter space and for each θ fixed the asymptotic distribution under H 0 is a mixture of χ 2 1 and zero (Chernoff, 1954;Self and Liang, 1987), also known asχ-distribution and which we dentote withχ 2 01 . It can be shown (Algeri and van Dyk, 2018) that in this setting the bound in (2.8) has the same form as in the χ 2 1 case, i.e., it is given by (1.5) with s = 1. In Example 3, we use the signed-root of the LRT Q n (θ) = sign(η θ − η 0 ) T n (θ), hence the sub-tests statistics are asymptotically normally distributed under H 0 (e.g., Davies, 1977).…”

“…The bound in (1.5) and the analogous bounds for Gaussian and related processes such as F and t-processes, can be derived using results of random fields theory as discussed in Algeri and van Dyk (2018). In this setting, it can be shown that, under mild smoothness conditions (see Taylor and Adler (2003, p. 547 corresponds to the so-called Lipschitz-Killing curvature of first order (e.g., Adler and Taylor, 2009) and is typically difficult to compute.…”

Informative goodness-of-fit for multivariate distributions

“…In direct searches for dark matter, the dominant background sources are neutron recoils which may produce signals mimicking those expected from dark matter candidates [e.g., 40]. As a toy example, suppose we are interested in assessing the validity of a given distribution for the nuclear recoil background specified over the energy region X = [5,20]KeV nr × [0, 17]KeV nr. Each observations in X corresponds to the scintillation of photons (X 1 ) and ionization electrons (X 2 ) [e.g., 7].…”