Minimax rates for sparse signal detection under correlation

Abstract

We fully characterize the nonasymptotic minimax separation rate for sparse signal detection in the Gaussian sequence model with p equicorrelated observations, generalizing a result of Collier, Comminges, and Tsybakov [7]. As a consequence of the rate characterization, we find that strong correlation is a blessing, moderate correlation is a curse, and weak correlation is irrelevant. Moreover, the threshold correlation level yielding a blessing exhibits phase transitions at the √ p and p − √ p sparsity levels. We also establish the emergence of new phase transitions in the minimax separation rate with a subtle dependence on the correlation level. Additionally, we study group structured correlations and derive the minimax separation rate in a model including multiple random effects.The group structure turns out to fundamentally change the detection problem from the equicorrelated case and different phenomena appear in the separation rate.

Sharp local minimax rates for goodness-of-fit testing in multivariate binomial and Poisson families and in multinomials

“…The bulk lower bound is close to that of [53]. The tail lower bound relies on a sparse prior that is an existing technique (for example, in sparse testing, see [10,23,42]) and is very different from the construction in [53]. Handling the indices I; A and U require careful manipulations that we believe are new techniques.…”

HILAMA: High-dimensional multi-omic mediation analysis with latent confounding

“…Currently, this correlation structure is omitted due to challenges in controlling FDR in large-scale regression with multivariate response. However, incorporating this information might be useful for removing the confounding effect and improving the power of the test [Zou et al, 2020, Kotekal andGao, 2021]. Finally, other methods of dealing with latent confounding can also be incorporated into HILAMA in its future version, such as the approaches [Miao et al, 2022, Tang et al, 2023 that directly leverage the majority rule [Kang et al, 2016] or the plurality rule [Guo et al, 2018].…”

Sparse Signal Detection in Heteroscedastic Gaussian Sequence Models: Sharp Minimax Rates

“…This comparison therefore proves that the testing problem (35) is either impossible or analogous to a Gaussian testing problem in L 1 separation, and that the correlation between the coordinates of X do not affect the minimax rates. Further interplays between correlation and sparsity in signal detection have been thoroughly discussed in [KG21], in the case of an isotropic covariance matrix and with Euclidean separation. The results of the present paper could therefore find natural applications to the local analog of Problem (35), which is left for future work.…”