Heavy-tailed distribution for combining dependent $p$-values with asymptotic robustness

Abstract

The issue of combining individual p-values to aggregate multiple small effects is prevalent in many scientific investigations and is a long-standing statistical topic. Many classical methods are designed for combining independent and frequent signals in a traditional meta-analysis sense using the sum of transformed p-values with the transformation of light-tailed distributions, in which Fisher's method and Stouffer's method are the most well-known. Since the early 2000, advances in big data promoted methods to aggregate independent, sparse and weak signals, such as the renowned higher criticism and Berk-Jones tests. Recently, Liu and Xie (2020) and Wilson (2019) independently proposed Cauchy and harmonic mean combination tests to robustly combine p-values under "arbitrary" dependency structure, where a notable application is to combine p-values from a set of often correlated SNPs in genome-wide association studies. The proposed tests are the transformation of heavy-tailed distributions for improved power with the sparse signal. It calls for a natural question to investigate heavy-tailed distribution transformation, to understand the connection among existing methods, and to explore the conditions for a method to possess robustness to dependency. In this paper, we investigate the regularly varying distribution, which is a rich family of heavy-tailed distribution and includes Pareto distribution as a special case. We show that only an equivalent class of Cauchy and harmonic mean tests have sufficient robustness to dependency in a practical sense. We also show an issue caused by large negative penalty in the Cauchy method and propose a simple, yet practical modification. Finally, we present simulations and apply to a neuroticism GWAS application to verify the discovered theoretical insights and provide practical guidance.

The Lévy combination test

“…Stable distributions vary in their suitability for the important practical applications of combined testing. Distributions with left and right heavy tails, including the CCT [13,15], exhibit sensitivity to p-values near 1 which is a serious limitation in many practical scenarios. The class of extremal Stable distributions (skewness parameter |β| = 1) that have just one heavy tail therefore appear more suitable.…”

“…Insensitivity to p-values near 1. Despite its advantages over the HMP procedure in terms of its convenient formula and exactness for any number of constituent p-values, the CCT suffers the drawback of undesirable sensitivity to p-values at or near 1 [12,13,15]. Unfortunately this is probably a fatal flaw for the elegant CCT in many settings because generally p-values are defined conservatively such that Pr(p ≤ α|H 0 ) ≤ α; they are said to be 'superuniform' [16].…”

“…One solution has proposed the use of a truncated Cauchy distribution to avoid problems with p-values near 1 [13]. However, the resulting distribution of this test tends to the Landau distribution, making the proposed test asymptotically equivalent to the HMP procedure, which is more interpretable.…”

“…Stable distributions is they exhibit robustness to dependence between the constituent p-values despite requiring no knowledge of the explicit dependence structure and despite relying on an independence assumption in their derivation [1,2,10,11,13,15,22,23].…”

“…Like the HMP, the CCT builds in some robustness to dependence despite employing an independence assumption in its derivation [10]. For small values of • p {1:L} and p T {1:L} , the HMP and CCT are approximately equivalent [11,12,13], although there are important differences. Taken together, the HMP and CCT show that heavy-tailed Stable distributions have special advantages for combining hypothesis tests that are relevant to large-scale data analysis.…”