A new measure of the effective number of tests, a practical tool for comparing families of non‐independent significance tests

Abstract

p-Values from tests of significance can be combined using the Sidák correction (or the closely related Bonferroni correction) or Fisher's method, but both these methods require that the p-values combined be independent when all null hypotheses tested are true. In this paper adjustments to these methods are proposed, using a new eigenvalue-based measure of the effective number of independent tests to which the actual tests performed are equivalent, and are compared with adjustments proposed by previous authors. The adjusted methods are evaluated using a sample of 726 Alzheimer's disease (AD) cases and 707 group-matched controls, genotyped at 84,975 single-nucleotide polymorphism loci in 2,000 randomly chosen genes. The tests for genetic association with AD at loci within each gene are combined. The number of loci tested per gene varies from 2 to 994. The adjusted combined p-values agree well with the significance of the combined p-values determined empirically by random permutation of the data (Sidák correction: r=0.990; Fisher's method: r=0.994). This indicates that the combined p-values can be used to assess the relative strength of evidence for association of these genes with AD. The adjustment proposed here is a refinement of that of Nyholt ([2004] Am. J. Hum. Genet. 74:765-769), giving improved agreement with the results of random permutation. The improvement obtained is similar to that given by the refinement proposed by Li and Ji ([2005] Heredity 95:221-227). It is concluded that the concept of an effective number of tests is a valid approximation that allows p-values to be combined in a highly informative way.

Factors affecting the effective number of tests in genetic association studies: a comparative study of three PCA-based methods

“…We have examined the performance of three common PCA-based formulas separately proposed by Cheverud 16 , Li and Ji 18 and Galwey 20 for deriving the effective number of tests in GAS. Considering the influences of LD measurements, the estimated effective numbers according to three formulas are indeed very different.…”

The SAM, Not the Electrodes, Dominates Charge Transport in Metal-Monolayer//Ga2O3/Gallium–Indium Eutectic Junctions

“…In order to answer this question, we used a multiple comparison test, which compares the distributions of log|r| for all compounds simultaneously, using a procedure similar to a t-test. 42,46,47 The multiple comparison test shows (at the 95% confidence level) whether, for example, the value of R for S(CH 2 ) 11 Fc is statistically distinguishable from the values of R for other compounds. Figure 6 graphically represents the results of the multiple comparison test.…”

Genetic Determinants of Statin-Induced Low-Density Lipoprotein Cholesterol Reduction

“…In the locus-wide analysis, probability values were corrected for multiple hypothesis testing with the Sidák procedure applied to an effective number of independent SNPs (M eff ) as described. 12 Additional analytic and graphical procedures, including the regional plots, 13 were performed in R. 14 Genetic risk scores were computed as the simple sum of inherited alleles associated with greater LDL-C reduction. Separate genetic risk scores were constructed for analysis of absolute and fractional LDL-C reduction.…”