A two-sample test for high-dimensional data with applications to gene-set testing

Abstract

We propose a two-sample test for the means of high-dimensional data when the data dimension is much larger than the sample size. Hotelling's classical T 2 test does not work for this "large p, small n" situation. The proposed test does not require explicit conditions in the relationship between the data dimension and sample size. This offers much flexibility in analyzing high-dimensional data. An application of the proposed test is in testing significance for sets of genes which we demonstrate in an empirical study on a leukemia data set.

Random matrix theory in statistics: A review

“…To fix this problem, tests based on certain quadratic functionals of the sample covariance and correlation matrices have been proposed by Chen and Qin (2010) and Srivastava and Du (2008). In contrast, proposed a regularized Hotelling's T 2 statistic: RHTðλÞ≔nX n ðS n þλI p Þ À 1 X n , which is well-defined for all λ 4 0.…”

A review of 20 years of naive tests of significance for high-dimensional mean vectors and covariance matrices

“…Chen and Qin (2010) [14] argued that the main term of Bai and Saranadasa's ANT contains squared terms of sample vectors that may cause non-robustness of the test statistic against outliers and thus proposed an unbiased estimator of the target function µ 1 − µ 2 2 , given by…”

“…The expression of the asymptotic power under the condition (2.15) ((3.5) in Chen and Qin [14]) may contain an error in that the denominator of the quantity inside the function Φ should be σ n2 in Chen and Qin's notation, that is, 2…”

“…In subsection 2.1, we introduce the findings of Bai and Saranadasa [3]. In subsection 2.2, we introduce Chen and Qin [14]'s test based on the unbiased estimator of the target function. In subsection 2.3, we review Srivastava and Du's work on the scale invariant NTM, based on the modified component-wise squared Mahalanobis distance.…”

High-Dimensional Two-Sample Covariance Matrix Testing via Super-Diagonals

“…The studies on the effects of high dimensionality on the classical Hotelling's test for the means (Hotelling, 1931) serve as an excellent example on how high dimensional scrutiny of a classical multivariate procedure can be carried out. This line of studies includes the work of Bai and Saranadasa (1996) for discovering the shortcomings of the Hotelling's test and a modification that removes the inverse of the sample covariance (S n ) from the original test statistic, while Srivastava and Du (2008) considered using the diagonal matrix of S n to replace S n , and Chen and Qin (2010) suggested using U-statistics. See also Cai, Liu and Xia (2014) for a test based on the maximal norm and Hall and Jin (2009) that utilizes the dependence to enhance the signal strength of the testing problem.…”