A nonparametric approach to high-dimensional k-sample comparison problems

Abstract

Summary High-dimensional $k$-sample comparison is a common task in applications. We construct a class of easy-to-implement distribution-free tests based on new nonparametric tools and unexplored connections with spectral graph theory. The test is shown to have various desirable properties and a characteristic exploratory flavour that has practical consequences for statistical modelling. Numerical examples show that the proposed method works surprisingly well across a broad range of realistic situations.

Nonparametric universal copula modeling

“…Construct the LP‐polynomial basis { T j ( X ; F X )} j ≥1 for the Hilbert space by applying Gram‐Schmidt orthonormalization (see Appendix A for more details) on the set of functions of the power of T 1 ( X ; F X ): LP‐bases obey the following orthonormality conditions with respect to the measure F : For data analysis, construct the empirical LP basis (in short eLP basis) , where m is strictly less than the number of unique values in the sample { X 1 ,…, X n }. Note that our custom‐constructed basis functions are orthonormal polynomials of mid‐rank transform—for more details see Mukhopadhyay, Mukhopadhyay and Wang, and Mukhopadhyay and Parzen . LP‐orthonormal system plays a fundamental role in construct...…”

Breiman's "Two Cultures'' Revisited and Reconciled

“…One can also compute pvalues using the χ 2 m null distribution of qDIV. For more details see Mukhopadhyay and Wang (2020).…”

“…As a result, one can expand d x pF Y pyqq in the orthonormal basis of F Y pyq. One such orthonormal system is the LP-family of rank-polynomials (Mukhopadhyay, 2017a, Bruce et al, 2019, Mukhopadhyay and Wang, 2020a), which we denote tT j py; F Y qu to emphasize that they are polynomials of F Y pY q not Y , hence extremely robust. As the true F Y is unknown, we will instead use the empirical LP-bases (eLP) tT j py; r F Y qu for our data analysis.…”

Informative goodness-of-fit for multivariate distributions

“…To prove Corollary 4.1, we need to show that there exists a value D α for whichP M K=1 D (K) > D α ∩ K * = K H 0 ≤ α, (B.22) with D (K) = K (k)=1 θ 2 (k)and the event {K * = K} indicates that K is the value selected by the AIC or BIC procedure in (4.7). The left-hand-side of (B 22). is the probability that at least one model leads to incorrectly reject H 0 when selected (while accounting for the selection probability).…”

“…http://fermi.gsfc.nasa.gov/ssc/data/analysis/software3 In the LP acronym, the letter L typically denotes nonparametric methods based on quantiles, whereas P stands for polynomials[22, Supp S1].…”