Kernel Partial Correlation Coefficient -- a Measure of Conditional Dependence

Abstract

In this paper we propose and study a class of simple, nonparametric, yet interpretable measures of conditional dependence between two random variables Y and Z given a third variable X, all taking values in general topological spaces. The population version of any of these nonparametric measures -defined using the theory of reproducing kernel Hilbert spaces (RKHSs) -captures the strength of conditional dependence and it is 0 if and only if Y and Z are conditionally independent given X, and 1 if and only if Y is a measurable function of Z and X. Thus, our measure -which we call kernel partial correlation (KPC) coefficient -can be thought of as a nonparametric generalization of the classical partial correlation coefficient that possesses the above properties when (X, Y, Z) is jointly normal. We describe two consistent methods of estimating KPC. Our first method of estimation is graph-based and utilizes the general framework of geometric graphs, including K-nearest neighbor graphs and minimum spanning trees. A sub-class of these estimators can be computed in near linear time and converges at a rate that automatically adapts to the intrinsic dimensionality of the underlying distribution(s). Our second strategy involves direct estimation of conditional mean embeddings using cross-covariance operators in the RKHS framework. Using these empirical measures we develop forward stepwise (high-dimensional) nonlinear variable selection algorithms. We show that our algorithm, using the graph-based estimator, yields a provably consistent model-free variable selection procedure, even in the high-dimensional regime when the number of covariates grows exponentially with the sample size, under suitable sparsity assumptions. Extensive simulation and real-data examples illustrate the superior performance of our methods compared to existing procedures. The recent conditional dependence measure proposed by Azadkia and Chatterjee [5] can also be viewed as a special case of our general framework.

Causal Domain Adaptation with Copula Entropy based Conditional Independence Test

“…Conditional independence testing with CE is a informationtheoretical based method. It is mathematically sound and hence advantageous to other similar tools for testing CI, such as partial correlation, conditional distance correlation [11], kernel-based conditional independence tests [12], conditional dependence coefficient [13], generalised covariance measure [14], and kernel partial correlation [15], etc. Mooij et al [10] applied causal structure learning algorithm to find the causal relationship.…”

“…Conditional Independence (CI) test is the basic building block of causal discovery methods. There are many non-parametric methods for such testing, such as conditional distance correlation [11], kernel-based conditional independence tests [12], conditional dependence coefficient [13], generalised covariance measure [14], and the basic and kernel partial correlation [15], etc.…”

Local permutation tests for conditional independence

“…Strobl et al (2019) use random Fourier features to approximate kernel computations, and propose a more computationally efficient version of the test of Zhang et al (2012). Other CI measures proposed by Doran et al (2014) and Huang et al (2020) are motivated by the kernel maximum mean discrepancy for two-sample testing (Gretton et al, 2012). In particular, the CI measure introduced by Huang et al (2020) compares whether Y |X, Z and Y |X have the same distribution, and their measure can be viewed as a kernelized version of the CI measure of Azadkia and Chatterjee (2019).…”

“…Other CI measures proposed by Doran et al (2014) and Huang et al (2020) are motivated by the kernel maximum mean discrepancy for two-sample testing (Gretton et al, 2012). In particular, the CI measure introduced by Huang et al (2020) compares whether Y |X, Z and Y |X have the same distribution, and their measure can be viewed as a kernelized version of the CI measure of Azadkia and Chatterjee (2019). Recently, Sheng and Sriperumbudur (2019) and Park and Muandet (2020) propose kernel CI measures that are closely connected to the Hillbert-Schmidt independence criterion (Gretton et al, 2005).…”

Exact Detection Thresholds for Chatterjee's Correlation

“…In particular, it yields an exactly distribution-free test of independence which is consistent against all fixed alternatives, computable in (near) linear time and has a simple null distribution theory. Consequently, it has attracted much attention over the past year both in terms of applications and theory (see [7,35,12,2,24,18,11]). Despite its gaining popularity, not much is known theoretically about the power of Chatterjee's test of independence beyond consistency against fixed alternatives.…”