Covariate-adjusted Spearman's Rank Correlation with Probability-scale Residuals

Abstract

Summary It is desirable to adjust Spearman’s rank correlation for covariates, yet existing approaches have limitations. For example, the traditionally defined partial Spearman’s correlation does not have a sensible population parameter, and the conditional Spearman’s correlation defined with copulas cannot be easily generalized to discrete variables. We define population parameters for both partial and conditional Spearman’s correlation through concordance-discordance probabilities. The definitions are natural extensions of Spearman’s rank correlation in the presence of covariates and are general for any orderable random variables. We show that they can be neatly expressed using probability-scale residuals (PSRs). This connection allows us to derive simple estimators. Our partial estimator for Spearman’s correlation between X and Y adjusted for Z is the correlation of PSRs from models of X on Z and of Y on Z, which is analogous to the partial Pearson’s correlation derived as the correlation of observed-minus-expected residuals. Our conditional estimator is the conditional correlation of PSRs. We describe estimation and inference, and highlight the use of semiparametric cumulative probability models, which allow preservation of the rank-based nature of Spearman’s correlation. We conduct simulations to evaluate the performance of our estimators and compare them with other popular measures of association, demonstrating their robustness and efficiency. We illustrate our method in two applications, a biomarker study and a large survey.

Ensemble Classification Based on Feature Selection for Environmental Sound Recognition

“…But there still exists a problem that the result is not stable enough. As for the stability, three indices, Pearson's correlation coefficient [35], Spearman's rank correlation coefficient [36], and Tanimoto distance [37], are commonly used to assess the performance of stability thoroughly in recent years. In the future, further research will focus on the stability of csFs to obtain a better performance from the perspective of three indices.…”

Probability-Scale Residuals in HIV/AIDS Research: Diagnostics and Inference

“…In practice, we have used orm, introduced in Section 3.2, to obtain maximum likelihood estimates based on the semiparametric transformation model. Extensive simulations have shown that Spearman's partial correlation using PSRs performs remarkably well with orm, even when models are misspecified (e.g., orm with a cloglog, instead of probit, link is fit to normal data) (Liu et al, 2017).…”

“…If Z is categorical, then this correlation can simply be calculated in each level of Z. With continuous Z, the conditional rank correlation can be computed either nonparametrically with, for example, kernal smoothers, or modeled with parametric functions (Liu et al, 2017).…”

“…As a residual it can be used for model diagnostics and for inference. We have proposed to use the correlation of PSRs to adjust Spearman's rank correlation for covariates (Liu et al, 2017). The goal of this chapter is to introduce the PSR to HIV researchers, to describe a few of its important properties, and to demonstrate its utility across a diverse set of analyses with HIV data.…”

Discussion of ‘regularized regression for categorical data’

“…This approach results in a covariate-adjusted partial Spearman’s rank correlation between X and Y that incorporates the order information of these variables but does not assign arbitrary numbers to the categories; the null hypothesis of conditional independence can be tested. Some additional work has considered other outcome types (Liu et al, 2016; Shepherd et al, 2016). …”