Conditional quantile screening in ultrahigh-dimensional heterogeneous data

Wu, Yuanshan; Yin, Guosheng

doi:10.1093/biomet/asu068

Cited by 80 publications

(43 citation statements)

References 20 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Its good numerical performance and novel theoretical properties have made SIS popular in ultrahigh dimensional analysis. As a result, SIS and its extensions have been applied to many important settings including generalized linear model (Fan and Song, 2010), multi-index semi-parametric models (Zhu et al, 2011), nonparametric regression (Fan et al, 2011; Liu et al, 2014), quantile regression (He et al, 2013; Wu and Yin, 2015) and so forth.…”

Section: Introductionmentioning

confidence: 99%

Variable Screening via Quantile Partial Correlation

Tsai

2017

Journal of the American Statistical Association

View full text Add to dashboard Cite

In quantile linear regression with ultra-high dimensional data, we propose an algorithm for screening all candidate variables and subsequently selecting relevant predictors. Specifically, we first employ quantile partial correlation for screening, and then we apply the extended Bayesian information criterion (EBIC) for best subset selection. Our proposed method can successfully select predictors when the variables are highly correlated, and it can also identify variables that make a contribution to the conditional quantiles but are marginally uncorrelated or weakly correlated with the response. Theoretical results show that the proposed algorithm can yield the sure screening set. By controlling the false selection rate, model selection consistency can be achieved theoretically. In practice, we proposed using EBIC for best subset selection so that the resulting model is screening consistent. Simulation studies demonstrate that the proposed algorithm performs well, and an empirical example is presented.

show abstract

Section: Introductionmentioning

confidence: 99%

Variable Screening via Quantile Partial Correlation

Tsai

2017

Journal of the American Statistical Association

View full text Add to dashboard Cite

show abstract

“…This result implies that the CC-SIS can select all the truly active predictors with an overwhelming probability. The dimensionality can be as high as p n = o(exp(n 1−2κ )), similar to other model-free feature screening methods (see Li et al (2012a) and Wu and Yin (2015) for example). Moreover, our result requires less condition on both the predictors and the response due to the nonparametric nature.…”

Section: Cc-based Variable Screeningmentioning

confidence: 66%

Copula-based Partial Correlation Screening: a Joint and Robust Approach

Xia¹,

Li²

2021

STAT SINICA

View full text Add to dashboard Cite

Screening for ultrahigh dimensional features may encounter complicated issues such as outlying observations, heterogeneous or heavy-tailed distribution, multi-collinearity and confounding effects. Standard correlation-based marginal screening methods may be a weak solution to these issues. We contribute a novel robust joint screener to safeguard against outliers and distribution mis-specification for both the response variable and the covariates, and to account for external variables at the screening step. Specifically, we introduce a copula-based partial correlation (CPC) screener. We show that the empirical process of the estimated CPC converges weakly to a Gaussian process and establish the sure screening property for CPC screener under very mild technical conditions, where we need not require any moment condition, weaker than existing alternatives in the literature. Moreover, our approach allows for a diverging number of conditional variables from the theoretical point of view. Extensive simulation studies 1 and two data applications are included to illustrate our proposal.

show abstract

“…For ultrahigh dimensional data, [12] considered the feature screening problem based on quantile regression and developed a nonparametric screening procedure. [13] proposed a conditional quantile screening procedure.…”

Section: Introductionmentioning

confidence: 99%

“…In order to deal with ultrahigh dimensional and heterogeneous survival data, [18] proposed rank-based independent screening method for survival data via weighted rank correlation. Using quantile regression technique, [12] proposed the inverse probability weighted approach to deal with censoring data and [13] proposed censored conditional quantile screening, which concentrated on the technique of redistribution of mass for censored observations. Our proposal in this paper can provide a new solution to survival screening and the performance is shown to be competitive with the existing approaches in our numerical analysis.…”

Section: Introductionmentioning

confidence: 99%

Conditional quantile correlation learning for ultrahigh dimensional varying coefficient models and its application in survival analysis

Xia¹,

Li²,

Fu³

2018

STAT SINICA

View full text Add to dashboard Cite

In this paper, we consider a robust approach to the ultrahigh dimensional variable screening under varying coefficient models. Different from the existing works focusing on the mean regression function, we propose a novel procedure based on the conditional quantile correlation sure independent screening (CQCSIS). This new proposal is applicable to heterogeneous or heavy-tailed data in general and is invariant to monotone transformation of the response. Furthermore, we generalize such a screening procedure to address censored lifetime data through inverse probability weighting. The CQCSIS can be easily implemented due to an application of nonparametric B-spline approxi-1 Statistica Sinica: Newly accepted Paper (accepted version subject to English editing) mation, and computed much faster than the kernel based screening method. Under some regularity conditions, we establish sure screening properties including screening consistency and ranking consistency for proposed approaches. In this paper we also attempt to construct a two-stage variable selection procedure for a further improvement of performance of CQCSIS based on a group SCAD penalization. Extensive simulation examples and real data applications are presented for illustration.

show abstract

Conditional quantile screening in ultrahigh-dimensional heterogeneous data

Cited by 80 publications

References 20 publications

Variable Screening via Quantile Partial Correlation

Variable Screening via Quantile Partial Correlation

Copula-based Partial Correlation Screening: a Joint and Robust Approach

Conditional quantile correlation learning for ultrahigh dimensional varying coefficient models and its application in survival analysis

Contact Info

Product

Resources

About