Ultrahigh-Dimensional Robust and Efficient Sparse Regression Using Non-Concave Penalized Density Power Divergence

A robust variable screening procedure for ultra-high dimensional data

“…39 In particular, under appropriate assumptions, we have proved the following asymptotic properties of the DPD-SIS.(R1) At the population level, for any j=1,…,p, the j -th marginal MDPDE functional corresponding to the j -th covariate X j is zero if and only if X j is uncorrelated with the response variable.(R2) For some optimally chosen convergent sequence Rn→0, we have for some constants κ>0 and λ>0. The first result provides the sure screening property of the DPD-SIS, whereas the second one proves its control of the false-discovery rate.(R3) Combining above results in (R2), for any α≥0, we have Ptrue(scriptMtrue^=scriptM0true)=1−o(1), i.e., DPD-SIS with any given α≥0 satisfies the model selection consistency. Finally, based on the above sure screening property of the DPD-SIS, and the consistency of the DPD-based penalized regression estimators from Ghosh and Majumdar, 24 we can easily argue the consistency of the final estimator obtained by the proposed DPD-SIS Algorithm 1. A more general result in this regard is presented in the following theorem, justifying the use of the DPD-SIS Algorithm 1 for ultra-high dimensional linear regression problems. Theorem A.1 Assume the conditions required for Results (R2) and the conditions of Theorem 4 of Ghosh and Majumdar [ 24 ] hold true for a given α≥…”

“…Set r k = j , if |trueβ^jMα| has rank k , for k=1,…,p. Construct the estimated model set scriptMtrue^α(d)={r1,…,rd}, with indices corresponding to the top d values of (absolute) marginal MDPDEs. Run a robust penalized regression model (low or moderate dimensional) with the covariates selected in scriptMtrue^α(d) to obtain an estimated coefficient vector, say bold-italicβtrue^d=(trueβ^d0,trueβ^dr1,…,trueβ^drd)T. (We suggest to use the DPD-based method of Ghosh and Majumdar 24 with the same α , which also gives an estimate trueσ^2 of the overall model error variance σ2. ) Output: The final estimated model …”

“…Run a robust penalized regression model (low or moderate dimensional) with the covariates selected in scriptMtrue^α(d) to obtain an estimated coefficient vector, say bold-italicβtrue^d=(trueβ^d0,trueβ^dr1,…,trueβ^drd)T. (We suggest to use the DPD-based method of Ghosh and Majumdar 24 with the same α , which also gives an estimate trueσ^2 of the overall model error variance σ2. )…”

“…Second, in Step 4 of DPD-ISIS, any fast robust penalized regression method, like RLARS, may be used without hampering the basic structure of DPD-ISIS. However, we strongly suggest to use the DPD-based penalized regression method of Ghosh and Majumdar 24 with the same α in Step 8 to obtain the final model; as in DPD-SIS, it makes the whole procedure structurally consistent and also provides an estimate trueσ^2 of the overall error (unexplained) variance σ2 in our final model.…”

“…As we have relatively few subjects, outliers might have a profound effect on the result, and hence, we are interested in performing a robust variable screening. Our analysis strategy is as follows: We will perform three iterations of the proposed robust DPD-ISIS (Algorithm 2) with RLARS in each iteration (Step 4), followed by a robust L 1 -penalized regression, the DPD-LASSO method of Ghosh and Majumdar 24 to be consistent (in Step 8), to produce our list of selected genes. In each iteration of DPD-ISIS we select the d=n/log⁡(n) ≈ 13 top variables, while we use penalization parameter λ=log⁡(p)/n in the final DPD-LASSO.…”