Penalized Composite Quasi-Likelihood for Ultrahigh Dimensional Variable Selection

Abstract

Summary In high-dimensional model selection problems, penalized least-square approaches have been extensively used. This paper addresses the question of both robustness and efficiency of penalized model selection methods, and proposes a data-driven weighted linear combination of convex loss functions, together with weighted L1-penalty. It is completely data-adaptive and does not require prior knowledge of the error distribution. The weighted L1-penalty is used both to ensure the convexity of the penalty term and to ameliorate the bias caused by the L1-penalty. In the setting with dimensionality much larger than the sample size, we establish a strong oracle property of the proposed method that possesses both the model selection consistency and estimation efficiency for the true non-zero coefficients. As specific examples, we introduce a robust method of composite L1-L2, and optimal composite quantile method and evaluate their performance in both simulated and real data examples.

Composite Estimation: An Asymptotically Weighted Least Squares Approach

“…For further developments of this methodology in semiparametric settings, see Kai, Li and Zou (2011). To achieve variance-reduction as well as robustness, Bradic, Fan and Wang (2011) introduced a penalized composite quasi-likelihood for ultrahigh dimensional variable selection by combining several convex loss functions, together with a weighted L 1 -penalty. As a common purpose of these methodologies is to reduce estimation variance, we call them the variance-reduction methodologies.…”

“…As the results do not have significant difference, we do not report them here. (1) The AWLS estimatorβ and the WCQR estimator of Bradic, Fan and Wang (2011) It is worth pointing out that the simulation result depends the assumption on the distribution of the error term. As shown by a referee, if the error is Gaussian, the OLS is by far the best method in this setting because the basic quantile regression estimators are much worse than OLS in this case.…”

“…According to Bradic, Fan and Wang (2011), the WCQR estimatorβ W CQ is determined by minimizing the composite objective function (4.1) with weight w k to each ρ τ k (·).…”

Single-Index-Based CoVaR With Very High-Dimensional Covariates

“…Kai et al (2011). Moreover, Bradic et al (2011) propose a general loss function framework for linear models, with a weighted sum of different kinds of loss functions, and the weights are selected to be data driven. Another type of loss considered is in Newey and Powell (1987) corresponding to expectile regression (ER).…”

Composite Quantile Regression for the Single-Index Model

“…Therefore, different types of flexible loss functions are considered in the literature to improve the estimation efficiency, such as, composite quantile regression, [29], [9] and [10]. Moreover, [3] proposed a general loss function framework for linear models, with a weighted sum of different kinds of loss functions, and the weights are selected to be data driven. Another special type of loss considered in [17] corresponds to expectile regression (ER) that is in spirit similar to QR but contains mean regression as its special case.…”