Regression coefficient and autoregressive order shrinkage and selection via the lasso

Abstract

The least absolute shrinkage and selection operator (lasso) has been widely used in regression shrinkage and selection. In this article, we extend its application to the REGression model with AutoRegressive errors (REGAR). Two types of lasso estimators are carefully studied. The first is similar to the traditional lasso estimator with only two tuning parameters (one for regression coefficients and the other for autoregression coefficients). These tuning parameters can be easily calculated via a data driven method, but the resulting lasso estimator may not be fully efficient (Fan and Li, 2001). In order to overcome this limitation, we propose a second lasso estimator which uses different tuning parameters for each coefficient. We show that this modified lasso is able to produce the estimator as efficiently as the oracle. Moreover, we propose an algorithm for tuning parameter estimates to obtain the modified lasso estimator. Simulation studies demonstrate that the modified estimator is superior to the traditional one. One empirical example is also presented to illustrate the usefulness of lasso estimators. The extension of lasso to the autoregression with exogenous variables (ARX) model is briefly discussed.

A note on adaptive group lasso

“…Similar methods were also developed for Cox's proportional hazard model (Zhang and Lu, 2007), least absolute deviation regression (Wang et al, 2007a), and linear regression with autoregressive residuals (Wang et al, 2007b).…”

On the distribution of the adaptive LASSO estimator

“…Apart from the papers already mentioned, there has been a recent surge of publications establishing the 'oracle' property for a variety of penalized maximum likelihood or related estimators (e.g., Bunea (2004), Bunea & McKeague (2005), Fan & Li (2002, 2004, Li & Liang (2007), Wang & Leng (2007), Wang, G. Li and Jiang (2007), Wang, G. Li and Tsai (2007), Wang, R. Li and Tsai (2007), Yuan & Lin (2007), Zhang & Lu (2007), Zou & Yuan (2008), Zou & Li (2008), Johnson et al (2008)). The 'oracle' property also paints a misleading picture of the behavior of the estimators considered in these papers; see the discussion in Leeb & Pötscher (2005), Yang (2005), Pötscher (2007), Pötscher & Leeb (2007), Leeb & Pötscher (2008b).…”

Adaptive Lasso-Type Estimation for Multivariate Diffusion Processes

“…For example, Wang, Li, and Tsai (2007) consider the problem of shrinkage estimation of regressive and autoregressive coefficients, while Nardi and Rinaldo (2011) consider penalized order selection in an AR( p) model. Furthermore, Caner and Knight (2010) show that econometricians can use a Bridge estimator to differentiate stationarity from unit root type of nonstationarity and select the optimal lag in autoregression (AR) series as well.…”