On the distribution of the adaptive LASSO estimator

Abstract

We study the distribution of the adaptive LASSO estimator (Zou (2006)) in finite samples as well as in the large-sample limit. The largesample distributions are derived both for the case where the adaptive LASSO estimator is tuned to perform conservative model selection as well as for the case where the tuning results in consistent model selection. We show that the finite-sample as well as the large-sample distributions are typically highly non-normal, regardless of the choice of the tuning parameter. The uniform convergence rate is also obtained, and is shown to be slower than n −1/2 in case the estimator is tuned to perform consistent model selection. In particular, these results question the statistical relevance of the 'oracle' property of the adaptive LASSO estimator established in Zou (2006). Moreover, we also provide an impossibility result regarding the estimation of the distribution function of the adaptive LASSO estimator. The theoretical results, which are obtained for a regression model with orthogonal design, are complemented by a Monte Carlo study using non-orthogonal regressors.MSC 2000 subject classification. Primary 62F11, 62F12, 62E15, 62J05, 62J07.

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

“…We also show that the finite-sample distribution of these estimators cannot be estimated in any reasonable sense, complementing results of this sort in the literature [11][12][13][14]. In a subsequent paper, [15], analogous results are obtained for the adaptive LASSO estimator.…”

“…By Theorem 16 we know that sup |θ|<d/n 1/2 P n,θ F n (s) − F n,θ (s) > ε ≥ 1 2 for each ε < (Φ(s + n 1/2 η n ) − Φ(s − n 1/2 η n ))/2 and for each d > |s|. Rewriting this in terms of t,Ĝ n (t), and G n,θ (t) and setting c = dn −1/2 /η n gives (15). Relation (16) is a trivial consequence of (15).…”

“…Let 0 < η n < ∞ and let t ∈ R be arbitrary. Then every estimatorĜ n (t) of G n,θ (t) satisfies sup |θ|<cη n P n,θ Ĝ n (t) − G n,θ (t) > ε ≥ 1 2 (15) for each ε < (Φ(n 1/2 η n (t + 1)) − Φ(n 1/2 η n (t − 1)))/2, for each c > |t|, and for each fixed sample size n. If η n satisfies η n → 0 and n 1/2 η n → ∞ as n → ∞, we thus have for each c > |t| lim inf n→∞ inf G n (t) sup |θ |<cη n P n,θ Ĝ n (t) − G n,θ (t) > ε ≥ 1 2 (16) for each ε < 1/2 if |t| < 1 and for ε < 1/4 if |t| = 1, where the infimum in (16) extends over all estimatorsĜ n (t).…”

Inference for survival prediction under the regularized Cox model

“…We focus on this setting because precise interval estimation based on shrinkage estimators is feasible with moderate sample sizes. When some of the true signals are of order O(n −1/2 ), adaptive lasso-type estimators are expected to yield significant bias for such weak signals and it becomes implausible to construct precise CIs as previously shown in Pötscher and Schneider (2009). To further examine the performance of our proposed procedures under such settings, we present additional simulation results in the Online Supplementary Materials (Web Appendix C, available at Biostatistics online) which follow the same structure as those above, but with h(z) = 1 · z 1 + 0.8 · z 2 + 0.6 · z 3 + 0.4 · z 4 + 0.2 · z 5 , and focus in particular on the small signal θ 05 = 0.2.…”

Discussion: Approximating data

“…(It is somewhat unsettling that the top journals in our profession publish papers using this sort of misleading asymptotics nowadays at a considerable rate.) For more detailed documentation of the irrelevance of the "oracle" property for judging the actual performance of an estimator see Leeb and Pötscher (2005), Leeb and Pötscher (2008), Pötscher (2007), Pötscher and Leeb (2007), and Pötscher and Schneider (2007). Other instances where pointwise asymptotics are misleading are discussed in Pötscher (2002) and are related to the ill-posedness of estimation problems as discussed in Davies' paper.…”