Asymptotic Confidence Regions for High-Dimensional Structured Sparsity

Abstract

In the setting of high-dimensional linear regression models, we propose two frameworks for constructing pointwise and group confidence sets for penalized estimators which incorporate prior knowledge about the organization of the non-zero coefficients. This is done by desparsifying the estimator as in van de Geer et al. [18] and van de Geer and Stucky [17], then using an appropriate estimator for the precision matrix Θ. In order to estimate the precision matrix a corresponding structured matrix norm penalty has to be introduced. After normalization the result is an asymptotic pivot. The asymptotic behavior is studied and simulations are added to study the differences between the two schemes.Applying equation (3.3) to the optimal solution of equation (3.1) leads to the KKT conditions (in the case of X J − X J cB J nuc = 0):Here we denoteΣ J := (X J − X J cB J ) T (X J − X J cB J )/n (assumed to be nonsingular). Let us additionally define the |J| de-sparsified Ω structured estimator with the help of the following notations.The normalizing matrix can then be written asThis leads to the definition of the |J| de-sparsified Ω structured estimator. Defining a de-sparsified estimator in this way lets us deal with group-wise confidence sets.Definition 3. The |J| de-sparsified Ω structured estimator iŝWith these definitions we are now ready to describe the asymptotic behavior of the estimator (3.5) in the following Theorem.As we can see from Lemma 2 (4) we can upper bound part of the reminder term from Theorem 1 as g(β J c − β 0 J c ) ≤ g(β − β 0 ). By the definition of the gauge function g, from Lemma 2 (2) we get thatWe can simplify the termThat is why we can conclude thatwhere Z comes from the KKT conditions which fulfills:. The remainder term can be bounded with the generalized Cauchy Schwartz inequality in the ∞ -norm byThe last inequality comes directly from the weak decomposability of the allowed 20 set J:Now with the calculation in the proof of Lemma 3 and by dividing everything by σ 0 the proof is finished.

The de-biased group Lasso estimation for varying coefficient models

“…• Construction of Θ : At the end of this section, we construct Θ by employing the group Lasso and adapting the idea in [28] to the current group structure. Note that our construction is different from those of [23] and [25] and that we can exploit just the standard R package for the Lasso for computation. We also describe some idea of how to Θ in (9)-(11) after the notation.…”

“…columnwise by employing the group Lasso differently from [25]. See Remark 1 at the end of this section.…”

“…To our knowledge, there are a few papers on linear regression models with group structure(e.g. [23], [25]). The authors of these papers derived interesting and useful results.…”

“…We also have to take care of approximation errors to true coefficient functions. Our purpose is to estimate coefficient functions and different from that of [23] and [25] does not deal with random design cases. Both of them consider only linear models.…”

IPAD: Stable Interpretable Forecasting with Knockoffs Inference

“…For example, [42] proposed a simple procedure for inference of the average partial effects based on a debiased 1 -regularized method in approximately sparse panel probit models. [38] used the de-sparsified estimator for constructing pointwise and group confidence sets. [43] conducted simultaneous inference for high-dimensional sparse linear models based on a bootstrap and desparsifying Lasso estimator.…”