χ 2-Confidence Sets in High-Dimensional Regression

Abstract

We study a high-dimensional regression model. Aim is to construct a confidence set for a given group of regression coefficients, treating all other regression coefficients as nuisance parameters. We apply a one-step procedure with the square-root Lasso as initial estimator and a multivariate square-root Lasso for constructing a surrogate Fisher information matrix. The multivariate square-root Lasso is based on nuclear norm loss with 1penalty. We show that this procedure leads to an asymptotically χ 2 -distributed pivot, with a remainder term depending only on the 1 -error of the initial estimator. We show that under 1 -sparsity conditions on the regression coefficients β 0 the square-root Lasso produces to a consistent estimator of the noise variance and we establish sharp oracle inequalities which show that the remainder term is small under further sparsity conditions on β 0 and compatibility conditions on the design. arXiv:1502.07131v2 [math.ST] 15 Sep 2015 Ω * (z j ).Thus, when Ω is the 1 -norm we have A 1,Ω = A 1 and Z ∞,Ω * = Z ∞ . We let the multivariate square-root Ω -sparse estimator bêThis estimator equals (6) when Ω is the 1 -norm.We let, as in (7), (8) and (10) but now with the newΓ J , the quantitiesT J ,T J and M be defined asTThe Ω -de-sparsified estimator of β 0 J is as in Definition 1but now withβ not necessarily the square root Lasso but a suitably chosen initial estimator and withΓ J the multivariate square-root Ω -sparse estimator. The normalized de-sparsified estimator is Mb J with normalization matrix M given above. We can then easily derive the following extension of Theorem 1. Proof of the main result in Subsection 3.2Proof of Theorem 1. We havewhere we invoked the KKT-conditions (9). We thus arrive atwhere rem = − √ nλẐ T J (β −J − β 0 −J )/σ 0 .The co-variance matrix of the first termT −1/2 J (X J − X −JΓJ ) T ε/ √ n in (15) is equal to

Group inference in high dimensions with applications to hierarchical testing

“…In comparison, the detection threshold of the χ 2 test is in the order of (|G|/n) 1/2 [26,35] which is inferior to the proposed test for a large |G|. Additionally, for certain difficult settings, our proposed test achieves the same detection boundaries as the maximum test [11,13,38], up to a constant, see the discussion after Corollary 2.…”

“…The proposed test is valid for any group size |G| in terms of type-I error control. As a generalization of the F-test to high-dimensions, the group tests proposed in [26] and [35] require the group size |G| to be smaller than the sample size. Our test has good power performance for testing large groups.…”

“…Additionally, the accuracy of the test statistic depends only weakly on the group size |G|, in the sense that the standard deviation of the test statistic depends on β G 2 , in the order of magnitude ( β G 2 + β G 2 2 )/n 1/2 ; but since β G 2 ≤ β 2 and β is sparse, the standard deviation is not always strictly increasing with a growing set G and this phenomenon explains the statistical efficiency of the proposed test, especially when the test size G is large. In contrast, the χ 2 test proposed in [26,35] has the standard deviation at order of (|G|/n) 1/2 , which is strictly increasing with |G|. This also explains their condition |G|…”

Asymptotic Confidence Regions for High-Dimensional Structured Sparsity

“…For the LASSO estimator, most work up until recently has been focusing on point estimation among other topics, with not much focus on establishing uncertainty in high dimensional models. Interest has been growing rapidly on the very important topic of constructing confidence regions for the LASSO estimator, see for example van de Geer et al [18], van de Geer and Stucky [17], Zhang and Zhang [21], Javanmard and Montanari [5] and Meinshausen [7]. When it comes to confidence regions for structured sparsity estimators there has not yet been done much work to our knowledge.…”

Scale calibration for high-dimensional robust regression

“…It is plausible that such tests exist using an initial M -estimator such as the regression estimator introduced in this paper (cf. van de Geer and Stucky [52] and Sur et al [49] for some theory in the non-robust setting).…”