On asymptotically optimal confidence regions and tests for high-dimensional models

Abstract

convergence to the limit is not uniform. Furthermore, bootstrap and even subsampling techniques are plagued by noncontinuity of limiting distributions. Nevertheless, in the low-dimensional setting, a modified bootstrap scheme has been proposed; [13] and [14] have recently proposed a residual based bootstrap scheme. They provide consistency guarantees for the highdimensional setting; we consider this method in an empirical analysis in Section 4.Some approaches for quantifying uncertainty include the following. The work in [50] implicitly contains the idea of sample splitting and corresponding construction of p-values and confidence intervals, and the procedure has been improved by using multiple sample splitting and aggregation of dependent p-values from multiple sample splits [32]. Stability selection [31] and its modification [41] provides another route to estimate error measures for false positive selections in general high-dimensional settings. An alternative method for obtaining confidence sets is in the recent work [29]. From another and mainly theoretical perspective, the work in [24] presents necessary and sufficient conditions for recovery with the lassoβ in terms of β − β 0 ∞ , where β 0 denotes the true parameter: bounds on the latter, which hold with probability at least say 1 − α, could be used in principle to construct (very) conservative confidence regions. At a theoretical level, the paper [35] derives confidence intervals in ℓ 2 for the case of two possible sparsity levels. Other recent work is discussed in Section 1.1 below.We propose here a method which enjoys optimality properties when making assumptions on the sparsity and design matrix of the model. For a linear model, the procedure is as the one in [52] and closely related to the method in [23]. It is based on the lasso and is "inverting" the corresponding KKT conditions. This yields a nonsparse estimator which has a Gaussian (limiting) distribution. We show, within a sparse linear model setting, that the estimator is optimal in the sense that it reaches the semiparametric efficiency bound. The procedure can be used and is analyzed for high-dimensional sparse linear and generalized linear models and for regression problems with general convex (robust) loss functions.1.1. Related work. Our work is closest to [52] who proposed the semiparametric approach for distributional inference in a high-dimensional linear model. We take here a slightly different view-point, namely by inverting the KKT conditions from the lasso, while relaxed projections are used in [52]. Furthermore, our paper extends the results in [52] by: (i) treating generalized linear models and general convex loss functions; (ii) for linear models, we give conditions under which the procedure achieves the semiparametric efficiency bound and our analysis allows for rather general Gaussian, sub-Gaussian and bounded design. A related approach as in [52] was proposed CONFIDENCE REGIONS FOR HIGH-DIMENSIONAL MODELS 3 in [8] based on ridge regression which is clearly suboptimal and ineffi...

An Adaptive Test on High-dimensional Parameters in Generalized Linear Models

“…In a penalized regression framework, several inference methods for a low-dimensional sub-vector of a high-dimensional regression coefficient vector have been developed (Van de Geer et al, 2014;Zhang and Zhang, 2014;Voorman et al, 2014), which however differs from the goal of testing on a high-dimensional parameter here and thus will not be further discussed.…”

Robust inference in high-dimensional approximately sparse quantile regression models

“…Several recent papers study the problem of constructing confidence regions after model selection allowing p ≫ n. In the case of linear mean regression, [6] proposed a double selection inference in a parametric with homoscedastic Gaussian errors, [10] studies the double selection procedure in a non-parametric setting with heteroscedastic errors, [39] and [35] proposed estimators based on ℓ 1 -penalized estimators based on "1-step" correction in parametric models. Going beyond mean models, [35] also provides high level conditions for the one-step estimator applied to smooth generalized linear problems, [7] analyzes confidence regions for a parametric homoscedastic LAD regression under primitive conditions based on the instrumental LAD regression, and [9] provides two post-selection procedures to build confidence regions for the logistic regression. None of the aforementioned papers deal with the problem of the present paper.…”

Calibration and Multiple Robustness When Data Are Missing Not At Random

“…Such an approach has been used in the construction of confidence intervals for high-dimensional linear regression in the recent literature. See, for example, [14,41,48,4]. We follow the same principle to de-bias the estimator β(λ) given in Algorithm 1.…”