A general theory of hypothesis tests and confidence regions for sparse high dimensional models

Abstract

We consider the problem of uncertainty assessment for low dimensional components in high dimensional models. Specifically, we propose a decorrelated score function to handle the impact of high dimensional nuisance parameters. We consider both hypothesis tests and confidence regions for generic penalized M-estimators. Unlike most existing inferential methods which are tailored for individual models, our approach provides a general framework for high dimensional inference and is applicable to a wide range of applications. From the testing perspective, we develop general theorems to characterize the limiting distributions of the decorrelated score test statistic under both null hypothesis and local alternatives. These results provide asymptotic guarantees on the type I errors and local powers of the proposed test. Furthermore, we show that the decorrelated score function can be used to construct point estimators that are semiparametrically efficient and the optimal confidence regions. We also generalize this framework to broaden its applications. First, we extend it to handle high dimensional null hypothesis, where the number of parameters of interest can increase exponentially fast with the sample size. Second, we establish the theory for model misspecification. Third, we go beyond the likelihood framework, by introducing the generalized score test based on general loss functions. Thorough numerical studies are conducted to back up the developed theoretical results.

Distributed testing and estimation under sparse high dimensional models

“…$$\mathbb{P}({\Vert \frac{1}{n_k}{X}^{false(jfalse)}false({\widehat{\mathrm{bold-italic\beta }}}^{\lambda }false({\mathcal{D}}_{j}false)-{\mathit{\beta }}^{\ast }false)\Vert }_2\succsim n^{-1/2}\sqrt{\mathit{sk}log(d/\delta )}<\delta$$ and by Lemma C.4 of Ning and Liu (2014),…”

“…The problems associated with the use of the classical score statistic in the presence of a high dimensional nuisance parameter are brought to light by Ning and Liu (2014), who propose a remedy via the decorrelated score. The problem stems from the inversion of the matrix $J_{-v,-v}^{\ast }$ in high dimensions.…”

“…Since ${\beta }_v^{\ast }$ is restricted under the null hypothesis, $H_0:{\beta }_v^{\ast }={\beta }_v^H$, the statistic in (3.18) is accessible once H 0 is imposed. As Ning and Liu (2014) point out, w * is the solution to…”

“…For the high dimensional analogue of Rao’s score statistic, the incorporation of a correction factor increases the convergence rate of higher order terms, thereby vanquishing the effect of the nuisance parameters. The approach is introduced in the k = 1 setting in Ning and Liu (2014), where the test statistic is shown to possess a tractable limit distribution. However, the computation complexity for the debiased estimators increases by an order of magnitude, due to solving d high-dimensional regularization problems.…”

An Adaptive Resampling Test for Detecting the Presence of Significant Predictors

“…Berk et al (2013) developed a valid method of post-model selection inference that is feasible for up to about p = 20 predictors, also assuming normal errors. In various sparse high-dimensional settings, Belloni et al (2013), Bühlmann (2013), Zhang and Zhang (2014) and Ning and Liu (2015) have established asymptotically valid confidence intervals for a preconceived regression parameter after variable selection on the remaining predictors, but this does not apply to marginal screening (where no regression parameter is singled-out a priori ).…”