Linear Hypothesis Testing in Dense High-Dimensional Linear Models

Abstract

We propose a methodology for testing linear hypothesis in high-dimensional linear models. The proposed test does not impose any restriction on the size of the model, i.e. model sparsity or the loading vector representing the hypothesis. Providing asymptotically valid methods for testing general linear functions of the regression parameters in high-dimensions is extremely challenging -- especially without making restrictive or unverifiable assumptions on the number of non-zero elements. We propose to test the moment conditions related to the newly designed restructured regression, where the inputs are transformed and augmented features. These new features incorporate the structure of the null hypothesis directly. The test statistics are constructed in such a way that lack of sparsity in the original model parameter does not present a problem for the theoretical justification of our procedures. We establish asymptotically exact control on Type I error without imposing any sparsity assumptions on model parameter or the vector representing the linear hypothesis. Our method is also shown to achieve certain optimality in detecting deviations from the null hypothesis. We demonstrate the favorable finite-sample performance of the proposed methods, via a number of numerical and a real data example.Comment: 42 pages, 8 figure

Approximate Residual Balancing: Debiased Inference of Average Treatment Effects in High Dimensions

“…To our knowledge, this was the first result for √ n-consistent inference about dense contrasts of β c at the time that we first circulated our manuscript. We note, however, simultaneous and independent work by Zhu and Bradic (2016), who developed a promising method for testing hypotheses of the form ξβ c = 0 for potentially dense vectors ξ; their approach uses an orthogonal moments construction that relies on regressing ξX i against a .p − 1/-dimensional design that captures the components of X i orthogonal to ξ.…”

“…(There are, of course, some exceptions to this assumption. In recent work, Javanmard and Montanari (2015) showed that inference of β c is possible even when k n log.p/ in a setting where X is a random Gaussian matrix with either a known or extremely sparse population precision matrix; Wager et al (2016) showed that lasso regression adjustments allow for efficient average treatment effect estimation in randomized trials even when k n log.p/, whereas the method of Zhu and Bradic (2016) for estimating dense contrasts ξβ c does not rely on sparsity of β c and instead places assumptions on the joint distribution of ξX i and the individual regressors. The point in common between these results is that they let us weaken the sparsity requirements at the expense of strengthening our assumptions about the X-distribution.…”

Fixed Effects Testing in High-Dimensional Linear Mixed Models

“…The sensitivity of our procedure to the choice of the tuning parameters is given in Table 4. Similarly to Zhu and Bradic (2016), we chooseη γ = 0.05 V P V /n.…”

“…Nominal level is taken to be 5%. LM procedure consists in applying the linear model procedure of Zhu and Bradic (2016) to the mixed model while MM procedure consists in applying our mixed model procedure. Parameters are n = 200, p = 500, N = 50, s = 5.…”

Targeted Undersmoothing

“…[18] consider construction of confidence sets for dense functionals given by a(β) = β l for various 1 l ∞. Perhaps the most closely related current papers are [58] and [59]. Both [58] and [59] construct hypothesis tests for objects similar to those considered in our paper via 1 -projections of coefficient estimates to the set of coefficients consistent with the null.…”