Generic error bounds for the generalized Lasso with sub-exponential data

Abstract

This work performs a non-asymptotic analysis of the generalized Lasso under the assumption of sub-exponential data. Our main results continue recent research on the benchmark case of (sub-)Gaussian sample distributions and thereby explore what conclusions are still valid when going beyond. While many statistical features remain unaffected (e.g., consistency and error decay rates), the key difference becomes manifested in how the complexity of the hypothesis set is measured. It turns out that the estimation error can be controlled by means of two complexity parameters that arise naturally from a generic-chaining-based proof strategy. The output model can be non-realizable, while the only requirement for the input vector is a generic concentration inequality of Bernstein-type, which can be implemented for a variety of sub-exponential distributions. This abstract approach allows us to reproduce, unify, and extend previously known guarantees for the generalized Lasso. In particular, we present applications to semi-parametric output models and phase retrieval via the lifted Lasso. Moreover, our findings are discussed in the context of sparse recovery and high-dimensional estimation problems.

Quantizing Heavy-Tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery

“…While we capture heavy-tailedness by bounded moment of some small order, there has been a line of works considering sub-exponential or more generally sub-Weibull distributions [39], [58], [80], [85], which have heavier tail than sub-Gaussian ones but still possess finite moment up to arbitrary order. Specifically, without truncation and quantization, sparse linear regression was studied under sub-exponential data in [85] and under sub-Weibull data in [58], and the obtained error rates match the ones in the sub-Gaussian case up to logarithmic factors.…”

“…Specifically, without truncation and quantization, sparse linear regression was studied under sub-exponential data in [85] and under sub-Weibull data in [58], and the obtained error rates match the ones in the sub-Gaussian case up to logarithmic factors. Additionally, under sub-exponential measurement matrix and noise, [80] established a uniform guarantee for 1-bit generative compressed sensing, while [39] analyzed generalized Lasso for a general nonlinear model. Because the tail assumptions in these works are substantially stronger than ours, there is not a common fair stage for further comparison.…”