Accuracy guaranties for $\ell_{1}$ recovery of block-sparse signals

Juditsky, Anatoli; Karzan, Fatma Kılınç; Nemirovski, Arkadi; Polyak, B. T.

doi:10.1214/12-aos1057

Cited by 12 publications

(25 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Much progress has been made over the last decade in high-dimensional statistics where the number of unknown parameters greatly exceeds sample size. The vast majority of work has been pursued for point estimation such as consistency for prediction [7,21], oracle inequalities and estimation of a high-dimensional parameter [6,11,12,24,33,34,47,51] or variable selection [17,30,49,53]. Other references and exposition to a broad class of models can be found in [18] or [10].…”

mentioning

confidence: 99%

“…So under (A2) for suitable λ ≍ log(p)/n β − β 0 2 = O P ( s 0 log(p)/n) (24) (see also [6]). This result will be applied in the next subsection, albeit to the lasso for node wise regression instead of for the original linear model.…”

mentioning

confidence: 99%

“…which follows from Lemma 5.3. Finally, we have using standard arguments for the ℓ 2 -norm bounds [see also (24)]…”

mentioning

confidence: 99%

See 2 more Smart Citations

On asymptotically optimal confidence regions and tests for high-dimensional models

et al. 2014

View full text Add to dashboard Cite

convergence to the limit is not uniform. Furthermore, bootstrap and even subsampling techniques are plagued by noncontinuity of limiting distributions. Nevertheless, in the low-dimensional setting, a modified bootstrap scheme has been proposed; [13] and [14] have recently proposed a residual based bootstrap scheme. They provide consistency guarantees for the highdimensional setting; we consider this method in an empirical analysis in Section 4.Some approaches for quantifying uncertainty include the following. The work in [50] implicitly contains the idea of sample splitting and corresponding construction of p-values and confidence intervals, and the procedure has been improved by using multiple sample splitting and aggregation of dependent p-values from multiple sample splits [32]. Stability selection [31] and its modification [41] provides another route to estimate error measures for false positive selections in general high-dimensional settings. An alternative method for obtaining confidence sets is in the recent work [29]. From another and mainly theoretical perspective, the work in [24] presents necessary and sufficient conditions for recovery with the lassoβ in terms of β − β 0 ∞ , where β 0 denotes the true parameter: bounds on the latter, which hold with probability at least say 1 − α, could be used in principle to construct (very) conservative confidence regions. At a theoretical level, the paper [35] derives confidence intervals in ℓ 2 for the case of two possible sparsity levels. Other recent work is discussed in Section 1.1 below.We propose here a method which enjoys optimality properties when making assumptions on the sparsity and design matrix of the model. For a linear model, the procedure is as the one in [52] and closely related to the method in [23]. It is based on the lasso and is "inverting" the corresponding KKT conditions. This yields a nonsparse estimator which has a Gaussian (limiting) distribution. We show, within a sparse linear model setting, that the estimator is optimal in the sense that it reaches the semiparametric efficiency bound. The procedure can be used and is analyzed for high-dimensional sparse linear and generalized linear models and for regression problems with general convex (robust) loss functions.1.1. Related work. Our work is closest to [52] who proposed the semiparametric approach for distributional inference in a high-dimensional linear model. We take here a slightly different view-point, namely by inverting the KKT conditions from the lasso, while relaxed projections are used in [52]. Furthermore, our paper extends the results in [52] by: (i) treating generalized linear models and general convex loss functions; (ii) for linear models, we give conditions under which the procedure achieves the semiparametric efficiency bound and our analysis allows for rather general Gaussian, sub-Gaussian and bounded design. A related approach as in [52] was proposed CONFIDENCE REGIONS FOR HIGH-DIMENSIONAL MODELS 3 in [8] based on ridge regression which is clearly suboptimal and ineffi...

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

On asymptotically optimal confidence regions and tests for high-dimensional models

et al. 2014

View full text Add to dashboard Cite

show abstract

“…As we are here focusing on the general problem, occurring also for numerous other forms of signals, we have here opted to exclude the treatment of inharmonicity, although note that the algorithm may be extended to allow for this along the lines presented in [31,32], or using a dictionary learning approach such as in [33,34]. The theoretical study of block sparse signals was initially suggested in [35], where it is shown that including this structure in the estimation procedure has great practical consequences, improving both theoretical recovery limits and numerical results in many cases (see, e.g., [35][36][37][38]). Generally, this form of group sparse convex optimization problems are computationally cumbersome; for this reason, we also derive an efficient algorithm to form the estimate based on the alternating directions methods of multipliers (ADMM) (see, e.g., [39,40]).…”

Section: Introductionmentioning

confidence: 99%

Multi-pitch estimation exploiting block sparsity

2015

View full text Add to dashboard Cite

“…We present the respective sparsity structures and provide verifiable sufficient conditions for the validity of associated nullspace properties (and thus -for the validity of the corresponding recovery routines); the prototypes of our verifiable conditions can be found in [5,6,7,8].…”

Section: Introductionmentioning

confidence: 99%

On a unified view of nullspace-type conditions for recoveries associated with general sparsity structures

Juditsky

Karzan

Nemirovski

2014

Linear Algebra and its Applications

Self Cite

View full text Add to dashboard Cite

We discuss a general notion of "sparsity structure" and associated recoveries of a sparse signal from its linear image of reduced dimension possibly corrupted with noise. Our approach allows for unified treatment of (a) the "usual sparsity" and "usual ℓ 1 recovery," (b) block-sparsity with possibly overlapping blocks and associated block-ℓ 1 recovery, and (c) low-rank-oriented recovery by nuclear norm minimization. The proposed recovery routines are natural extensions of the usual ℓ 1 minimization used in Compressed Sensing. Specifically, we present nullspace-type sufficient conditions for the recovery to be precise on sparse signals in the noiseless case. Then we derive error bounds for imperfect (nearly sparse signal, presence of observation noise, etc.) recovery under these conditions. In all of these cases, we present efficiently verifiable sufficient conditions for the validity of the associated nullspace properties.

show abstract

Accuracy guaranties for $\ell_{1}$ recovery of block-sparse signals

Cited by 12 publications

References 27 publications

On asymptotically optimal confidence regions and tests for high-dimensional models

On asymptotically optimal confidence regions and tests for high-dimensional models

Multi-pitch estimation exploiting block sparsity

On a unified view of nullspace-type conditions for recoveries associated with general sparsity structures

Contact Info

Product

Resources

About