Greedy signal space methods for incoherence and beyond

Giryes, Raja; Needell, Deanna

doi:10.1016/j.acha.2014.07.004

Cited by 24 publications

(49 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The works in [35], [36], [39] use a similar concept of nearoptimal projection (compared to [4] that assumes only exact projections). The main difference between these contributions and ours is that these papers focus on specific models, while we present a general framework that is not specific to a certain low-dimensional prior.…”

Section: Ipgd Convergence Analysismentioning

confidence: 99%

Tradeoffs Between Convergence Speed and Reconstruction Accuracy in Inverse Problems

Giryes

Eldar

Bronstein

et al. 2018

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

Solving inverse problems with iterative algorithms is popular, especially for large data. Due to time constraints, the number of possible iterations is usually limited, potentially affecting the achievable accuracy. Given an error one is willing to tolerate, an important question is whether it is possible to modify the original iterations to obtain faster convergence to a minimizer achieving the allowed error without increasing the computational cost of each iteration considerably. Relying on recent recovery techniques developed for settings in which the desired signal belongs to some low-dimensional set, we show that using a coarse estimate of this set may lead to faster convergence at the cost of an additional reconstruction error related to the accuracy of the set approximation. Our theory ties to recent advances in sparse recovery, compressed sensing, and deep learning. Particularly, it may provide a possible explanation to the successful approximation of the ℓ1-minimization solution by neural networks with layers representing iterations, as practiced in the learned iterative shrinkage-thresholding algorithm (LISTA).

show abstract

Section: Ipgd Convergence Analysismentioning

confidence: 99%

Tradeoffs Between Convergence Speed and Reconstruction Accuracy in Inverse Problems

Giryes

Eldar

Bronstein

et al. 2018

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [64], the redundancy issue is addressed by an extension of OMP, i.e., -OMP algorithm. Moreover, some other works extend classical greedy methods, such as CoSaMP, SP and IHT, to handle coherent dictionaries [61]- [63]. Lastly, a new family of pursuit algorithms have been proposed for the cosparse analysis model that is an interesting alternative to the standard SR [71].…”

Section: B Existing Greedy Methods For Sparse Recoverymentioning

confidence: 99%

“…3 We first generate a Gaussian random matrix , and then simply set (in Matlab notation). Since there are repeated columns, the RIP condition of does not hold [63]. Moreover, we generate a 40-sparse ground truth by letting and .…”

Section: A Convergence On Non-rip Dictionariesmentioning

confidence: 99%

Matching Pursuit LASSO Part I: Sparse Recovery Over Big Dictionary

Tan

Tsang

Wang

2015

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Large-scale sparse recovery (SR) by solving -norm relaxations over Big Dictionary is a very challenging task. Plenty of greedy methods have therefore been proposed to address big SR problems, but most of them require restricted conditions for the convergence. Moreover, it is non-trivial for them to incorporate the -norm regularization that is required for robust signal recovery. We address these issues in this paper by proposing a Matching Pursuit LASSO (MPL) algorithm, based on a novel quadratically constrained linear program (QCLP) formulation, which has several advantages over existing methods. Firstly, it is guaranteed to converge to a global solution. Secondly, it greatly reduces the computation cost of the -norm methods over Big Dictionaries. Lastly, the exact sparse recovery condition of MPL is also investigated.Index Terms-Sparse recovery, compressive sensing, LASSO, matching pursuit, big dictionary, convex programming.(4) where denotes the regularization parameter.Many algorithms have been proposed to address the nonsmooth LASSO problem. An interior-point method is proposed in [33], and a gradient projection (GPSR) method was subsequently proposed by transforming LASSO into a quadratic programming problem [27]. A proximal gradient (PG) method is proposed to solve (4) based on a shrinkage-threshold operator [34]. To speed up the PG, a fast iterative shrinkage-threshold algorithm (FISTA) (also known as the accelerated proximal gradient method) is proposed in [29]. Recently, to tackle large-scale 1053-587X

show abstract

“…We share a similar conclusion with those works; for a large ε more measurements are required for CS recovery and the convergence becomes slow. Hegde et al [30] proposed such projection oracle for tree-sparse signals and use it for the related model-based CS problem using a CoSamp type algorithm (see also [31,32] for related works on inexact CoSamp type algorithms). In a later work [33] the authors consider a modified variant of IPG with (1 + ε)-approximate projections for application to structured sparse reconstruction problems (specifically tree-sparse and earthmover distance CS models).…”

Section: Related Workmentioning

confidence: 99%

Inexact Gradient Projection and Fast Data Driven Compressed Sensing

Golbabaee¹,

Davies²

2018

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We study convergence of the iterative projected gradient (IPG) algorithm for arbitrary (possibly nonconvex) sets and when both the gradient and projection oracles are computed approximately. We consider different notions of approximation of which we show that the Progressive Fixed Precision (PFP) and the (1 + ε)-optimal oracles can achieve the same accuracy as for the exact IPG algorithm. We show that the former scheme is also able to maintain the (linear) rate of convergence of the exact algorithm, under the same embedding assumption. In contrast, the (1 + ε)-approximate oracle requires a stronger embedding condition, moderate compression ratios and it typically slows down the convergence.We apply our results to accelerate solving a class of data driven compressed sensing problems, where we replace iterative exhaustive searches over large datasets by fast approximate nearest neighbour search strategies based on the cover tree data structure. For datasets with low intrinsic dimensions our proposed algorithm achieves a complexity logarithmic in terms of the dataset population as opposed to the linear complexity of a brute force search. By running several numerical experiments we conclude similar observations as predicted by our theoretical analysis.

show abstract

Greedy signal space methods for incoherence and beyond

Cited by 24 publications

References 32 publications

Tradeoffs Between Convergence Speed and Reconstruction Accuracy in Inverse Problems

Tradeoffs Between Convergence Speed and Reconstruction Accuracy in Inverse Problems

Matching Pursuit LASSO Part I: Sparse Recovery Over Big Dictionary

Inexact Gradient Projection and Fast Data Driven Compressed Sensing

Contact Info

Product

Resources

About