Joint &lt;formula formulatype="inline"&gt;&lt;tex Notation="TeX"&gt;$k$&lt;/tex&gt; &lt;/formula&gt;-Step Analysis of Orthogonal Matching Pursuit and Orthogonal Least Squares

Soussen, Charles; Gribonval, Rémi; Idier, Jérôme; Herzet, Cédric

doi:10.1109/tit.2013.2238606

Cited by 69 publications

(111 citation statements)

References 28 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…When the dictionary atoms are close to orthogonal, OLS and OMP have similar behaviors, as emphasized in [10]. On the contrary, for correlated dictionaries (e.g., in ill-conditioned inverse problems), their behaviors significantly differ and OLS may be a better choice [20]. The above arguments motivate our analysis of both OMP and OLS.…”

Section: A Results and State-of-the-art Conditions For Oxx And Oxxqmentioning

confidence: 81%

“…For conciseness reasons, we therefore skip it. Let us just mention that the original formulation of [20,Th. 3] involves a vector y ∈ span(AQ⋆∪Q).…”

Section: Definition 3 (Successful Recovery) Oxxq With Y Defined In (5mentioning

confidence: 99%

“…A slightly different, but related, perspective was considered in [19] for OMP and in [20] for both OMP and OLS. In these papers, the authors derived guarantees of success for OMP and OLS by assuming that atoms belonging to some subset Q have been selected during the first iterations.…”

Section: Introductionmentioning

confidence: 99%

“…Secondly, in [14]- [16], a series of sufficient conditions based on restricted isometry constants (RICs) were proposed to guarantee the success of (P1,Q) (or some variants thereof). Concerning OMPQ/OLSQ, the authors in [20] derived a partial "Exact Recovery Condition" (ERC) extending Tropp's ERC to the partially-informed paradigm considered in this paper. The extended condition was shown to be sufficient but also worst-case necessary for the success of OMP/OLS when some support Q has been selected at an intermediate iteration.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Exact Recovery Conditions for Sparse Representations With Partial Support Information

Herzet

Soussen

Idier³

et al. 2013

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

To cite this version:Cédric Herzet, Charles Soussen, Jérôme Idier, Rémi Gribonval. Abstract-We address the exact recovery of a k-sparse vector in the noiseless setting when some partial information on the support is available. This partial information takes the form of either a subset of the true support or an approximate subset including wrong atoms as well. We derive a new sufficient and worst-case necessary (in some sense) condition for the success of some procedures based on ℓp-relaxation, Orthogonal Matching Pursuit (OMP) and Orthogonal Least Squares (OLS). Our result is based on the coherence µ of the dictionary and relaxes the wellknown condition µ < 1/(2k − 1) ensuring the recovery of any k-sparse vector in the non-informed setup. It reads µ < 1/(2k − g + b − 1) when the informed support is composed of g good atoms and b wrong atoms. We emphasize that our condition is complementary to some restrictedisometry based conditions by showing that none of them implies the other.Because this mutual coherence condition is common to all procedures, we carry out a finer analysis based on the Null Space Property (NSP) and the Exact Recovery Condition (ERC). Connections are established regarding the characterization of ℓp-relaxation procedures and OMP in the informed setup. First, we emphasize that the truncated NSP enjoys an ordering property when p is decreased. Second, the partial ERC for OMP (ERC-OMP) implies in turn the truncated NSP for the informed ℓ 1 problem, and the truncated NSP for p < 1.

show abstract

Section: A Results and State-of-the-art Conditions For Oxx And Oxxqmentioning

confidence: 81%

“…For conciseness reasons, we therefore skip it. Let us just mention that the original formulation of [20,Th. 3] involves a vector y ∈ span(AQ⋆∪Q).…”

Section: Definition 3 (Successful Recovery) Oxxq With Y Defined In (5mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exact Recovery Conditions for Sparse Representations With Partial Support Information

Herzet

Soussen

Idier³

et al. 2013

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other support exploration strategies maintain the desired sparsity level at each iteration, and perform local combinatorial exploration steps [9]. Optimality proofs for all such strategies also rely on very restrictive hypotheses [10], [11]. More " 0 -oriented" approaches were proposed, e.g., by successive continuous approximations of the 0 norm [12], by descent-based Iterative Hard Thresholding (IHT) [13], [14] and by penalty decomposition methods [15].…”

Section: A Sparse Estimation For Inverse Problemsmentioning

confidence: 99%

Exact Sparse Approximation Problems via Mixed-Integer Programming: Formulations and Computational Performance

Bourguignon¹,

Ninin

Carfantan

et al. 2016

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Abstract-Sparse approximation addresses the problem of approximately fitting a linear model with a solution having as few non-zero components as possible. While most sparse estimation algorithms rely on suboptimal formulations, this work studies the performance of exact optimization of 0-norm-based problems through Mixed-Integer Programs (MIPs). Nine different sparse optimization problems are formulated based on 1, 2 or ∞ data misfit measures, and involving whether constrained or penalized formulations. For each problem, MIP reformulations allow exact optimization, with optimality proof, for moderate-size yet difficult sparse estimation problems. Algorithmic efficiency of all formulations is evaluated on sparse deconvolution problems. This study promotes error-constrained minimization of the 0 norm as the most efficient choice when associated with 1 and ∞ misfits, while the 2 misfit is more efficiently optimized with sparsity-constrained and sparsity-penalized problems. Then, exact 0-norm optimization is shown to outperform classical methods in terms of solution quality, both for over-and underdetermined problems. Finally, numerical simulations emphasize the relevance of the different p fitting possibilities as a function of the noise statistical distribution. Such exact approaches are shown to be an efficient alternative, in moderate dimension, to classical (suboptimal) sparse approximation algorithms with 2 data misfit. They also provide an algorithmic solution to less common sparse optimization problems based on 1 and ∞ misfits. For each formulation, simulated test problems are proposed where optima have been successfully computed. Data and optimal solutions are made available as potential benchmarks for evaluating other sparse approximation methods.

show abstract

Compressed Sensing and Dictionary Learning

Swamy

2019

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

Joint <formula formulatype="inline"><tex Notation="TeX">$k$</tex> </formula>-Step Analysis of Orthogonal Matching Pursuit and Orthogonal Least Squares

Cited by 69 publications

References 28 publications

Exact Recovery Conditions for Sparse Representations With Partial Support Information

Exact Recovery Conditions for Sparse Representations With Partial Support Information

Exact Sparse Approximation Problems via Mixed-Integer Programming: Formulations and Computational Performance

Compressed Sensing and Dictionary Learning

Contact Info

Product

Resources

About