Highly sparse representations from dictionaries are unique and independent of the sparseness measure

Gribonval, Rémi; Nielsen, Morten

doi:10.1016/j.acha.2006.09.003

Cited by 145 publications

(192 citation statements)

References 38 publications

Supporting

Mentioning

189

Contrasting

Unclassified

Order By: Relevance

“…It is easy to check that M ⊂ S, (see [12] and Appendix A.1). One can define a partial order [12] on S by letting f g if, and only if, there is some h ∈ M such that f = h • g (S is stable by composition, see Appendix A.1).…”

Section: Main Concepts and Resultsmentioning

confidence: 99%

“…One can define a partial order [12] on S by letting f g if, and only if, there is some h ∈ M such that f = h • g (S is stable by composition, see Appendix A.1). With respect to this partial order, the 0 and 1 "norms" are respectively the smallest and the largest admissible sparsity measures, in that…”

Section: Main Concepts and Resultsmentioning

confidence: 99%

“…By [12,Lemma 7], for any admissible sparsity measure h ∈ M, any sequence z = (z k ) and any integer M, we have…”

Section: Lemma 1 If F Is a Sub-additive Sparsity Measure Thenmentioning

confidence: 99%

“…Now, for any t, u ≥ 0 we have Figure 1 illustrates the fact that admissible sparsity measures are not necessarily as "nice" as one could imagine from their most natural examples the τ measures. Using the fact that the min or the max of two admissible sparsity measures yields another admissible sparsity measure [12], we can combine the 0 and the 1 norms into the "exotic" measure f (t) := min(f 1 (t), max(f 0 (t), f 1 (t)/2)).…”

Section: Lemmamentioning

confidence: 99%

See 3 more Smart Citations

A simple test to check the optimality of a sparse signal approximation

2006

Self Cite

View full text Add to dashboard Cite

Abstract:Approximating a signal or an image with a sparse linear expansion from an overcomplete dictionary of atoms is an extremely useful tool to solve many signal processing problems. Finding the sparsest approximation of a signal from an arbitrary dictionary is a NP-hard problem. Despite of this, several algorithms have been proposed that provide suboptimal solutions. However, it is generally difficult to know how close the computed solution is to being "optimal", and whether another algorithm could provide a better result. In this paper we provide a simple test to check whether the output of a sparse approximation algorithm is nearly optimal, in the sense that no significantly different linear expansion from the dictionary can provide both a smaller approximation error and a better sparsity. As a by-product of our theorems, we obtain results on the identifiability of sparse overcomplete models in the presence of noise, for a fairly large class of sparse priors.Key-words: sparse representation, approximation, overcomplete dictionary, matching pursuit, basis pursuit, FOCUSS, convex programming, greedy algorithm, identifiability, inverse problem, sparse prior, Bayesian estimation. Mots clés : représentation parcimonieuse, approximation, dictionnaire redondant, matching pursuit, basis pursuit, FOCUSS, programmation convexe, algorithme glouton, identifiabilité, problème inverse, estimation Bayesienne.A simple test to check the optimality of a sparse signal approximation 3

show abstract

Section: Main Concepts and Resultsmentioning

confidence: 99%

Section: Main Concepts and Resultsmentioning

confidence: 99%

“…By [12,Lemma 7], for any admissible sparsity measure h ∈ M, any sequence z = (z k ) and any integer M, we have…”

Section: Lemma 1 If F Is a Sub-additive Sparsity Measure Thenmentioning

confidence: 99%

Section: Lemmamentioning

confidence: 99%

See 2 more Smart Citations

A simple test to check the optimality of a sparse signal approximation

2006

Self Cite

View full text Add to dashboard Cite

show abstract

“…The global optimality of (3) has been studied and various conditions have been derived, for example, those based on restricted isometry property [7][8][9]12] and null space property [10,13]. Among them, a necessary and sufficient condition is based on the null space property and its constant [10,13,14].…”

Section: Introductionmentioning

confidence: 99%

On the Null Space Constant for <formula formulatype="inline"><tex Notation="TeX">${\ell _p}$</tex></formula> Minimization

Chen¹,

Gu²

2015

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

The literature on sparse recovery often adopts the p "norm" (p ∈ [0, 1]) as the penalty to induce sparsity of the signal satisfying an underdetermined linear system. The performance of the corresponding p minimization problem can be characterized by its null space constant. In spite of the NP-hardness of computing the constant, its properties can still help in illustrating the performance of p minimization. In this letter, we show the strict increase of the null space constant in the sparsity level k and its continuity in the exponent p. We also indicate that the constant is strictly increasing in p with probability 1 when the sensing matrix A is randomly generated. Finally, we show how these properties can help in demonstrating the performance of p minimization, mainly in the relationship between the the exponent p and the sparsity level k.

show abstract

Iteratively reweighted least squares minimization for sparse recovery

Daubechies

DeVore

Fornasier

et al. 2009

Comm Pure Appl Math

1,138

892

View full text Add to dashboard Cite

Under certain conditions (known as the Restricted Isometry Property or RIP) on the m × Nmatrix Φ (where m < N ), vectors x ∈ R N that are sparse (i.e. have most of their entries equal to zero) can be recovered exactly from y := Φx even though Φ −1 (y) is typically an (N − m)-dimensional hyperplane; in addition x is then equal to the element in Φ −1 (y) of minimal ℓ 1 -norm. This minimal element can be identified via linear programming algorithms. We study an alternative method of determining x, as the limit of an Iteratively Re-weighted Least Squares (IRLS) algorithm. The main step of this IRLS finds, for a given weight vector w, the element in Φ −1 (y) with smallest ℓ 2 (w)-norm. If x (n) is the solution at iteration step n,, i = 1, . . . , N , for a decreasing sequence of adaptively defined ǫ n ; this updated weight is then used to obtain x (n+1) and the process is repeated. We prove that when Φ satisfies the RIP conditions, the sequence xconverges for all y, regardless of whether Φ −1 (y) contains a sparse vector. If there is a sparse vector in Φ −1 (y), then the limit is this sparse vector, and when x (n) is sufficiently close to the limit, the remaining steps of the algorithm converge exponentially fast (linear convergence in the terminology of numerical optimization). The same algorithm with the "heavier" weight, i = 1, . . . , N , where 0 < τ < 1, can recover sparse solutions as well; more importantly, we show its local convergence is superlinear and approaches a quadratic rate for τ approaching to zero.

show abstract

Highly sparse representations from dictionaries are unique and independent of the sparseness measure

Cited by 145 publications

References 38 publications

A simple test to check the optimality of a sparse signal approximation

A simple test to check the optimality of a sparse signal approximation

On the Null Space Constant for <formula formulatype="inline"><tex Notation="TeX">${\ell _p}$</tex></formula> Minimization

Iteratively reweighted least squares minimization for sparse recovery

Contact Info

Product

Resources

About