Sparse representations in unions of bases

Gribonval, Rémi; Nielsen, Morten

doi:10.1109/tit.2003.820031

Cited by 838 publications

(814 citation statements)

References 6 publications

Supporting

Mentioning

786

Contrasting

Unclassified

Order By: Relevance

“…Note: After presenting our work publicly we learned of related work about to be published by R. Gribonval and M. Nielsen (18). Their work initially addresses the same question of obtaining the solution of (P 0 ) by instead solving (P 1 ), with results paralleling ours.…”

mentioning

confidence: 93%

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization

Donoho

Elad

2003

Proc. Natl. Acad. Sci. U.S.A.

2,557

1,915

View full text Add to dashboard Cite

Given a dictionary D ‫؍‬ {d ᠪ k} of vectors d ᠪ k, we seek to represent a signal S ᠪ as a linear combination S ᠪ ‫؍‬ ͚ k ␥(k)d ᠪ k, with scalar coefficients ␥(k). In particular, we aim for the sparsest representation possible. In general, this requires a combinatorial optimization process. Previous work considered the special case where D is an overcomplete system consisting of exactly two orthobases and has shown that, under a condition of mutual incoherence of the two bases, and assuming that S ᠪ has a sufficiently sparse representation, this representation is unique and can be found by solving a convex optimization problem: specifically, minimizing the ഞ 1 norm of the coefficients ᠪ ␥. In this article, we obtain parallel results in a more general setting, where the dictionary D can arise from two or several bases, frames, or even less structured systems. We sketch three applications: separating linear features from planar ones in 3D data, noncooperative multiuser encoding, and identification of over-complete independent component models.W orkers throughout engineering and the applied sciences frequently want to represent data (signals, images) in the most parsimonious terms. In signal analysis specifically, they often consider models proposing that the signal of interest is sparse in some transform domain, such as the wavelet or Fourier domain (1). However, there is a growing realization that many signals are mixtures of diverse phenomena, and no single transform can be expected to describe them well; instead, we should consider models making sparse combinations of generating elements from several different transforms (1-4). Unfortunately, as soon as we start considering general collections of generating elements, the attempt to find sparse solutions enters mostly uncharted territory, and one expects at best to use plausible heuristic methods (5-8) and certainly to give up hope of rigorous optimality. In this article, we will develop some rigorous results showing that it can be possible to find optimally sparse representations by efficient techniques in certain cases.Suppose we are given a dictionary D of generating ele-L , each one a vector in C N , which we assume normal-The dictionary D can be viewed a matrix of size N ϫ L, with generating elements for columns. We do not suppose any fixed relationship between N and L. In particular, the dictionary can be overcomplete and contain linearly dependent subsets, and in particular need not be a basis. As examples of such dictionaries, we can mention: wavelet packets and cosine packets dictionaries of Coifman et al. (3), which contain L ϭ N log(N) elements, representing transient harmonic phenomena with a variety of durations and locations; wavelet frames, such as the directional wavelet frames of Ron and Shen (9), which contain L ϭ CN elements for various constants C Ͼ 1; and the combined ridgelet͞wavelet systems of Starck, Candès, and Donoho (8,10). Faced with such variety, we cannot call individual elements in the dictionary basis elements; we will use the term...

show abstract

mentioning

confidence: 93%

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization

Donoho

Elad

2003

Proc. Natl. Acad. Sci. U.S.A.

2,557

1,915

View full text Add to dashboard Cite

show abstract

“…However, the efficiency of a dictionary also critically depends on its size and on the existence of fast operators, without which restoration algorithms (that are iterative) cannot converge in a reasonable time. Concatenation or unions of representation spaces are now classically used in denoising and inverse problems because they can account for more complex morphological features better than standard transforms used separately (an approach advocated early in Mallat & Zhang 1993;and Chen et al 1998; see also Donoho & Huo 2001;Gribonval & Nielsen 2003;Starck et al 2010). Such unions may allow maintaining a reasonable computational cost if fast transforms are associated to each representation space.…”

Section: Representations and Dictionariesmentioning

confidence: 99%

MORESANE: MOdel REconstruction by Synthesis-ANalysis Estimators

Dabbech

Ferrari

Mary

et al. 2015

A&A

View full text Add to dashboard Cite

Context. Recent years have been seeing huge developments of radio telescopes and a tremendous increase in their capabilities (sensitivity, angular and spectral resolution, field of view, etc.). Such systems make designing more sophisticated techniques mandatory not only for transporting, storing, and processing this new generation of radio interferometric data, but also for restoring the astrophysical information contained in such data. Aims. In this paper we present a new radio deconvolution algorithm named MORESANE and its application to fully realistic simulated data of MeerKAT, one of the SKA precursors. This method has been designed for the difficult case of restoring diffuse astronomical sources that are faint in brightness, complex in morphology, and possibly buried in the dirty beam's side lobes of bright radio sources in the field. Methods. MORESANE is a greedy algorithm that combines complementary types of sparse recovery methods in order to reconstruct the most appropriate sky model from observed radio visibilities. A synthesis approach is used for reconstructing images, in which the synthesis atoms representing the unknown sources are learned using analysis priors. We applied this new deconvolution method to fully realistic simulations of the radio observations of a galaxy cluster and of an HII region in M 31. Results. We show that MORESANE is able to efficiently reconstruct images composed of a wide variety of sources (compact pointlike objects, extended tailed radio galaxies, low-surface brightness emission) from radio interferometric data. Comparisons with the state of the art algorithms indicate that MORESANE provides competitive results in terms of both the total flux/surface brightness conservation and fidelity of the reconstructed model. MORESANE seems particularly well suited to recovering diffuse and extended sources, as well as bright and compact radio sources known to be hosted in galaxy clusters.

show abstract

“…As it is now well understood, if the k-sparse synthesis model holds for sufficiently small k (we will also say that x 0 is synthesis-sparse or simply sparse), then the optimization problem (3) with τ = 1 indeed allows the recovery of x 0 by convex optimization ( [5], [6]), which has bounded complexity. Just as for the sparse synthesis data model, one can hope to estimate x 0 using (4) only if x 0 is 'sufficiently cosparse'.…”

Section: Cosparse Analysis Data Modelmentioning

confidence: 99%

Cosparse analysis modeling - uniqueness and algorithms

Nam

Davies²,

Elad

et al. 2011

2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In the past decade there has been a great interest in a synthesis-based model for signals, based on sparse and redundant representations. Such a model assumes that the signal of interest can be composed as a linear combination of few columns from a given matrix (the dictionary). An alternative analysis-based model can be envisioned, where an analysis operator multiplies the signal, leading to a cosparse outcome. In this paper, we consider this analysis model, in the context of a generic missing data problem (e.g., compressed sensing, inpainting, source separation, etc.). Our work proposes a uniqueness result for the solution of this problem, based on properties of the analysis operator and the measurement matrix. This paper also considers two pursuit algorithms for solving the missing data problem, an L1-based and a new greedy method. Our simulations demonstrate the appeal of the analysis model, and the success of the pursuit techniques presented.

show abstract

Sparse representations in unions of bases

Cited by 838 publications

References 6 publications

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization

MORESANE: MOdel REconstruction by Synthesis-ANalysis Estimators

Cosparse analysis modeling - uniqueness and algorithms

Contact Info

Product

Resources

About

Sparse representations in unions of bases

Cited by 838 publications

References 6 publications

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ 1 minimization

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ 1 minimization

MORESANE: MOdel REconstruction by Synthesis-ANalysis Estimators

Cosparse analysis modeling - uniqueness and algorithms

Contact Info

Product

Resources

About

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization

Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization