Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization

Abstract

Given a dictionary D ‫؍‬ {d ᠪ k} of vectors d ᠪ k, we seek to represent a signal S ᠪ as a linear combination S ᠪ ‫؍‬ ͚ k ␥(k)d ᠪ k, with scalar coefficients ␥(k). In particular, we aim for the sparsest representation possible. In general, this requires a combinatorial optimization process. Previous work considered the special case where D is an overcomplete system consisting of exactly two orthobases and has shown that, under a condition of mutual incoherence of the two bases, and assuming that S ᠪ has a sufficiently sparse representation, this representation is unique and can be found by solving a convex optimization problem: specifically, minimizing the ഞ 1 norm of the coefficients ᠪ ␥. In this article, we obtain parallel results in a more general setting, where the dictionary D can arise from two or several bases, frames, or even less structured systems. We sketch three applications: separating linear features from planar ones in 3D data, noncooperative multiuser encoding, and identification of over-complete independent component models.W orkers throughout engineering and the applied sciences frequently want to represent data (signals, images) in the most parsimonious terms. In signal analysis specifically, they often consider models proposing that the signal of interest is sparse in some transform domain, such as the wavelet or Fourier domain (1). However, there is a growing realization that many signals are mixtures of diverse phenomena, and no single transform can be expected to describe them well; instead, we should consider models making sparse combinations of generating elements from several different transforms (1-4). Unfortunately, as soon as we start considering general collections of generating elements, the attempt to find sparse solutions enters mostly uncharted territory, and one expects at best to use plausible heuristic methods (5-8) and certainly to give up hope of rigorous optimality. In this article, we will develop some rigorous results showing that it can be possible to find optimally sparse representations by efficient techniques in certain cases.Suppose we are given a dictionary D of generating ele-L , each one a vector in C N , which we assume normal-The dictionary D can be viewed a matrix of size N ϫ L, with generating elements for columns. We do not suppose any fixed relationship between N and L. In particular, the dictionary can be overcomplete and contain linearly dependent subsets, and in particular need not be a basis. As examples of such dictionaries, we can mention: wavelet packets and cosine packets dictionaries of Coifman et al. (3), which contain L ϭ N log(N) elements, representing transient harmonic phenomena with a variety of durations and locations; wavelet frames, such as the directional wavelet frames of Ron and Shen (9), which contain L ϭ CN elements for various constants C Ͼ 1; and the combined ridgelet͞wavelet systems of Starck, Candès, and Donoho (8,10). Faced with such variety, we cannot call individual elements in the dictionary basis elements; we will use the term...

Extraction of multiple pure component 1H and 13C NMR spectra from two mixtures: Novel solution obtained by sparse component analysis-based blind decomposition

“…The reason is that even when A is known solution of the linear system of equations (1) or (7) is not unique, because there are more unknowns (M) than equations (N). If pure components are (M-N+1)-sparse, a unique solution is obtained at the minimum of the 1  norm of s, [24][25][26][27][30][31][32][33][34]. We could formulate linear programming based solution in the time-scale basis (7).…”

“…The SCA method in [8] solves BSS problem by finding de-mixing matrix W by minimizing cost function that measures sparseness of the sources, however it still requires N=M. On the other side the SCA approach used here, and referred in [24][25][26][27], breaks down BSS problem into two separate problems: estimation of the mixing or concentration matrix A using geometric concept known as data clustering [24][25][26][27][28][29], and estimation of the magnitude spectra of the pure components (based on estimated A) by solving resulting underdetermined system of linear equations through linear programming [24,25,30,31], 1  -regularized least square problem [32,33] or 2  -regularized linear problem, [34]. In the case of the NMR spectroscopy it is customary to assume that Fourier basis yields sparse representation, however wavelet basis with properly chosen wavelet function can yield even sparser representation.…”

Sparse Representations and Invariant Sequence-Feature Extraction for Event Detection

“…In practice, of course, questions about the required amount of sparsity and the uniqueness of the sparsest solution arise. Donoho and Elad (2003) showed that if some x with less than m 2 nonzero entries verifies y = Tx, then this is the unique sparsest solution. This means that we have a good chance of finding the correct and unique sparsest solution to (1) even for two-class problems or for configurations in which the number of training vectors per class is not the same over all classes, provided we have enough vectors in the training set.…”

Improving the image quality in compressed sensing MRI by the exploitation of data properties

“…As there are many equivalent ways to characterize the property, we introduce the most common one: spark [31].…”

“…Theorem 2.1 [31]: For any vector ∈ , there exists at most one solution ∈ ∑ , such that = if and only if ( ) > 2s.…”