Signal processing with the sparseness constraint

Rao, Bhaskar D.

doi:10.1109/icassp.1998.681826

Cited by 78 publications

(75 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, there has been a great deal of interest in obtaining sparse codings of y with this procedure for the overcomplete (n > m) case (Mallat & Zhang, 1993;Field, 1994). In our earlier work, we have shown that given an overcomplete dictionary, A (with the columns of A comprising the dictionary vectors), a maximum a posteriori (MAP) estimate of the source vector, x, will yield a sparse coding of y in the low-noise limit if the negative log prior, −log (P(x)), is concave/Schur-concave (CSC) (Rao, 1998;, as discussed below. For P(x) factorizable into a product of marginal probabilities, the resulting code is also known to provide an independent component analysis (ICA) representation of y.…”

Section: Stochastic Modelsmentioning

confidence: 99%

“…FOCUSS, which stands for FOCal Underdetermined System Solver, is an algorithm designed to obtain suboptimally (and, at times, maximally) sparse solutions to the following m × n, underdetermined linear inverse problem 1 (Gorodnitsky, George, & Rao, 1995;Adler, Rao, & Kreutz-Delgado, 1996;Rao, 1997Rao, ,1998, (1.1) for known A. The sparsity of a vector is the number of zero-valued elements (Donoho, 1994), and is related to the diversity, the number of nonzero elements, Since our initial investigations into the properties of FOCUSS as an algorithm for providing sparse solutions to linear inverse problems in relatively noise-free environments (Gorodnitsky et al, 1995;Rao, 1997;Adler et al, 1996;, we now better understand the behavior of FOCUSS in noisy environments and as an interior point-like optimization algorithm for optimizing concave functionals subject to linear constraints , 1998c, 1999KreutzDelgado, Rao, Engan, Lee, & Sejnowski, 1999a;Engan, Rao, & Kreutz-Delgado, 2000;Rao, Engan, Cotter, & Kreutz-Delgado, 2002).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dictionary Learning Algorithms for Sparse Representation

et al. 2003

View full text Add to dashboard Cite

Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations.Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an over-complete

show abstract

Section: Stochastic Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Dictionary Learning Algorithms for Sparse Representation

et al. 2003

View full text Add to dashboard Cite

show abstract

“…References [3,13,4] have proposed the use of the Shannon entropy function as a measure of diversity appropriate for sparse basis selection. Given a probability distribution, the i logx i ;x = xx 0 ; (10) the differences arise in how one definesx as a function of x. These differences affect the properties of HS as a function of x.…”

Section: Entropy Measuresmentioning

confidence: 99%

“…There has been considerable recent interest in the issue of best basis selection for sparse signal representation, including approaches that select basis vectors by minimizing diversity measures subject to the constraint Ax = b; (1) where A is an m n matrix formed using the vectors from an overdetermined dictionary of basis vectors, m n , and it is assumed that rankA = m [3,13,1,10].…”

Section: Introductionmentioning

confidence: 99%

Measures and algorithms for best basis selection

Kreutz-Delgado

Rao

Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181

Self Cite

View full text Add to dashboard Cite

A general framework based on majorization, Schur-concavity, and concavity is given that facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed diversity measures useful for best basis selection. Admissible sparsity measures are given by the Schur-concave functions, which are the class of functions consistent with the partial ordering on vectors known as majorization. Concave functions form an important subclass of the Schur-concave functions which attain their minima at sparse solutions to the basis selection problem. Based on a particular functional factorization of the gradient, we give a general affine scaling optimization algorithm that converges to a sparse solution for measures chosen from within this subclass.

show abstract

“…Matching pursuit algorithm is a method that has been used in many sparse applications [5], [6], [7]. Recently, a parametric method for selecting the structure of sparse model is proposed by Stoica [8].…”

Section: Introductionmentioning

confidence: 99%

Sparse channel estimation with regularization method using convolution inequality for entropy

Han

Kim

Prı́ncipe

Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005.

View full text Add to dashboard Cite

Abstract-In this paper, we show that the sparse channel estimation problem can be formulated as a regularization problem between mean squared error(MSE) and the L 1 -norm constraint of the channel impulse response. A simple adaptive method to solve regularization problem using the convolution inequality for entropy is proposed. Performance of this proposed regularization method will be compared to the Wiener filter, the matching pursuit (MP) algorithm and the information criterion based method. The results show that the estimate of the sparse channel using the MSE criterion with the L1-norm constraint outperforms the Wiener filter and the conventional sparse solution methods in terms of MSE of the estimates and the generalization performance.

show abstract

Signal processing with the sparseness constraint

Cited by 78 publications

References 25 publications

Dictionary Learning Algorithms for Sparse Representation

Dictionary Learning Algorithms for Sparse Representation

Measures and algorithms for best basis selection

Sparse channel estimation with regularization method using convolution inequality for entropy

Contact Info

Product

Resources

About