Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications

Wen, Bihan; Ravishankar, Saiprasad; Bresler, Yoram

doi:10.1007/s11263-014-0761-1

Cited by 158 publications

(172 citation statements)

References 55 publications

Supporting

Mentioning

161

Contrasting

Order By: Relevance

“…The chosen k0 is the one that provides the smallest individual sparsification (at sparsity s) error (i.e., solves (P1)). Thus, the optimal sparse code(s) in (P1) is equal to the block(s) of the optimalx in (P2) satisfying x k 0 ≤ s. The full proof of this result is presented elsewhere [21]. The preceding arguments establish the equivalence between the union-of-transforms model and the corresponding OCTOBOS model.…”

Section: Octobos Model and Its Learningmentioning

confidence: 74%

See 1 more Smart Citation

Learning overcomplete sparsifying transforms with block cosparsity

Wen

Ravishankar

Bresler

2014

2014 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

The sparsity of images in a transform domain or dictionary has been widely exploited in image processing. Compared to the synthesis dictionary model, sparse coding in the (single) transform model is cheap. However, natural images typically contain diverse textures that cannot be sparsified well by a single transform. Hence, we propose a union of sparsifying transforms model, which is equivalent to an overcomplete transform model with block cosparsity (OC-TOBOS). Our alternating algorithm for transform learning involves simple closed-form updates. When applied to images, our algorithm learns a collection of well-conditioned transforms, and a good clustering of the patches or textures. Our learnt transforms provide better image representations than learned square transforms. We also show the promising denoising performance and speedups provided by the proposed method compared to synthesis dictionary-based denoising.

show abstract

Section: Octobos Model and Its Learningmentioning

confidence: 74%

“…The objective in our algorithm being monotone decreasing and lower bounded [21], it converges. The computational cost per iteration (of sparse coding and clustering, and transform update) for learning an m × n (m = Kn) OCTOBOS transform using our algorithm scales as O(mnN ).…”

Section: Transform Update Stepmentioning

confidence: 89%

Learning overcomplete sparsifying transforms with block cosparsity

Wen

Ravishankar

Bresler

2014

2014 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The proposed SOUP-DILLO (dictionary-blind) image reconstruction method outperformed standard benchmarks involving the K-SVD algorithm, as well as some other recent methods in the compressed sensing MRI application. Recent works have investigated the data-driven adaptation of alternative signal models such as the analysis dictionary [14] or transform model [4], [15], [16], [56]. While we focused on synthesis dictionary learning methodologies in this work, we plan to compare various kinds of data-driven models in future work.…”

Section: Discussionmentioning

confidence: 99%

“…This is different from the motivation for multi-class models such as in [16], [56] (or [11], [18]), where patches from different regions of an image are assumed to contain different “types” of features or textures or edges, and thus common sub-dictionaries or sub-transforms are learned for groups of patches with similar features. …”

mentioning

confidence: 95%

Efficient Sum of Outer Products Dictionary Learning (SOUP-DIL) and Its Application to Inverse Problems

Ravishankar

Nadakuditi

Fessler

2017

IEEE Trans. Comput. Imaging

Self Cite

View full text Add to dashboard Cite

The sparsity of signals in a transform domain or dictionary has been exploited in applications such as compression, denoising and inverse problems. More recently, data-driven adaptation of synthesis dictionaries has shown promise compared to analytical dictionary models. However, dictionary learning problems are typically non-convex and NP-hard, and the usual alternating minimization approaches for these problems are often computationally expensive, with the computations dominated by the NP-hard synthesis sparse coding step. This paper exploits the ideas that drive algorithms such as K-SVD, and investigates in detail efficient methods for aggregate sparsity penalized dictionary learning by first approximating the data with a sum of sparse rank-one matrices (outer products) and then using a block coordinate descent approach to estimate the unknowns. The resulting block coordinate descent algorithms involve efficient closed-form solutions. Furthermore, we consider the problem of dictionary-blind image reconstruction, and propose novel and efficient algorithms for adaptive image reconstruction using block coordinate descent and sum of outer products methodologies. We provide a convergence study of the algorithms for dictionary learning and dictionary-blind image reconstruction. Our numerical experiments show the promising performance and speedups provided by the proposed methods over previous schemes in sparse data representation and compressed sensing-based image reconstruction.

show abstract

“…In particular, the sparse coding technique has received much attention in recent years. It has shown flexibility and capability in many applications, such as image denoising [7], [8], image super-resolution [9] and face recognition [10]. In these tasks, the input signal (e.g., image or patch) is represented as a sparse linear combination of the bases in a dictionary.…”

Section: Introductionmentioning

confidence: 99%

Multi-Level Discriminative Dictionary Learning With Application to Large Scale Image Classification

Shen

Sun

Huang

et al. 2015

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

The sparse coding technique has shown flexibility and capability in image representation and analysis. It is a powerful tool in many visual applications. Some recent work has shown that incorporating the properties of task (such as discrimination for classification task) into dictionary learning is effective for improving the accuracy. However, the traditional supervised dictionary learning methods suffer from high computation complexity when dealing with large number of categories, making them less satisfactory in large scale applications. In this paper, we propose a novel multi-level discriminative dictionary learning method and apply it to large scale image classification. Our method takes advantage of hierarchical category correlation to encode multi-level discriminative information. Each internal node of the category hierarchy is associated with a discriminative dictionary and a classification model. The dictionaries at different layers are learnt to capture the information of different scales. Moreover, each node at lower layers also inherits the dictionary of its parent, so that the categories at lower layers can be described with multi-scale information. The learning of dictionaries and associated classification models is jointly conducted by minimizing an overall tree loss. The experimental results on challenging data sets demonstrate that our approach achieves excellent accuracy and competitive computation cost compared with other sparse coding methods for large scale image classification.

show abstract

Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications

Cited by 158 publications

References 55 publications

Learning overcomplete sparsifying transforms with block cosparsity

Learning overcomplete sparsifying transforms with block cosparsity

Efficient Sum of Outer Products Dictionary Learning (SOUP-DIL) and Its Application to Inverse Problems

Multi-Level Discriminative Dictionary Learning With Application to Large Scale Image Classification

Contact Info

Product

Resources

About