Block and Group Regularized Sparse Modeling for Dictionary Learning

Abstract

This paper proposes a dictionary learning framework that combines the proposed block/group (BGSC) or reconstructed block/group (R-BGSC) sparse coding schemes with the novel Intra-block Coherence Suppression Dictionary Learning (ICS-DL) algorithm. An important and distinguishing feature of the proposed framework is that all dictionary blocks are trained simultaneously with respect to each data group while the intra-block coherence being explicitly minimized as an important objective. We provide both empirical evidence and heuristic support for this feature that can be considered as a direct consequence of incorporating both the group structure for the input data and the block structure for the dictionary in the learning process. The optimization problems for both the dictionary learning and sparse coding can be solved efficiently using block-gradient descent, and the details of the optimization algorithms are presented. We evaluate the proposed methods using well-known datasets, and favorable comparisons with state-of-the-art dictionary learning methods demonstrate the viability and validity of the proposed framework.

Discriminative Training of Structured Dictionaries via Block Orthogonal Matching Pursuit

“…It not only enables us to assign each block primarily to a specific class, but also to manipulate the degree of block sharing between different classes. Furthermore, we demonstrate that B-OMP significantly speeds up the sparse coefficients update compared with l 1 /l 2 norm constrained convex relaxation solver such as block/group sparse coding (BGSC) [15].…”

“…atoms in each block can learn more diverse patterns while avoiding redundancy between elements [15]. Adding such a diversity constraint between atoms in the …”

“…Specifically, we apply SB-OMP to find ρ non-class specific blocks and compute S using λ different dictionary blocks in total. Since label is unknown during testing time, we compute the sparse coefficients S for the corrupted testing examples using block sparse coding algorithms, BGSC, as proposed in [15]. Finally, we combine the sparse coefficients S and the learned dictionary D to reconstruct the ground truth with no occlusions.…”

“…In [15], the idea of block structure is explored: their algorithm learns structured dictionaries and utilize an intra-block coherence penalty term (see Section 4.4 for definition). However, [15] does not incorporate the label information.…”

“…We can still obtain a closed-form solution without the unit norm constraint for atom; for more details, please see! [15] (Supplementary Materials, 1.2).…”