Analysis SimCO Algorithms for Sparse Analysis Model Based Dictionary Learning

Abstract

Abstract-In this paper, we consider the dictionary learning problem for the sparse analysis model. A novel algorithm is proposed by adapting the simultaneous codeword optimization (SimCO) algorithm, based on the sparse synthesis model, to the sparse analysis model. This algorithm assumes that the analysis dictionary contains unit ℓ2-norm atoms and learns the dictionary by optimization on manifolds. This framework allows multiple dictionary atoms to be updated simultaneously in each iteration. However, similar to several existing analysis dictionary learning algorithms, dictionaries learned by the proposed algorithm may contain similar atoms, leading to a degenerate (coherent) dictionary. To address this problem, we also consider restricting the coherence of the learned dictionary and propose Incoherent Analysis SimCO by introducing an atom decorrelation step following the update of the dictionary. We demonstrate the competitive performance of the proposed algorithms using experiments with synthetic data and image denoising as compared with existing algorithms.

Online and Stable Learning of Analysis Operators

“…It shows that the sequence of values of the target function v k = f (Γ (k) ) converges. This, however, does not imply convergence of the algorithm as suggested in [3], at least not in the sense that the sequence Γ (k) converges. Indeed the sequence Γ (k) could orbit around the set L = {Γ ∈ A : f (Γ) = v}, where v = lim k→∞ v k .…”

“…Apart from additional side constraints on Γ, such as incoherence, the optimisation program above has already been used successfully as starting point for the development of two analysis operator learning algorithms, Analysis K-SVD [15] and Analysis SimCO [2], [3]. AKSVD is an alternating minimisation algorithm, which alternates between finding the best X ∈ X for the current Γ and updating Γ based on the current X.…”

“…Now the decrease of the target function can be guaranteed if not all rows of the current iterate Γ are already the eigenvectors corresponding to the smallest eigenvalues of the matrices A k . 3 Proof: To show this, denote by (A k ) k∈ [K] and (Ā k ) k∈[K] the matrices defined in Table 5 for the operators Γ andΓ, respectively. Further denote by (σ k ) k∈ [K] and (σ k ) k∈[K] their smallest singular values.…”

“…Analysis K-SVD [15] uses an analysis sparse coding step to find the cosupports for the given data and then computes the singular vector corresponding to the smallest singular value of a matrix computed from the data. Finally, in [2], [3] a projected gradient descent based algorithm with line search, called Analysis SimCO, is presented. There, the only restriction on the analysis operator is that its rows are normalised and the target function enforces sparsity of ΩY .…”

Analysis Dictionary Learning Based Classification: Structure for Robustness

“…Zhang et al [21] use Recursive Least Square method to accelerate dictionary learning by updating a dictionary based on the dictionaries in the previous iterations. Li et al [22] and Dong et al [23] proposed Simultaneous codeword optimization (SimCo) related algorithms to update multiple atoms in each iteration and by adding an extra incoherent regularity term to avoid linear dependency among dictionary atoms. On other hand, Li et al [24], [25] used l 1/2 norm instead of l 1 norm to have stronger sparsity and mathematically guaranteed a strong convergence.…”