Forward sequential algorithms for best basis selection

Abstract

Recently, the problem of signal representation in terms of basis vectors from a large, "overcomplete", spanning dictionary has been the focus of much research. Achieving a succinct, or "sparse", representation is known as the problem of best basis representation. We consider methods which seek to solve this problem by sequentially building up a basis set for the signal. Three distinct algorithm types have appeared in the literature which we term Basic Matching Pursuit (BMP), Order Recursive Matching Pursuit (ORMP) and Modified Matching Pursuit (MMP). The algorithms are first described and then their computation is closely examined. Modifications are made to each of the procedures which improve their computational efficiency. Each algorithm's complexity is considered in two contexts: one where the dictionary is variable (time dependent), and the other where the dictionary is fixed (time independent). Experimental results are presented which demonstrate that the ORMP method is the best procedure in terms of its ability to give the most compact signal representation, followed by

Image compression using Analytical and Learned Dictionaries

“…Then keep the dictionary D fixed and find U according to some sparseness constraint rule. In practice, to find the coefficients 'w', vector selection algorithms such as Matching Pursuit algorithm [11,12], Basic Matching Pursuit (BMP) [13], Orthogonal Matching Pursuit (OMP) [14,15], or Order Recursive Matching Pursuit (ORMP) [16] are used. This stage is computationally intensive, since it needs to update each dictionary, particularly for large training sets.…”

Sparse Approximations for Quaternionic Signals

“…Coding coefficients x k are computed via the orthogonal projection of y on D k (step 8). This is often carried out recursively by different methods using the current correlation value C k m k : QR factorization [7], Cholesky factorization [6], or block matrix inversion [14]. The obtained coefficients vector Different stopping criteria (step 11) can be used: a threshold on k for the number of iterations, a threshold on the relative root MSE (rRMSE) ǫ k 2 / y 2 , or a threshold on the decrease in the rRMSE.…”

Learning Sparse Overcomplete Codes for Images

“…Here, we use modified matching pursuit (MMP) [12] which selects each vector (in series) to minimize the residual representation error. The simpler matching pursuit (MP) algorithm is more computationally efficient, but provides less accurate reconstruction.…”

“…The MP algorithm is more computationally efficient but provides less accurate reconstruction. More details and comparisons can be found in [12,29].…”

“…We present and experimentally compare algorithms for both of these tasks. In Section 2, we discuss sparse coding assuming a known, fixed dictionary using the following algorithms: focalunderdetermined system solver (FOCUSS) [10], sparse Bayesian learning with adjustable-variance Gaussians (SBL-AVG) [11] and modified matching pursuit (MMP) [12]. With earlier algorithms such as PCA, ICA and DCT transforms, finding the coefficients requires only a matrix multiply, however with an overcomplete dictionary the representation of a signal is underdetermined, so an additional criteria such as sparseness must be used.…”