Efficient Identification of Butterfly Sparse Matrix Factorizations

Abstract

Many well-known matrices Z are associated to fast transforms corresponding to factorizations of the form Z = X (L) . . . X (1) , where each factor X ( ) is sparse. Based on general result for the case with two factors, established in a companion paper, we investigate essential uniqueness of such factorizations. We show some identifiability results for the sparse factorization into two factors of the discrete Fourier Transform, discrete cosine transform or discrete sine transform matrices of size N = 2 L , when enforcing N 2 -sparsity by column on the left factor, and 2-sparsity by row on the right factor. We also show that the analysis with two factors can be extended to the multilayer case, based on a hierarchical factorization method. We prove that any matrix which is the product of L factors whose supports are exactly the so-called butterfly supports, admits a unique sparse factorization into L factors. This applies in particular to the Hadamard or the discrete Fourier transform matrix of size 2 L .

Fast Learning of Fast Transforms, with Guarantees

“…However, there are nontrivial conditions on the supports that ensure both tractability and identifiability of Problem (2), i.e., uniqueness of its solution up to natural scaling ambiguities. 1 We consider XY ⊺ instead of XY for consistency with existing analysis [8,9,10] where it eases the notations without changing the problem.…”

“…where M i denotes the i-th column of matrix M. The support constraint S = (S L , S R ) is then represented by the r-tuple of rank-one supports S = (S i ) r i=1 ∶= ϕ(S L , S R ), where the matrix supports S L , S R are identified to their binary matrix representation. With this lifting approach [14,15,16], identifiability of the two factors (X, Y) is shown to be equivalent to identifiability of the rank-one contributions ϕ(X, Y) from their sum, except in trivial degenerate cases [9,10]. Assume now that Z is a sum of rank-one contributions C i :…”

“…Interestingly, SMF with fixed support and J = 2 becomes well-posed [9,10] and tractable [8] under certain assumptions on the prescribed supports of the factors. These analyses have led to a factorization method for this setting that exploits nontrivially the Singular Value Decomposition (SVD) on blocks and computes an optimal factorization.…”

“…This structure appears ubiquitously in fast transforms such as the Discrete Fourier Transform (DFT) or the Hadamard Transform (HT), hence the algorithm allows to approximate a given matrix by a learned fast transform. The algorithm incorporates the two-factor, SVD-based, algorithm from [8] into a generalized version of a hierarchical greedy sparse factorization strategy [5,10].…”

“…Based on the identifiability and recovery guarantees of [8,9,10], the following exact recovery guarantee is ensured: if Z admits an exact butterfly factorization (i.e, Z = X J . .…”