Identifiability in Two-Layer Sparse Matrix Factorization

Abstract

Sparse matrix factorization is the problem of approximating a matrix Z by a product of L sparse factors X (L) X (L−1) . . . X (1) . This paper focuses on identifiability issues that appear in this problem, in view of better understanding under which sparsity constraints the problem is well-posed. We give conditions under which the problem of factorizing a matrix into two sparse factors admits a unique solution, up to unavoidable permutation and scaling equivalences. Our general framework considers an arbitrary family of prescribed sparsity patterns, allowing us to capture more structured notions of sparsity than simply the count of nonzero entries. These conditions are shown to be related to essential uniqueness of exact matrix decomposition into a sum of rank-one matrices, with structured sparsity constraints. A companion paper further exploits these conditions to derive identifiability properties in multilayer sparse matrix factorization of some well-known matrices like the Hadamard or the discrete Fourier transform matrices.

Fast Learning of Fast Transforms, with Guarantees

“…However, there are nontrivial conditions on the supports that ensure both tractability and identifiability of Problem (2), i.e., uniqueness of its solution up to natural scaling ambiguities. 1 We consider XY ⊺ instead of XY for consistency with existing analysis [8,9,10] where it eases the notations without changing the problem.…”

“…Identifiability of Problem (2) can be studied when Z admits an exact factorization Z = XY ⊺ with (X, Y) ∈ Σ S . Following the general framework introduced in [9], we represent a pair (X, Y) by its r-tuple of so-called rank-one contributions…”

“…where M i denotes the i-th column of matrix M. The support constraint S = (S L , S R ) is then represented by the r-tuple of rank-one supports S = (S i ) r i=1 ∶= ϕ(S L , S R ), where the matrix supports S L , S R are identified to their binary matrix representation. With this lifting approach [14,15,16], identifiability of the two factors (X, Y) is shown to be equivalent to identifiability of the rank-one contributions ϕ(X, Y) from their sum, except in trivial degenerate cases [9,10]. Assume now that Z is a sum of rank-one contributions C i :…”

“…Interestingly, SMF with fixed support and J = 2 becomes well-posed [9,10] and tractable [8] under certain assumptions on the prescribed supports of the factors. These analyses have led to a factorization method for this setting that exploits nontrivially the Singular Value Decomposition (SVD) on blocks and computes an optimal factorization.…”

“…Based on the identifiability and recovery guarantees of [8,9,10], the following exact recovery guarantee is ensured: if Z admits an exact butterfly factorization (i.e, Z = X J . .…”