Efficient and Non-Convex Coordinate Descent for Symmetric Nonnegative Matrix Factorization

Vandaele, Arnaud; Gillis, Nicolas; Lei, Qi; Zhong, Kai; Dhillon, Inderjit S.

doi:10.1109/tsp.2016.2591510

“…where e 0 is the initial error (see Section 4.1), and e(t) is the error X −X F achieved by an algorithm for a given initialization within t seconds. Since our algorithms are nonincreasing, we have E(t) ∈ [0, 1] for all t, with m n k slack matrix of the 12-gon 12 12 5 slack matrix of the 16-gon 16 16 5 slack matrix of the 20-gon 20 20 6 slack matrix of the 24-gon 24 24 6 slack matrix of the 28-gon 28 28 E(0) = 1 and E(t) → t→∞ 0 if the corresponding algorithm converges towards an exact factorization. In order to illustrate the efficiency of a given algorithm, (11) has the advantage that it makes sense to take the average of E(t) for several initializations and data sets and display a single curve.…”

Section: Comparisons For Different Values Of the Parametersmentioning

confidence: 99%

Algorithms for positive semidefinite factorization

Vandaele

¹

,

Glineur

²

,

Gillis

³

2018

Self Cite

View full text Add to dashboard Cite

This paper considers the problem of positive semidefinite factorization (PSD factorization), a generalization of exact nonnegative matrix factorization. Given an m-by-n nonnegative matrix X and an integer k, the PSD factorization problem consists in finding, if possible, symmetric k-by-k positive semidefinite matrices {A 1 , ..., A m } and {B 1 , ..., B n } such that Xi,j = trace(A i B j ) for i = 1, ..., m, and j = 1, ..., n. PSD factorization is NP-hard. In this work, we introduce several local optimization schemes to tackle this problem: a fast projected gradient method and two algorithms based on the coordinate descent framework. The main application of PSD factorization is the computation of semidefinite extensions, that is, the representations of polyhedrons as projections of spectrahedra, for which the matrix to be factorized is the slack matrix of the polyhedron. We compare the performance of our algorithms on this class of problems. In particular, we compute the PSD extensions of size k = 1 + log 2 (n) for the regular n-gons when n = 5, 8 and 10. We also show how to generalize our algorithms to compute the square root rank (which is the size of the factors in a PSD factorization where all factor matrices A i and B j have rank one) and completely PSD factorizations (which is the special case where the input matrix is symmetric and equality A i = B i is required for all i).

show abstract

“…However, CD may not converge to the set of KKT points of SymNMF. Instead, there is an additional NS-SymNMF PGD [22] PNewton [22] ANLS [11] SNMF [10] CD [23] (a) N = 500, K = 60.…”

Section: Numerical Resultsmentioning

confidence: 99%

A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality

Lu

¹

,

Mei

²

,

Wang

³

2017

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Symmetric nonnegative matrix factorization (SymNMF) has important applications in data analytics problems such as document clustering, community detection, and image segmentation. In this paper, we propose a novel nonconvex variable splitting method for solving SymNMF. The proposed algorithm is guaranteed to converge to the set of Karush-Kuhn-Tucker (KKT) points of the nonconvex SymNMF problem. Furthermore, it achieves a global sublinear convergence rate. We also show that the algorithm can be efficiently implemented in parallel. Further, sufficient conditions are provided that guarantee the global and local optimality of the obtained solutions. Extensive numerical results performed on both synthetic and real datasets suggest that the proposed algorithm converges quickly to a local minimum solution. KeywordsSymmetric nonnegative matrix factorization, Karush-Kuhn-Tucker points, variable splitting, global and local optimality, clustering Abstract-Symmetric nonnegative matrix factorization (SymNMF) has important applications in data analytics problems such as document clustering, community detection and image segmentation. In this paper, we propose a novel nonconvex variable splitting method for solving SymNMF. The proposed algorithm is guaranteed to converge to the set of Karush-Kuhn-Tucker (KKT) points of the nonconvex SymNMF problem. Furthermore, it achieves a global sublinear convergence rate. We also show that the algorithm can be efficiently implemented in parallel. Further, sufficient conditions are provided which guarantee the global and local optimality of the obtained solutions. Extensive numerical results performed on both synthetic and real data sets suggest that the proposed algorithm converges quickly to a local minimum solution.

show abstract

“…condition given in [23] for checking whether the generated sequence converges to a unique limit point. A heuristic method for checking the condition is additionally provided, which requires, e.g., plotting the norm between the different iterates.…”

Section: Numerical Resultsmentioning

confidence: 99%

“…We remark that the previous work [23] has made the observation that solving SymNMF with the additional constraints X i 2 ≤ 2 Z F , ∀i will not result in any loss of the global optimality. Lemma 2 provides a stronger result, that all KKT points of SymNMF are preserved within a smaller…”

Section: The Proposed Algorithmmentioning

confidence: 96%

A nonconvex splitting method for symmetric nonnegative matrix factorization: Convergence analysis and optimality

Mei

¹

,

Wang

²

2017

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Symmetric nonnegative matrix factorization (SymNMF) has important applications in data analytics problems such as document clustering, community detection, and image segmentation. In this paper, we propose a novel nonconvex variable splitting method for solving SymNMF. The proposed algorithm is guaranteed to converge to the set of Karush-Kuhn-Tucker (KKT) points of the nonconvex SymNMF problem. Furthermore, it achieves a global sublinear convergence rate. We also show that the algorithm can be efficiently implemented in parallel. Further, sufficient conditions are provided that guarantee the global and local optimality of the obtained solutions. Extensive numerical results performed on both synthetic and real datasets suggest that the proposed algorithm converges quickly to a local minimum solution. KeywordsSymmetric nonnegative matrix factorization, Karush-Kuhn-Tucker points, variable splitting, global and local optimality, clustering Abstract-Symmetric nonnegative matrix factorization (SymNMF) has important applications in data analytics problems such as document clustering, community detection and image segmentation. In this paper, we propose a novel nonconvex variable splitting method for solving SymNMF. The proposed algorithm is guaranteed to converge to the set of Karush-Kuhn-Tucker (KKT) points of the nonconvex SymNMF problem. Furthermore, it achieves a global sublinear convergence rate. We also show that the algorithm can be efficiently implemented in parallel. Further, sufficient conditions are provided which guarantee the global and local optimality of the obtained solutions. Extensive numerical results performed on both synthetic and real data sets suggest that the proposed algorithm converges quickly to a local minimum solution.

show abstract

Efficient and Non-Convex Coordinate Descent for Symmetric Nonnegative Matrix Factorization

Cited by 43 publications

References 31 publications

Algorithms for positive semidefinite factorization

Algorithms for positive semidefinite factorization

A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality

A nonconvex splitting method for symmetric nonnegative matrix factorization: Convergence analysis and optimality

Contact Info

Product

Resources

About