Nonnegative matrix factorization using ADMM: Algorithm and convergence analysis

Hajinezhad, Davood; Chang, Tsung-Hui; Wang, Xiangfeng; Shi, Qingjiang; Mei, Hong

doi:10.1109/icassp.2016.7472577

Cited by 64 publications

(36 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ADMM is a popular solver for optimization problems with separable target functions and linear side constraints [Boyd et al 2011]. Using auxiliary variables and indicator functions, such formulation allows for non-smooth optimization with hard constraints, with wide applications in signal processing [Erseghe et al 2011;Simonetto and Leus 2014;Shi et al 2014], image processing [Figueiredo and Bioucas-Dias 2010;Almeida and Figueiredo 2013], computer vision [Hu et al 2013;Yang et al 2017], computational imaging [Chan et al 2017], automatic control [Lin et al 2013], and machine learning [Zhang and Kwok 2014;Hajinezhad et al 2016]. ADMM has also been used in computer graphics to handle non-smooth optimization problems [Bouaziz et al 2013;Neumann et al 2013;Xiong et al 2014;Neumann et al 2014] or to benefit from its fast initial convergence [Gregson et al 2014;Heide et al 2016;Xiong et al 2017;Pan and Manocha 2017;Wang et al 2018].…”

Section: Related Workmentioning

confidence: 99%

Accelerating ADMM for efficient simulation and optimization

et al. 2019

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Accelerating ADMM for efficient simulation and optimization

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The central objects of this paper are multilinear and multiaffine maps, which generalize linear and affine maps. 1 [23] shows that every limit point of ADMM for the problem (NMF) is a constrained stationary point, but does not show that such limit points necessarily exist.…”

Section: Multiaffine Constrained Problemsmentioning

confidence: 97%

“…by applying ADMM with alternating minimization on the blocks Y and (X, Z). The convergence of ADMM employed to solve the (NMF1) problem appears to have been an open question until a proof was given in [23] 1 . A method derived from ADMM has also been proposed for optimizing a biaffine model for training deep neural networks [53].…”

Section: Algorithm 1 Admmmentioning

confidence: 99%

ADMM for multiaffine constrained optimization

Gao

Goldfarb

Curtis

2019

Optimization Methods and Software

View full text Add to dashboard Cite

We expand the scope of the alternating direction method of multipliers (ADMM). Specifically, we show that ADMM, when employed to solve problems with multiaffine constraints that satisfy certain verifiable assumptions, converges to the set of constrained stationary points if the penalty parameter in the augmented Lagrangian is sufficiently large. When the Kurdyka-Lojasiewicz (K-L) property holds, this is strengthened to convergence to a single constrained stationary point. Our analysis applies under assumptions that we have endeavored to make as weak as possible. It applies to problems that involve nonconvex and/or nonsmooth objective terms, in addition to the multiaffine constraints that can involve multiple (three or more) blocks of variables. To illustrate the applicability of our results, we describe examples including nonnegative matrix factorization, sparse learning, risk parity portfolio selection, nonconvex formulations of convex problems, and neural network training. In each case, our ADMM approach encounters only subproblems that have closed-form solutions.

show abstract

“…where ν k = maximum row sum of 2H k+1 H k+1 T . Note that once H k+1 and W k+1 are computed using (26) and (29), the negative elements must be projected back to R + , to satisfy the nonnegative constraint. Also, observe that the update equations looks similar to gradient descent algorithm with "adaptive" step sizes equal to µ k and ν k .…”

Section: Proofmentioning

confidence: 99%

“…hence is computationally faster than HALS algorithm. Recently, Alternating Direction Method of Multipliers (ADMM) was also used to solve the NMF problem [26], [27]. Vandaele et.al.…”

Section: Introductionmentioning

confidence: 99%

Novel Algorithms Based on Majorization Minimization for Nonnegative Matrix Factorization

2019

View full text Add to dashboard Cite

Matrix decomposition is ubiquitous and has applications in various fields like speech processing, data mining and image processing to name a few. Under matrix decomposition, nonnegative matrix factorization is used to decompose a nonnegative matrix into a product of two nonnegative matrices which gives some meaningful interpretation of the data. Thus, nonnegative matrix factorization has an edge over the other decomposition techniques. In this paper, we propose two novel iterative algorithms based on Majorization Minimization (MM)-in which we formulate a novel upper bound and minimize it to get a closed form solution at every iteration. Since the algorithms are based on MM, it is ensured that the proposed methods will be monotonic. The proposed algorithms differ in the updating approach of the two nonnegative matrices. The first algorithm-Iterative Nonnegative Matrix Factorization (INOM) sequentially updates the two nonnegative matrices while the second algorithm-Parallel Iterative Nonnegative Matrix Factorization (PARINOM) parallely updates them. We also prove that the proposed algorithms converge to the stationary point of the problem. Simulations were conducted to compare the proposed methods with the existing ones and was found that the proposed algorithms performs better than the existing ones in terms of computational speed and convergence.

show abstract

Nonnegative matrix factorization using ADMM: Algorithm and convergence analysis

Cited by 64 publications

References 8 publications

Accelerating ADMM for efficient simulation and optimization

Accelerating ADMM for efficient simulation and optimization

ADMM for multiaffine constrained optimization

Novel Algorithms Based on Majorization Minimization for Nonnegative Matrix Factorization

Contact Info

Product

Resources

About