Fast bregman divergence NMF using taylor expansion and coordinate descent

Abstract

Non-negative matrix factorization (NMF) provides a lower rank approximation of a matrix. Due to nonnegativity imposed on the factors, it gives a latent structure that is often more physically meaningful than other lower rank approximations such as singular value decomposition (SVD). Most of the algorithms proposed in literature for NMF have been based on minimizing the Frobenius norm. This is partly due to the fact that the minimization problem based on the Frobenius norm provides much more flexibility in algebraic manipulation than other divergences. In this paper we propose a fast NMF algorithm that is applicable to general Bregman divergences. Through Taylor series expansion of the Bregman divergences, we reveal a relationship between Bregman divergences and Euclidean distance. This key relationship provides a new direction for NMF algorithms with general Bregman divergences when combined with the scalar block coordinate descent method. The proposed algorithm generalizes several recently proposed methods for computation of NMF with Bregman divergences and is computationally faster than existing alternatives. We demonstrate the effectiveness of our approach with experiments conducted on artificial as well as real world data.

Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

“…Depending on the probabilistic model of the underlying data, NMF can be formulated with various divergences. Formulations and algorithms based on Kullback-Leibler divergence [67,79], Bregman divergence [24,68], Itakura-Saito divergence [29], and Alpha and Beta divergences [21,22] have been developed. For discussion on nonnegative rank as well as the geometric interpretation of NMF, see Lin and Chu [72], Gillis [34], and Donoho and Stodden [27].…”

“…Matrices W and H are found by solving an optimization problem defined with Frobenius norm, Kullback-Leibler divergence [67,68], or other divergences [24,68]. In this paper, we focus on the NMF based on Frobenius norm, which is the most commonly used formulation:…”

Intraday trading volume and non-negative matrix factorization

“…In a generalized way, the Bregman divergence D ϕ (D D ) is used as the objective function to be minimized [21,22]. Considering only separable Bregman divergences, (5) where ϕ(.)…”

A column-wise update algorithm for nonnegative matrix factorization in Bregman divergence with an orthogonal constraint

“…To enable such a column-wise update even in ONMF, we derive a set of column-wise orthogonal constraints, taking into consideration both nonnegativity and orthogonality at the same time. Furthermore, we show that the columnwise orthogonal constraint can also be applied to column-wise update algorithms called scalar Block Coordinate Descent for solving Bregman divergence NMF (sBCD-NMF) (Li et al 2012) where the Frobenius norm in (1) is replaced with more general Bregman divergence (Li et al 2012). This sBCD-ONMF algorithm is the first algorithm to solve ONMF with Bregman divergence.…”

“…In Sect. 4, we incorporate the column-wise orthogonal constraint into sBCD-NMF proposed by Li et al (2012) in order to propose sBCD-ONMF algorithm. In Sect.…”