New SVD based initialization strategy for non-negative matrix factorization

Abstract

a b s t r a c tThere are two problems need to be dealt with for Non-negative Matrix Factorization (NMF): choose a suitable rank of the factorization and provide a good initialization method for NMF algorithms. This paper aims to solve these two problems using Singular Value Decomposition (SVD). At first we extract the number of main components as the rank, actually this method is inspired from [15,16]. Second, we use the singular value and its vectors to initialize NMF algorithms. We title this new method as SVD-NMF. Boutsidis and Gollopoulos (2008) [2] provided the method titled NNDSVD to enhance initialization of NMF algorithms. They extracted the positive section and respective singular triplet information of the unit matrices {C ( j) } k j=1 which obtained by singular vector pairs based on SVD. In this strategy, they use the triplet information of SVD twice with low computational cost. The differences between SVD-NMF and NNDSVD are the once utilization of SVD in former method and different approximations of initializations for NMF algorithms. We report numerical experiments on two face databases ORL, YALE (C.U.C. Laboratory [10]; U.C. Version [17]) and one object database COIL-20 with two versions which are from Columbia University Image Library (Nene et al., 1996) [13]. Results show that SVD-NMF has faster convergence and provides an approximation with smaller errors than that of obtained by NNDSVD and random initialization.

A Consensus Approach to Improve NMF Document Clustering

“…NMF provides a different solution for every different initialisation making it substantially sensitive to starting points as its convergence directly relies on the characteristics of the given entries. Several publications have shown interest in finding the best way to start a NMF algorithm by providing a structured initialization, in some cases obtained from results of clustering algorithms such as k-means or Spherical K-means [27,28] (especially for applying NMF on document-term matrices), Nonnegative Singular Value decomposition (NNDSVD) [4] or SVD based strategies [17]. The optimization procedures for D respectively equal to the Frobenius norm and the KL divergence, based on multiplicative update rules are given in Algorithms 1 and 2.…”

Investigating Potential Human Tetrachromacy in Individuals with Tetrachromat Genotypes Using Multispectral Techniques

“…In addition, data dimensionality is taken into account in our NMF algorithm. The NMF algorithm uses Singular Value Decomposition (SVD) for initialization [28]. The SVD method uses p + 1 leading components of SVD decomposition, where p leading components contain less than 90% of a total variance.…”

Robust feature representation for classification of bird song syllables

“…However, the NMF method is computationally more demanding than, e.g., the SVD technique, as it requires an iterative solution. Different algorithms for better convergence have been proposed, and recently, an approach using the SVD basis functions as initialization of the NMF algorithm was suggested, [25].…”