Multilayer Nonnegative Matrix Factorization Using Projected Gradient Approaches

Abstract

The most popular algorithms for Nonnegative Matrix Factorization (NMF) belong to a class of multiplicative Lee-Seung algorithms which have usually relative low complexity but are characterized by slow-convergence and the risk of getting stuck to in local minima. In this paper, we present and compare the performance of additive algorithms based on three different variations of a projected gradient approach. Additionally, we discuss a novel multilayer approach to NMF algorithms combined with multi-start initializations procedure, which in general, considerably improves the performance of all the NMF algorithms. We demonstrate that this approach (the multilayer system with projected gradient algorithms) can usually give much better performance than standard multiplicative algorithms, especially, if data are ill-conditioned, badly-scaled, and/or a number of observations is only slightly greater than a number of nonnegative hidden components. Our new implementations of NMF are demonstrated with the simulations performed for Blind Source Separation (BSS) data.

Rational Variety Mapping for Contrast-Enhanced Nonlinear Unsupervised Segmentation of Multispectral Images of Unstained Specimen

“…These methods are referred to as i) dependent component analysis (DCA), 10,11 ii) multilayer hierarchical alternating least-square non-negative matrix factorization (HALS NMF), 16,17 iii) non-negative matrix underapproximation (NMU), 15 and iv) non-negative tensor factorization (NTF). 18 DCA is an extension of independent component analysis for problems in which statistical independence assumption between sources is not fulfilled.…”

“…An additional performance improvement of the NMF algorithms is obtained when they are applied in the multilayer mode. 16 The NMU has been introduced recently as a refinement of the NMF algorithms toward sparse factorization of (X) in (1). In addition to non-negativity constraints imposed on A and (S), the cost function D͑͑X͒ʈA ͑S͒͒ ϭ ʈ͑X͒ Ϫ A ͑S͒ʈ 2 2 is minimized, imposing an underapproximation constraint on A and (S): A (S) Յ (X).…”

“…We apply second-and third-order mappings (RVM2 and RVM3, respectively) in combination with several state-of-the-art matrix and tensor factorization methods [15][16][17][18] to perform nonlinear unsupervised segmentation of the multispectral microscopy images of the unstained specimen of sciatic nerve fibers (nervus ischiadicus) and spleen tissue. In the first case, our proposed methods yielded significant contrast enhancement between nerve cells and background, relative to segmentation performed by factorization algorithms only.…”

Blind decomposition of low‐dimensional multi‐spectral image by sparse component analysis

“…Great improvement in the performance of the NMF algorithms is obtained when they are applied in the multilayer mode [26], whereas sequential decomposition of the nonnegative matrices is performed as follows. In the first layer, the basic approximation decomposition is performed…”

“…Because it employs alternating least squares minimization to estimate mixing matrix and matrix of the materials it is coined as HALS NMF algorithm. When employed in multilayer mode [26], the HALS NMF algorithm has demonstrated good performance in solving underdetermined BSS problems. Thus, the multilayer HALS NMF algorithms and its application in blind multi-spectral image decomposition represent the main contributions of this paper.…”