Recurrent neural network for approximate nonnegative matrix factorization

Abstract

A recurrent neural network solving the approximate nonnegative matrix factorization (NMF) problem is presented in this paper. The proposed network is based on the Lagrangian approach, and exploits a partial dual method in order to limit the number of dual variables. Sparsity constraints on basis or activation matrices are included by adding a weighted sum of constraint functions to the least squares reconstruction error. However, the corresponding Lagrange multipliers are computed by the network dynamics itself, avoiding empirical tuning or a validation process. It is proved that local solutions of the NMF optimization problem correspond to as many stable steady-state points of the network dynamics. The validity of the proposed approach is verified through several simulation examples concerning both synthetic and real-world datasets for feature extraction and clustering applications

Self-organization of a recurrent RBF neural network using an information-oriented algorithm

“…can be used to reduce the correlation between hidden neurons and output layer. Based on the former analysis, the independent component contribution of hidden neurons is defined as (15) where I j (t) is the independent component contribution of the jth hidden neuron.…”

“…Then, the neuron splitting mechanism and deleting mechanism can mimic the dynamics of hidden neurons to choose the right active or inactive neurons. Moreover, the self-organizing mechanisms of IOA-RRBFNN are performed without presetting any thresholds, which is different from the existing methods [15]- [44].…”

“…In this case, the network size is often increased with the number of the sample data and a very large network size may be caused, which has the problem of poor generalization and high computational expense. Thus hidden neurons selection and structure optimization are of particular significance [15]- [17].…”