Single channel speech music separation using nonnegative matrix factorization and spectral masks

Abstract

A single channel speech-music separation algorithm based on nonnegative matrix factorization (NMF) with spectral masks is proposed in this work. The proposed algorithm uses training data of speech and music signals with nonnegative matrix factorization followed by masking to separate the mixed signal. In the training stage, NMF uses the training data to train a set of basis vectors for each source. These bases are trained using NMF in the magnitude spectrum domain. After observing the mixed signal, NMF is used to decompose its magnitude spectra into a linear combination of the trained bases for both sources. The decomposition results are used to build a mask, which explains the contribution of each source in the mixed signal. Experimental results show that using masks after NMF improves the separation process even when calculating NMF with fewer iterations, which yields a faster separation process. © 2011 IEEE

Speech/music separation using non-negative matrix factorization with combination of cost functions

“…Weights for basis vectors appear in corresponding columns in matrix W. To approximate data in V as a non-negative linear combination of its component vectors, the non-negative basis vectors in matrix B are optimized. The matrices B and W are estimated by solving following optimization problem [7]:…”

“…With two sets of training data for speech and music signals, the Fast Fourier Transform (FFT) is computed for each signal to obtain magnitude spectrogram of speech and music signals. Then, NMF is used for decompose speech and music spectrograms into base and weight matrices In other word, the aim of using NMF is to model the training data as a set of basis vectors to represent the spectral characteristics for each source signal [7]. ≈ ℎ ℎ…”

“…and matrices contain estimations for the magnitude spectral frames of the music and speech signals [7].…”

Deep Karaoke: Extracting Vocals from Musical Mixtures Using a Convolutional Deep Neural Network

“…For comparison to the DNN approach, an equivalent non-negative matrix factorization (NMF) based approach was implemented using the same training and test data (as described above). We used the same unpacking strategy, which has been tested before for NMF-based separation of speech and music [13]. The spectrograms of the training data were sampled and unpacked analogously to the DNN approach, resulting in a large (220500x15000) matrix that was then decomposed using the traditional multiplicative updates algorithm with KL divergence [14].…”

“…For the testing stage, we concatenated both matrices and initialized a corresponding H u matrix randomly, so that for each unpacked spectrogram, V u , of the set of test songs, V u = [W v W nv ] H u . We then ran the same multiplicative updates algorithm but keeping the composite W u matrix fixed [13], and updating H u . The test spectrogram was then re-composed for either vocal (V v = W v H v ) or non-vocal (V nv = W nv H nv ) vectors, and used to define a soft mask via the element-wise division…”