Deep neural networks for single channel source separation

Abstract

In this paper, a novel approach for single channel source separation (SCSS) using a deep neural network (DNN) architecture is introduced. Unlike previous studies in which DNN and other classifiers were used for classifying time-frequency bins to obtain hard masks for each source, we use the DNN to classify estimated source spectra to check for their validity during separation. In the training stage, the training data for the source signals are used to train a DNN. In the separation stage, the trained DNN is utilized to aid in estimation of each source in the mixed signal. Single channel source separation problem is formulated as an energy minimization problem where each source spectra estimate is encouraged to fit the trained DNN model and the mixed signal spectrum is encouraged to be written as a weighted sum of the estimated source spectra. The proposed approach works regardless of the energy scale differences between the source signals in the training and separation stages. Nonnegative matrix factorization (NMF) is used to initialize the DNN estimate for each source. The experimental results show that using DNN initialized by NMF for source separation improves the quality of the separated signal compared with using NMF for source separation. © 2014 IEEE

A perceptually-weighted deep neural network for monaural speech enhancement in various background noise conditions

“…speech recognition [2] and speech synthesis [3]. More recently, the DNN has been applied to speech separation [4]- [7] and enhancement/denoising [8]- [10], particularly for monaural recordings [4]- [6], [8]- [10]. When processing mixtures of target speech signals and competing noise, speech separation may be considered as speech enhancement.…”

“…The inputs to the DNN are often (hybrid) features such as timefrequency (TF) domain spectral features [4]- [6], [8]- [10] and filterbank features [4], [5], [11]; while the output can be the TF unit level features that can be used to recover the speech source, such as ideal binary/ratio masks (IBM/IRM) [4]- [6], [11], direct magnitude spectra [9], [10] or their transforms such as log power (LP) spectra [8].…”

“…In order to recover the underlying target speech embedded in noise, most of the deep neural networks, either recurrent [4], [5], [10] or feedforward [4], [6], [8], [9], [11], are trained to optimize some objective functions such as the mean squared error (MSE) between the true and predicted outputs. The inputs to the DNN are often (hybrid) features such as timefrequency (TF) domain spectral features [4]- [6], [8]- [10] and filterbank features [4], [5], [11]; while the output can be the TF unit level features that can be used to recover the speech source, such as ideal binary/ratio masks (IBM/IRM) [4]- [6], [11], direct magnitude spectra [9], [10] or their transforms such as log power (LP) spectra [8].…”

Single-channel blind separation using L1-sparse complex non-negative matrix factorization for acoustic signals

“…Many SCBSS solutions in various approaches have been proposed but the dominant approach is the non-negative matrix factorization (NMF) [4][5][6][7] and its variants. [8][9][10][11][12][13][14][15] In this paper, we introduced a new method based on complex non-negative matrix factorization (CMF).…”

Deep Karaoke: Extracting Vocals from Musical Mixtures Using a Convolutional Deep Neural Network

“…INTRODUCTION Much work in audio source separation has been inspired by the ability of human listeners to maintain separate auditory neural and perceptual representations of competing speech in 'cocktail party' listening scenarios [1]- [3]. A common engineering approach is to decompose a mixed audio signal, comprising two or more competing speech signals, into a spectrogram in order to assign each time-frequency element to the respective sources [4]- [6]. Hence, this form of source separation may be interpreted as a classification problem.…”