Speech separation based on improved deep neural networks with dual outputs of speech features for both target and interfering speakers

Abstract

In this paper, a novel deep neural network (DNN) architecture is proposed to generate the speech features of both the target speaker and interferer for speech separation. DNN is adopted here to directly model the highly nonlinear relationship between speech features of the mixed signals and the two competing speakers. With the modified output speech features for learning the parameters of the DNN, the generalization capacity to unseen interferers is improved for separating the target speech. Meanwhile, without any prior information from the interferer, the interfering speech can also be separated. Experimental results show that the proposed new DNN enhances the separation performance in terms of different objective measures under the semi-supervised mode where the training data of the target speaker is provided while the unseen interferer in the separation stage is predicted by using multiple interfering speakers mixed with the target speaker in the training stage.

Combining non-negative matrix factorization and deep neural networks for speech enhancement and automatic speech recognition

“…Contrarily to [2,9], where DNNs are trained of raw spectral features, we train the DNN on SNMF activation coefficients. Hence, to evaluate the influence of the input features of the DNN, we introduce a variant of our framework denoted (DNN-SNMF-Spec), where the DNN is learned on spectral features to predict activation coefficients, and uses the modified cost function computed on signal reconstruction.…”

“…The activation coefficients are then used as input features of the DNN, instead of raw spectral coefficients as in [9] or the log spectrum in [2]. For each frame of noisy speech (at index position t), we build a large vector composed of the concatenation of the activation coefficients of speechĥ S,t and noisê h N,t vectors extracted on each frame on an analysis windows of width (2K + 1) frames centred on the t th frame.…”

“…This topic has been studied for more than 50 years, and has produced successful approaches, especially statistics based methods [1] able to efficiently reduce the contribution of noise in degraded signals as long as the stationary noise assumption is respected. More recently, several works based-on machine learning algorithms such as Sparse Non-negative Matrix Factorization (SNMF) and Deep Neural Networks (DNN) have achieved significant improvements for non-stationary noises [2,3].…”

“…DNN-based SE [2,10] relies on the ability of Deep Neural Networks to estimate complex non-linear functions used to directly map log spectral features of noisy speech into corresponding clean speech signals and therefore may be more efficient in separating noise and speech in case of overlapping sub-domains. Temporal dependencies of speech are usually considered by extracting features on sliding context windows.…”

“…Our proposal has been evaluated both for the tasks of Speech Enhancement and Automatic Speech Recognition (ASR) using several objective metrics. Our results have been systematically compared to several state-of-the-art DNN and SNMF-based SE systems [2,3,4,9]. Evaluations have been conducted using the framework provided recently by the CHiME-3 challenge [11] on speech separation and recognition on challenging real noisy speech recordings.…”