Discriminatively trained recurrent neural networks for single-channel speech separation

Abstract

This paper describes an in-depth investigation of training criteria, network architectures and feature representations for regression-based single-channel speech separation with deep neural networks (DNNs). We use a generic discriminative training criterion corresponding to optimal source reconstruction from time-frequency masks, and introduce its application to speech separation in a reduced feature space (Mel domain). A comparative evaluation of time-frequency mask estimation by DNNs, recurrent DNNs and non-negative matrix factorization on the 2nd CHiME Speech Separation and Recognition Challenge shows consistent improvements by discriminative training, whereas long short-term memory recurrent DNNs obtain the overall best results. Furthermore, our results confirm the importance of fine-tuning the feature representation for DNN training.

Speech Enhancement with LSTM Recurrent Neural Networks and its Application to Noise-Robust ASR

“…There, only the gradient ∂E SA /∂m of the objective function with respect to the network output is specific to source separation, whereas the rest of the algorithm is unchanged. Using L instead of conventional sigmoid or half-wave activation functions helps reducing the vanishing temporal gradient problem of RNNs [5], allowing them to outperform DNNs with static context windows in speech enhancement [16].…”

“…Here, we consider deep recurrent neural networks (DRNNs). as proposed in [16]. The maskm t is estimated by the DRNN forward pass, which is defined as follows, for hidden layers k = 1, .…”

“…There, the source separation problem is formulated as a regression task: determine a time-frequency mask for separating the wanted source, based on acoustic features such as the magnitude spectrogram. Due to their ability to capture the temporal dynamics of speech, RNNs have delivered particularly promising results in the context of regression-based speech enhancement [16,2]. In contrast, the performance of RNN-based speech recognition in noisy conditions is still limited when compared to feedforward deep neural network (DNN) based systems [3,15].…”

“…Supervised training of speech enhancement schemes is becoming increasingly popular especially in the context of single-channel speech enhancement in nonstationary noise [16,7]. There, the source separation problem is formulated as a regression task: determine a time-frequency mask for separating the wanted source, based on acoustic features such as the magnitude spectrogram.…”

A Comparison of Perceptually Motivated Loss Functions for Binary Mask Estimation in Speech Separation

“…Previous studies have focused on either developing improved feature extraction methods or using more sophisticated classifiers -for example moving from Gaussian mixture models (GMMs) to deep neural networks (DNNs). Some attention has been focussed on improving the classifier to reduce perceptual error by changing the loss function for text-to-speech applications [14], and introducing signal approximation loss functions [15,16] as a replacement for mask approximation within speech separation applications. Signal approximation loss functions apply the output of the network to the noisy spectrum within the loss function, and minimise this with respect to the target.…”