Single-channel speech separation with memory-enhanced recurrent neural networks

Abstract

In this paper we propose the use of Long Short-Term Memory recurrent neural networks for speech enhancement. Networks are trained to predict clean speech as well as noise features from noisy speech features, and a magnitude domain soft mask is constructed from these features. Extensive tests are run on 73 k noisy and reverberated utterances from the Audio-Visual Interest Corpus of spontaneous, emotionally colored speech, degraded by several hours of real noise recordings comprising stationary and non-stationary sources and convolutive noise from the Aachen Room Impulse Response database. In the result, the proposed method is shown to provide superior noise reduction at low signal-to-noise ratios while creating very little artifacts at higher signal-to-noise ratios, thereby outperforming unsupervised magnitude domain spectral subtraction by a large margin in terms of source-distortion ratio.

Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks

“…Supervised learning of time-frequency masks for the noisy spectrum has been investigated in [11][12][13][14][15], using stereo training data in which noisy speech is the input, and a target time-frequency mask based on the corresponding clean speech data forms the output. Subsequent work [6] focused on modeling dynamics well using long short-term memory (LSTM) recurrent neural networks which helped achieve state of the art performance on a difficult task with nonstationary interference.…”

On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement

“…II and the learning rates given in Table II. 5 value originally proposed in [48] and currently default in https://keras.io/. Figure 3 presents the waveforms of a specific 10 ms realization of clean, noisy, and enhanced speech signals from the experiments in Table I.…”

Multichannel Speech Separation with Recurrent Neural Networks from High-Order Ambisonics Recordings

“…"mask" corresponds to the first channel ofs in (8) and "filter" to y, the output of the GEVD-MWF in (4). The best result in each situation is shown in bold.…”

“…DNNs were originally applied to single-channel inputs to derive a singlechannel filter, a.k.a. a mask [8][9][10]. In the multichannel case, several approaches have been proposed to pass spatial information directly to a DNN, for instance using phase difference features between non-coincident microphones [11] or coherence features [12].…”