Speech Denoising with Deep Feature Losses

Abstract

We present an end-to-end deep learning approach to denoising speech signals by processing the raw waveform directly. Given input audio containing speech corrupted by an additive background signal, the system aims to produce a processed signal that contains only the speech content. Recent approaches have shown promising results using various deep network architectures. In this paper, we propose to train a fully-convolutional context aggregation network using a deep feature loss. That loss is based on comparing the internal feature activations in a different network, trained for acoustic environment detection and domestic audio tagging. Our approach outperforms the stateof-the-art in objective speech quality metrics and in large-scale perceptual experiments with human listeners. It also outperforms an identical network trained using traditional regression losses. The advantage of the new approach is particularly pronounced for the hardest data with the most intrusive background noise, for which denoising is most needed and most challenging.

Speech Enhancement Using Self-Adaptation and Multi-Head Self-Attention

“…An important requirement in DNN-based speech enhancement and separation is generalization that means working for any speaker. To achieve this, in speech enhancement, several studies train a global M using many speech samples spoken by many speakers [3][4][5][6][7][8][9][10][11][12][13][14]. Unfortunately, in speech separation, generalization cannot be achieved solely using a large scale training dataset because there is no way of knowing which signal in the speech-mixture is the target.…”

“…Trained using other 27 speakers' samples, that is, the speaker independent model (the scope of conventional studies). We used the perceptual evaluation of speech quality (PESQ), CSIG, CBLK, and COVL as the performance metrics which are the standard metrics for this dataset [4,8,9,12]. The three composite measures CSIG, CBAK, and COVL are the popular predictor of the mean opinion score (MOS) of the target signal distortion, background noise interference, and overall speech quality, respectively [30].…”

“…The proposed method was compared with speech enhancement generative adversarial network (SEGAN) [4], MMSE-GAN [8], deep feature loss (DFL) [9], and Metric-GAN [12], because these methods have been evaluated on the same dataset. Table 2 shows the experimental results.…”

“…We conducted a subjective experiment. The proposed method was compared with SEGAN [4] and DFL [9] because speech samples of both methods are openly available in DFL's web-page [32]. We selected 15 samples from Tranche 1-3 data from the web-page (low SNR conditions).…”

Phonetic Feedback for Speech Enhancement with and Without Parallel Speech Data

“…In style transfer, the perceptual loss ensures that the high-level contents of an image remain the same, while allowing the texture of the image to change. For speech-related tasks a perceptual loss has been used to denoise time-domain speech data [8], where the loss was called a "deep feature loss". The perceiving network was trained for acoustic environment detection and domestic audio tagging.…”