Weighted Speech Distortion Losses for Neural-Network-Based Real-Time Speech Enhancement

Abstract

This paper investigates several aspects of training a RNN (recurrent neural network) that impact the objective and subjective quality of enhanced speech for real-time single-channel speech enhancement. Specifically, we focus on a RNN that enhances short-time speech spectra on a single-frame-in, single-frame-out basis, a framework adopted by most classical signal processing methods. We propose two novel mean-squared-error-based learning objectives that enable separate control over the importance of speech distortion versus noise reduction. The proposed loss functions are evaluated by widely accepted objective quality and intelligibility measures and compared to other competitive online methods. In addition, we study the impact of feature normalization and varying batch sequence lengths on the objective quality of enhanced speech. Finally, we show subjective ratings for the proposed approach and a state-of-the-art real-time RNN-based method.

Combining Adaptive Filtering And Complex-Valued Deep Postfiltering For Acoustic Echo Cancellation

“…We compare the proposed complex-valued PF to a real-valued one. Its design is adapted from the noise suppression network proposed in [34]. This PF uses one dense layer as a feature extraction layer followed by two stacked GRU layers and a dense output layer.…”

Towards Efficient Models for Real-Time Deep Noise Suppression

“…While this method non-surprisingly achieves only minor MOS improvement of around 0.12 on the test set including many highly nonstationary noise types, the achievement relative to its computational burden in the order of a few 1000 MACs is a tiny fraction of even our smallest models. For direct comparison of the following DNNbased baselines, we only took the architectures, but trained them using the same features, prediction targets, loss, and training procedure as for all other networks in this paper: i) NSnet [4] yields similar MOS as the classic NS. ii) The fully convolutional architecture proposed in [27] underperforms in this task, which we mainly suspect to the absence long-term temporal modeling as it uses only 8 frames temporal context.…”

“…Earlier network architectures were mainly recurrent neural network (RNN) structures, which were believed promising in terms of efficiency due to its efficient temporal modeling capabilities [2][3][4]. While such models seem to hit a performance saturation, the use of convolutional recurrent networks (CRNs) and convolutional neural networks (CNNs) raised the performance, but resulted in development of enormously large architectures [5][6][7][8] that are impractical to run on typical edge devices like consumer laptops, mobile phones, or even less powerful devices like wearables or hearing aids.…”

A Modulation-Domain Loss for Neural-Network-Based Real-Time Speech Enhancement

“…The STFT is first obtained using a 20-millisecond Hamming window with 50% overlap and a 512-point discrete Fourier transform. Then we take the natural logarithm of the power of the STFT and normalize the LPS with frequency-dependent online normalization following [22].…”