Towards Efficient Models for Real-Time Deep Noise Suppression

Abstract

With recent research advancements, deep learning models are becoming attractive and powerful choices for speech enhancement in real-time applications. While state-of-the-art models can achieve outstanding results in terms of speech quality and background noise reduction, the main challenge is to obtain compact enough models, which are resource efficient during inference time. An important but often neglected aspect for data-driven methods is that results can be only convincing when tested on real-world data and evaluated with useful metrics. In this work, we investigate reasonably small recurrent and convolutional-recurrent network architectures for speech enhancement, trained on a large dataset considering also reverberation. We show interesting tradeoffs between computational complexity and the achievable speech quality, measured on real recordings using a highly accurate MOS estimator. It is shown that the achievable speech quality is a function of network complexity, and show which models have better tradeoffs.

Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement

“…Training Data for SE: We utilized a large-scale and highquality simulated dataset described in [24], which includes around 1,000 hours of paired speech samples 1 . As a clean speech corpus, the dataset collects 544 hours of speech recordings with high mean opinion score (MOS) values from the LibriVox corpus [25].…”

“…In addition, the clean speech in each mixture is convolved with an acoustic room impulse response (RIR) sampled from 7,000 measured and simulated responses. See [24] for details of this dataset. The data are available publicly, except for the 65 hours of the internal noise recordings 2 .…”

Low Complexity Online Convolutional Beamforming

“…1. We use the convolutional recurrent network for speech enhancement (CRUSE) proposed in [19], which…”

“…4. As baselines we have the unprocessed reference microphone, delay&sum and superdirective MVDR beamformers, the single-channel DNN (CRUSE) [19] applied on the reference mic, mask-based MVDR beamformer using the DNN-mask to adaptively update the noise covariance [21], and the competitive RLS-WPD [15] as the state-of-the-art online convolutional beamformer.…”

“…The steering vector a(k, n) is estimated using a spatial probability-based far-field localization method [17] based on the simple plane wave sound propagation model. CRUSE is trained on the data from [24] as described in [19], only with adjusted STFT parameters. As the test signals are very short, mostly below 10 s, all adaptive methods are initialized with a prior pass to give the adaptive algorithms a chance to converge.…”