Deep Noise Suppression Maximizing Non-Differentiable PESQ Mediated by a Non-Intrusive PESQNet

Abstract

Speech enhancement employing deep neural networks (DNNs) for denoising is called deep noise suppression (DNS). The DNS trained with mean squared error (MSE) losses cannot guarantee good perceptual quality. Perceptual evaluation of speech quality (PESQ) is a widely used metric for evaluating speech quality. However, the original PESQ algorithm is non-differentiable, therefore, cannot directly be used as optimization criterion for gradient-based learning. In this work, we propose an end-to-end non-intrusive PESQNet DNN to estimate the PESQ scores of the enhanced speech signal. Thus, by providing a reference-free perceptual loss, it serves as a mediator towards the DNS training, allowing to maximize the PESQ score of the enhanced speech signal. We illustrate the potential of our proposed PESQNet-mediated training on a strong baseline DNS. As further novelty, we propose to train the DNS and the PESQNet alternatingly to keep the PESQNet up-todate and perform well specifically for the DNS under training. Detailed analysis shows that the PESQNet mediation further increases the DNS performance by about 0.1 PESQ points on synthetic test data and by 0.03 DNSMOS points on real test data, compared to training with the MSE-based loss. Our proposed method outperforms the Interspeech 2021 DNS Challenge baseline by 0.2 PESQ points on synthetic test data and 0.1 DNSMOS points on real test data. Furthermore, it improves on the same DNS trained with an approximated differentiable PESQ loss by about 0.4 PESQ points on synthetic test data and 0.2 DNSMOS points on real test data.

Coded Speech Quality Measurement by a Non-Intrusive PESQ-DNN

“…Both versions of DNSMOS require the input speech signals having a fixed length of nine seconds. In our recent works [21], [22], [24], we proposed an end-to-end PESQNet for DNS applications, adapted from a BLSTM-based speech emotion recognition DNN [35], to predict PESQ scores of the enhanced speech signal. In these works, the trained PESQNet is employed as a mediator to provide a differentiable PESQ loss during a speech enhancement DNN training, aiming at maximizing the PESQ score of the enhanced speech signal.…”

“…As concerns topology, we build upon [22], but many changes are required for the DNN to serve the speech communication monitoring needs targeted in this work: (1) Compared to PESQNet, the novel PESQ-DNN employs a complex spectrogram as input to explicitly consider phase influences in the perceived speech quality. Except for a few works, e.g., WaweNet, most speech quality prediction DNNs employ amplitude or power spectrogram input, leading to the problem that speech quality degradations caused by phase distortions cannot be measured.…”

“…The number of input channels is set to either C 1 or C 2 for the PESQ-DNN employing the amplitude or complex spectrogram as input, respectively. The "Embedding Processing" model used in the baseline PESQNet [22] is recapitulated in Fig. 2, while the employed model for our proposed PESQ-DNN containing frame-level or block-level embeddings is illustrated in Fig.…”

MetricGAN-U: Unsupervised Speech Enhancement/ Dereverberation Based Only on Noisy/ Reverberated Speech

“…Machine-learning-based methods have been proposed to eliminate the dependence on clean speech references during inference and can be further divided into two categories. The first attempts to non-intrusively estimate the objective scores mentioned above (Fu et al, 2018;Dong & Williamson, 2020;Zezario et al, 2020;Catellier & Voran, 2020;Yu et al, 2021b;Xu et al, 2022;Kumar et al, 2023). However, during training, noisy/processed and clean speech pairs are still required to obtain the objective scores as model targets.…”

MetricGAN+/-: Increasing Robustness of Noise Reduction on Unseen Data

“…However, such objective functions must be carefully designed as many objective measures contain calculations that are non-differentiable. Several systems circumvent this limitation via use of an additional model that mimics the behaviour of the metric [12]- [14], with this network being used as a surrogate of the metric used as an objective function in training of the speech enhancement model. The baseline system that this work builds upon is one such system, MetricGAN+ [15] (itself an extension of previous work MetricGAN [16]).…”