Dnsmos P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors

Reddy, Chandan K.; Gopal, Vishak; Cutler, Ross

doi:10.1109/icassp43922.2022.9746108

Cited by 67 publications

(27 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non-intrusive objective speech quality assessment tools like ITU-T P.563 [14] do not require a reference, though it has low correlation to subjective quality [15]. Newer neural net-based methods such as [15,16,17,18] provide better correlations to subjective quality. NISQA [19] is an objective metric for P.804, though the correlation to subjective quality is not sufficient to use as a challenge metric.…”

Section: Related Workmentioning

confidence: 99%

ICASSP 2022 Acoustic Echo Cancellation Challenge

Cutler

Saabas

Pärnamaa

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The ICASSP 2023 Speech Signal Improvement Challenge is intended to stimulate research in the area of improving the speech signal quality in communication systems. The speech signal quality can be measured with SIG in ITU-T P.835 and is still a top issue in audio communication and conferencing systems. For example, in the ICASSP 2022 Deep Noise Suppression challenge, the improvement in the background and overall quality is impressive, but the improvement in the speech signal is statistically zero. To improve the speech signal the following speech impairment areas must be addressed: coloration, discontinuity, loudness, and reverberation. A dataset and test set were provided for the challenge, and the winners were determined using an extended crowdsourced implementation of ITU-T P.80's listening phase . The results show significant improvement was made across all measured dimensions of speech quality.

show abstract

Section: Related Workmentioning

confidence: 99%

ICASSP 2022 Acoustic Echo Cancellation Challenge

Cutler

Saabas

Pärnamaa

et al. 2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Some of these metrics are intrusive while others are not, and some predict only an overall score while others predict multiple score categories. The recently published DNSMOSP835 [ 25 ] (which is an extension of DNSMOS [ 14 ]) is a non-intrusive metric predicts the three ITU P.835 recommended score categories. It consists of a convolutional neural network trained on MOS ratings, and aims at evaluating modern speech enhancement algorithms.…”

Section: Related Workmentioning

confidence: 99%

Non-intrusive deep learning-based computational speech metrics with high-accuracy across a wide range of acoustic scenes

Diehl

Thorbergsson²,

Singer³

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

Speech with high sound quality and little noise is central to many of our communication tools, including calls, video conferencing and hearing aids. While human ratings provide the best measure of sound quality, they are costly and time-intensive to gather, thus computational metrics are typically used instead. Here we present a non-intrusive, deep learning-based metric that takes only a sound sample as an input and returns ratings in three categories: overall quality, noise, and sound quality. This metric is available via a web API and is composed of a deep neural network ensemble with 5 networks that use either ResNet-26 architectures with STFT inputs or fully-connected networks with wav2vec features as inputs. The networks are trained and tested on over 1 million crowd-sourced human sound ratings across the three categories. Correlations of our metric with human ratings exceed or match other state-of-the-art metrics on 51 out of 56 benchmark scenes, while not requiring clean speech reference samples as opposed to metrics that are performing well on the other 5 scenes. The benchmark scenes represent a wide variety of acoustic environments and a large selection of post-processing methods that include classical methods (e.g. Wiener-filtering) and newer deep-learning methods.

show abstract

“…Recently, DNNs have been utilized for finding a subjective alternative score [120]- [125]. Unlike the previous composite measure, most of these methods will take the track as an input and the network is trained to mimic the subjective ratings.…”

Section: Subjective Evaluationmentioning

confidence: 99%

“…Additionally, these scores are non-intrusive, hence evaluating enhanced tracks without the need for clean reference is possible. The standard score used as a subjective baseline for many recent studies is the DNSMOS proposed by Microsoft in [124], [125]. The DNSMOS is trained on 75 hours of rated speech.…”

Section: Subjective Evaluationmentioning

confidence: 99%

CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement

Abdulatif¹,

Cao²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

<p>Convolution-augmented transformers (Conformers) are recently proposed in various speech-domain applications, such as automatic speech recognition (ASR) and speech separation, as they can capture both local and global dependencies. In this paper, we propose a conformer-based metric generative adversarial network (CMGAN) for speech enhancement (SE) in the time-frequency (TF) domain. The generator encodes the magnitude and complex spectrogram information using two-stage conformer blocks to model both time and frequency dependencies. The decoder then decouples the estimation into a magnitude mask decoder branch to filter out unwanted distortions and a complex refinement branch to further improve the magnitude estimation and implicitly enhance the phase information. Additionally, we include a metric discriminator to alleviate metric mismatch by optimizing the generator with respect to a corresponding evaluation score. Objective and subjective evaluations illustrate that CMGAN is able to show superior performance compared to state-of-the-art methods in three speech enhancement tasks (denoising, dereverberation and super-resolution). For instance, quantitative denoising analysis on Voice Bank+DEMAND dataset indicates that CMGAN outperforms various previous models with a margin, i.e., PESQ of 3.41 and SSNR of 11.10 dB. </p>

show abstract

Dnsmos P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors

Cited by 67 publications

References 19 publications

ICASSP 2022 Acoustic Echo Cancellation Challenge

ICASSP 2022 Acoustic Echo Cancellation Challenge

Non-intrusive deep learning-based computational speech metrics with high-accuracy across a wide range of acoustic scenes

CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement

Contact Info

Product

Resources

About