Effect of Environmental Noise in Speech Quality Assessment Studies using Crowdsourcing

Jiménez, Rafael Zequeira; Naderi, Babak; Möller, Sebastian

doi:10.1109/qomex48832.2020.9123144

Cited by 6 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a stricter screening as performed in CS 401 seems to substantially decrease the number of conditions with substantial differences. Previous studies showed that presence of environmental noise at 50dB(A) SPL leads to a significant difference of the perceived quality [42].…”

Section: Comparison Between Degradationsmentioning

confidence: 97%

Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods

et al. 2020

Self Cite

View full text Add to dashboard Cite

Subjective speech quality assessment has traditionally been carried out in laboratory environments under controlled conditions. With the advent of crowdsourcing platforms tasks, which need human intelligence, can be resolved by crowd workers over the Internet. Crowdsourcing also offers a new paradigm for speech quality assessment, promising higher ecological validity of the quality judgments at the expense of potentially lower reliability. This paper compares laboratory-based and crowdsourcing-based speech quality assessments in terms of comparability of results and efficiency. For this purpose, three pairs of listening-only tests have been carried out using three different crowdsourcing platforms and following the ITU-T Recommendation P.808. In each test, listeners judge the overall quality of the speech sample following the Absolute Category Rating procedure. We compare the results of the crowdsourcing approach with the results of standard laboratory tests performed according to the ITU-T Recommendation P.800. Results show that in most cases, both paradigms lead to comparable results. Notable differences are discussed with respect to their sources, and conclusions are drawn that establish practical guidelines for crowdsourcing-based speech quality assessment.

show abstract

Section: Comparison Between Degradationsmentioning

confidence: 97%

Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Crowdsourcing does have its challenges: the test participants take part in the test in their working environment using their own hardware without supervision or direct quality control. Previous works showed that background noise in the participant's surroundings could mask the degradation under the test and lead to a significantly different rating [28], [29]. Different listening devices can also strongly influence the perceived quality [30].…”

Section: Related Workmentioning

confidence: 99%

ICASSP 2023 Deep Noise Suppression Challenge

Dubey,

Aazami,

Gopal

et al. 2024

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

The ICASSP 2023 Deep Noise Suppression (DNS) Challenge marks the fifth edition of the DNS challenge series. DNS challenges were organized from 2019 to 2023 to foster research in the field of DNS. Previous DNS challenges were held at INTERSPEECH 2020, ICASSP 2021, INTERSPEECH 2021, and ICASSP 2022 This challenge aims to advance models capable of jointly addressing denoising, dereverberation, and interfering talker suppression, with separate tracks focusing on headset and speakerphone scenarios. The challenge facilitates personalized deep noise suppression by providing accompanying enrollment clips for each test clip, each containing the primary talker only, which can be used to compute a speaker identity feature and disentangle primary and interfering speech. While the majority of models submitted to the challenge were personalized, the same teams emerged as the winners in both tracks. The best models demonstrated improvements of 0.145 and 0.141 in the challenge's score, respectively, when compared to the noisy blind test set. We present additional analysis and draw comparisons to previous challenges.

show abstract

Introduction

Jiménez

2021

Influencing Factors in Speech Quality Assessment Using Crowdsourcing

View full text Add to dashboard Cite

Effect of Environmental Noise in Speech Quality Assessment Studies using Crowdsourcing

Cited by 6 publications

References 14 publications

Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods

Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods

ICASSP 2023 Deep Noise Suppression Challenge

Introduction

Contact Info

Product

Resources

About