Detailed comparative analysis of PESQ and VISQOL behaviour in the context of playout delay adjustments introduced by VOIP jitter buffer algorithms

Abstract

This paper undertakes a detailed comparative analysis of both PESQ and VISQOL model behaviour, when tested against speech samples modified through playout delay adjustments. The adjustments are typical (in extent and magnitude) to those introduced by VoIP jitter buffer algorithms. Furthermore, the analysis examines the impact of adjustment location as well as speaker factors on MOS scores predicted by both models and seeks to determine if both models are able to correctly predict the impact on quality perceived by the end user from earlier subjective tests. The earlier results showed speaker voice preference and potentially wideband experience dominating subjective tests more than playout delay adjustment duration or location. By design, PESQ and VISQOL do not qualify speaker voice difference reducing their correlation with the subjective tests. In addition, it was found that PESQ scores are impacted by playout delay adjustments and thus the impact of playout delay adjustments on a quality perceived by the end user is not well modelled. On the other hand, VISQOL model is better in predicting an impact of playout delay adjustments on a quality perceived by the user but there are still some discrepancies in the predicted scores. The reasons for those discrepancies are particularly analysed and discussed.

ViSQOL: an objective speech quality model

“…PESQ is still worthy of analysis as recently published research continues to use PESQ for VoIP speech quality assessment, e.g., [26,27]. This paper presents the culmination of work from the authors [4,5,28] in developing a new objective metric of speech quality, called ViSQOL. ViSQOL has been designed to be particularly sensitive to VoIP degradation but without sacrificing wider deployability.…”

“…Experiment 2 evaluates the impact of small playout adjustments due to jitter buffers on objective quality assessment. Experiment [28,42], where POLQA and ViSQOL show inconsistent quality estimations for some combinations of speaker and playout adjustments. Experiment 4 uses a subjectively labeled database of VoIP degradations to benchmark model performance for clock drift, packet loss, and jitter.…”

“…The results were analyzed for PESQ and POLQA [42] and subsequently for ViSQOL [28], showing MOS scores grouped by speaker and variant instead of playout condition. The combined results from both studies are presented in Figure 9.…”

ViSQOLAudio: An objective audio quality metric for low bitrate codecs

“…[14][15][16] It is a full reference speech quality metric, comparable to POLQA, that uses similarity between spectrograms to measure quality. This paper presents ViSQOLAudio, an adaption of the ViSQOL speech quality metric.…”

The uncertainty entropy of low-rate speech quality evaluation and the analyses of the gray correlation

“…The normal distribution function, the statistical regression and distortion are the main methods to analyze the time delay, jitter, coding distortion [10,11,12,13,14]. And adjust the model by the data from the real network.…”