On the Evaluation of the Conversational Speech Quality in Telecommunications

Abstract

We propose an objective method to assess speech quality in the conversational context by taking into account the talking and listening speech qualities and the impact of delay. This approach is applied to the results of four subjective tests on the effects of echo, delay, packet loss, and noise. The dataset is divided into training and validation sets. For the training set, a multiple linear regression is applied to determine a relationship between conversational, talking, and listening speech qualities and the delay value. The multiple linear regression leads to an accurate estimation of the conversational scores with high correlation and low error between subjective and estimated scores, both on the training and validation sets. In addition, a validation is performed on the data of a subjective test found in the literature which confirms the reliability of the regression. The relationship is then applied to an objective level by replacing talking and listening subjective scores with talking and listening objective scores provided by existing objective models, fed by speech signals recorded during the subjective tests. The conversational model achieves high performance as revealed by comparison with the test results and with the existing standard methodology "E-model," presented in the ITU-T (International Telecommunication Union) Recommendation G.107.

Audiovisual quality integration for interactive communications

“…In particular, the situation of assessment in which the judging subject is placed (listening/viewing or conversational context referred as experimental context) can impact the judgement process [11]. Subjective tests should reflect the ecological environment of the service or application under assessment.…”

“…Early studies on the impact of conversational delay on speech quality, like the one from Kitawaki and Itoh [8], predicted a severe impact of transmission delay on the perceived speech quality especially when considering highly interactive tasks like the 'Random Number Verification' task [9]. However, recent studies [10][11][12] suggested that the impact of delay in close-to-natural conversation scenarios is far less important as suggested in [8]. In the case of everyday life conversations, it was observed that people tend to adapt their communication strategy affecting both the structure and the semantic flow of the conversation (e.g., misunderstandings and interruptions).…”

IEEE Signal Processing Magazine - November 2007

“…It takes into account the impact of degradations such as talker echo, coding artifacts, and additive noise on the talker's perception of his/her own voice. Guéguin et al [20] proposed a signal-based model of conversational speech quality that combines both PESQ and PESQM with an estimation of the conversational impact of pure delay, the latter being derived from the E-model. None of these approaches has been standardized yet, but the definition and validation of a signal-based model for conversational speech quality is a work item in ITU-T Study Group 12.…”

Quality model for monitoring QoE in VoIP services

“…In [9] an objective method to assess the speech quality in conversational context, based on both talking and listening qualities, including the impact of delay, is proposed. Two datasets are used: one for training and the other one for validation.…”