Analysis of problem tokens to rank factors impacting quality in VoIP applications

Abstract

User-perceived quality-of-experience (QoE) in internet telephony systems is commonly evaluated using subjective ratings computed as a Mean Opinion Score (MOS). In such systems, while user MOS can be tracked on an ongoing basis, it does not give insight into which factors of a call induced any perceived degradation in QoE -it does not tell us what caused a user to have a sub-optimal experience. For effective planning of product improvements, we are interested in understanding the impact of each of these degrading factors, allowing the estimation of the return (i.e., the improvement in user QoE) for a given investment. To obtain such insights, we advocate the use of an end-of-call "problem token questionnaire" (PTQ) which probes the user about common call quality issues (e.g., distorted audio or frozen video) which they may have experienced. In this paper, we show the efficacy of this questionnaire using data gathered from over 700,000 end-of-call surveys gathered from Skype (a large commercial VoIP application). We present a method to rank call quality and reliability issues and address the challenge of isolating independent factors impacting the QoE. Finally, we present representative examples of how these problem tokens have proven to be useful in practice.

Lumos: A Library for Diagnosing Metric Regressions in Web-Scale Applications

“…RTC applications track user-perceived quality-of-experience in the form of a call quality feedback survey [11]. Participants are invited to rank the call on a scale from 1 to 5.…”

Design and Deployment of IPSec VPN Using CISCO Network Infrastructure

“…For some codecs the overall performance, become lots higher without the encryption. Their research indicates that even though IPsec can upload safety, it could reduce the VoIP overall performance in terms of higher put off [13][14][15][16]. IPSec…”