Understanding and estimating quality of experience in WebRTC applications

García, Boni; Gallego, Micael; Gortázar, Francisco; Bertolino, Antonia

doi:10.1007/s00607-018-0669-7

Cited by 33 publications

(14 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another tool of the Selenium ecosystem is Selenium-Jupiter, an JUnit 5 extension for Selenium providing seamless integration with Docker. References [14,15] show how to use Selenium-Jupiter to implement automated tests aimed to evaluate the Quality of Experience (QoE) of real-time communication using web browsers (WebRTC) [16].…”

Section: Related Workmentioning

confidence: 99%

A Survey of the Selenium Ecosystem

et al. 2020

Self Cite

View full text Add to dashboard Cite

Selenium is often considered the de-facto standard framework for end-to-end web testing nowadays. It allows practitioners to drive web browsers (such as Chrome, Firefox, Edge, or Opera) in an automated fashion using different language bindings (such as Java, Python, or JavaScript, among others). The term ecosystem, referring to the open-source software domain, includes various components, tools, and other interrelated elements sharing the same technological background. This article presents a descriptive survey aimed to understand how the community uses Selenium and its ecosystem. This survey is structured in seven categories: Selenium foundations, test development, system under test, test infrastructure, other frameworks, community, and personal experience. In light of the current state of Selenium, we analyze future challenges and opportunities around it.

show abstract

Section: Related Workmentioning

confidence: 99%

A Survey of the Selenium Ecosystem

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…While such a testing framework could be used to further explore the QoS-to-QoE mapping, this is not explicitly addressed by the authors. In their subsequent work, Garcia et al [5] discuss a set of measurable factors to estimate the QoE of a WebRTC application, namely call establishment time, end-to-end delay, audio quality, video quality, and audiovisual quality.…”

Section: Background and Related Workmentioning

confidence: 99%

Exploring the Usefulness of Machine Learning in the Context of WebRTC Performance Estimation

Ammar

Moor

Skorin‐Kapov

et al. 2019

2019 IEEE 44th Conference on Local Computer Networks (LCN)

View full text Add to dashboard Cite

We address the challenge faced by service providers in monitoring Quality of Experience (QoE) related metrics for WebRTC-based audiovisual communication services. By extracting features from various application-layer performance statistics, we explore the potential of using machine learning (ML) models to estimate perceivable quality impairments and to identify root causes. We argue that such performance-related data can be valuable and informative from a QoE assessment point of view, by allowing to identify the party/parties in a call that is/are experiencing quality impairments, and to trace the origins and causes of the problem. The paper includes case studies of multi-party videoconferencing that are established in a laboratory environment and exposed to various network disturbances and CPU limitations. Our results show that perceivable quality impairments in terms of video blockiness and audio distortions may be estimated with a high level of accuracy, thus proving the potential of exploiting ML models for automated QoE-driven monitoring and estimation of WebRTC performance.

show abstract

“…Before a WebRTC peer can start a media session and the media can start flowing, a group of signaling activities has to take place: the SDP negotiation and the gathering of the ICE candidates. Due to this signaling, media in the viewer starts a few moments after the media started in the presenter [23]. In order to alleviate this problem, we present a simple but effective mechanism.…”

mentioning

confidence: 99%

Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications

et al. 2019

Self Cite

View full text Add to dashboard Cite

WebRTC is the umbrella term for several emergent technologies aimed to exchange real-time media in the Web. Like other media-related services, the perceived quality of WebRTC communication can be measured using Quality of Experience (QoE) indicators. QoE assessment methods can be classified as subjective (users’ evaluation scores) or objective (models computed as a function of different parameters). In this paper, we focus on VMAF (Video Multi-method Assessment Fusion), which is an emergent full-reference objective video quality assessment model developed by Netflix. VMAF is typically used to assess video streaming services. This paper evaluates the use of VMAF in a different type of application: WebRTC. To that aim, we present a practical use case built on the top of well-known open source technologies, such as JUnit, Selenium, Docker, and FFmpeg. In addition to VMAF, we also calculate other objective QoE video metrics such as Visual Information Fidelity in the pixel domain (VIFp), Structural Similarity (SSIM), or Peak Signal-to-Noise Ratio (PSNR) applied to a WebRTC communication in different network conditions in terms of packet loss. Finally, we compare these objective results with a subjective evaluation using a Mean Opinion Score (MOS) scale to the same WebRTC streams. As a result, we found a strong correlation of the subjective video quality perceived in WebRTC video calls with the objective results computed with VMAF and VIFp in comparison with SSIM and PSNR and their variants.

show abstract

Understanding and estimating quality of experience in WebRTC applications

Cited by 33 publications

References 40 publications

A Survey of the Selenium Ecosystem

A Survey of the Selenium Ecosystem

Exploring the Usefulness of Machine Learning in the Context of WebRTC Performance Estimation

Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications

Contact Info

Product

Resources

About