Full-reference video quality metric assisted the development of no-reference bitstream video quality metrics for real-time network monitoring

Abstract

High-quality video is being increasingly delivered over Internet Protocol networks, which means that network operators and service providers need methods to measure the quality of experience (QoE) of the video services. In this paper, we propose a method to speed up the development of no-reference bitstream objective metrics for estimating QoE. This method uses full-reference objective metrics, which makes the process significantly faster and more convenient than using subjective tests. In this process, we have evaluated six publicly available full-reference objective metrics in three different databases, the EPFL-PoliMI database, the HDTV database, and the Live Video Wireless database, all containing transmission distortions in H.264 coded video. The objective metrics could be used to speed up the development process of no-reference real-time video QoE monitoring methods that are receiving great interest from the research community. We show statistically that the full-reference metric Video Quality Metric (VQM) performs best considering all the databases. In the EPFL-PoliMI database, SPATIAL MOVIE performed best and TEMPORAL MOVIE performed worst. When transmission distortions are evaluated, using the compressed video as the reference provides greater accuracy than using the uncompressed original video as the reference, at least for the studied metrics. Further, we use VQM to train a lightweight no-reference bitstream model, which uses the packet loss rate and the interval between instantaneous decoder refresh frames, both easily accessible in a video quality monitoring system.

Multivariate statistical technique over QoS variables to analyze video quality metrics on IEEE 802.11ac networks

“…In particular, N R metrics are more interesting from the practical point of view. Using similar steps as we show in this paper, in [8] the authors design N R video quality metrics based on bitstream, assisted by FR metrics, for real-time network monitoring. Notice that other N R standarized metrics are O.23 (ITU-T P.1201 [9]) and G.1070 [10].…”

Application of full-reference video quality metrics in IPTV

“…In [4], VQM was found to perform better than Peak Signalto-Noise Ratio, Structural SIMilarity [6], Multi-scale SSIM [7], Visual Signal-to-Noise Ratio [8] and MOtion-based Video Integrity Evaluation [9]. Therefore, we have selected VQM for our evaluation.…”

“…Therefore, it can be unfeasible to analyze all the videos received by users in an IPTV network for example consisting of 10.000 users using a single computer running the VQM metric. Lightweight no-reference metrics [4] can be used for discarding potential situations to evaluate because they are accurate enough for that task, and where more accuracy is required the full-reference metric VQM could be used. The main work in this paper is focused on determining the number of situations/users that can be analyzed simultaneously using the VQM metric in a computer with good performance.…”

Applicability of Existing Objective Metrics of Perceptual Quality for Adaptive Video Streaming

“…One of the underlying reasons of the limitation of the scope of a metric is related to design-requirements taken into consideration at the time of development [2], [3]. This limitation is very often not clear or not known for subsequent users of the metrics.…”