P.1203 evaluation of real OTT video services

Satti, Shahid M.; Schmidmer, Christian; Obermann, Matthias; Bitto, Roland; Agarwal, Lavita; Keyhl, Michael

doi:10.1109/qomex.2017.7965682

Cited by 10 publications

(7 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The P.1203 model has a modular architecture, using a short-term (per-1-second) estimation of video (P.1203.1 [37,38]), and audio quality (P.1203.2 [49]) and their integration with additional information on initial loading delay and stalling into an estimate of streaming session QoE (P.1203.3 [50]). More details about the P.1203 model series and an open-source implementation can be found in [38,74], and an independent evaluation in [96].…”

Section: Overview Of the Competitionmentioning

confidence: 99%

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Raake

Borer²,

Satti³

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

The paper presents a series of three new video quality model standards for the assessment of sequences of up to UHD/4K resolution. They were developed in a competition within the International Telecommunication Union (ITU-T), Study Group 12, in collaboration with the Video Quality Experts Group (VQEG), over a period of more than two years. A large video quality test set with a total of 26 individual databases was created, with 13 used for training and 13 for validation and selection of the winning models. For each database, video quality laboratory tests were run with at least 24 subjects each. The 5-point Absolute Category Rating (ACR) scale was used for rating, calculating Mean Opinion Scores (MOS) as ground-truth. To represent today's commonly applied HTTP-based adaptive streaming context, the test sequences comprise a variety of encoding settings, bitrates, resolutions and framerates for the three codecs H.264/AVC, H.265/HEVC and VP9, applied to a wide range of source sequences of around 8 s duration. Processing was carried out with an FFmpeg-based processing chain developed specifically for the competition, and via upload and encoding through exemplary online streaming services. The resulting data represents the largest, lab-test-based dataset used for video quality model development to date, with a total of around 5,000 test sequences. The paper addresses the three models ultimately standardized in the P.1204 Recommendation series, resulting in different model types and for different applications: (i) Rec. P.1204.3, no-reference bitstream-based, with access to encoded bitstream information; (ii) P.1204.4, pixel-based, using information from the reference and the processed video, and (iii) P.1204.5, no-reference hybrid, using both bitstream-and pixel-information without knowledge of the reference. The paper outlines the development process and provides holistic details about the statistical evaluation, test databases, model algorithms and validation results, as well as a performance comparison with state-of-the-art models.INDEX TERMS bitstream, full reference, http adaptive streaming (HAS), hybrid, pixel-based, QoE, reduced reference, video quality.

show abstract

Section: Overview Of the Competitionmentioning

confidence: 99%

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Raake

Borer²,

Satti³

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Specifically, P.1203 targets HAS-type streaming of segmented H.264-encoded video sessions with lengths between 1 min and 5 min [33,36]. The P.1203 comprises three modules, namely an audio module Pa, a video module PV and an audio-visual integration module Pq [37]. As mentioned in the work of Satti et al (2017) [37], depending on the available information, the Pv module offers four input classes termed "modes":…”

Section: The Itu-t P1203 Standardmentioning

confidence: 99%

“…The P.1203 comprises three modules, namely an audio module Pa, a video module PV and an audio-visual integration module Pq [37]. As mentioned in the work of Satti et al (2017) [37], depending on the available information, the Pv module offers four input classes termed "modes":…”

Section: The Itu-t P1203 Standardmentioning

confidence: 99%

Integrating Visual and Network Data with Deep Learning for Streaming Video Quality Assessment

Margetis

Tsagkatakis

Stamou

et al. 2023

Sensors

View full text Add to dashboard Cite

Existing video Quality-of-Experience (QoE) metrics rely on the decoded video for the estimation. In this work, we explore how the overall viewer experience, quantified via the QoE score, can be automatically derived using only information available before and during the transmission of videos, on the server side. To validate the merits of the proposed scheme, we consider a dataset of videos encoded and streamed under different conditions and train a novel deep learning architecture for estimating the QoE of the decoded video. The major novelty of our work is the exploitation and demonstration of cutting-edge deep learning techniques in automatically estimating video QoE scores. Our work significantly extends the existing approach for estimating the QoE in video streaming services by combining visual information and network conditions.

show abstract

“…While adaptive streaming applications such as HAS are codec agnostic, the P.NATS model is bitstream-based (except for mode 0), which makes the proposed model's performance codec dependent. Satti et al performed a preliminary real-streaming application analysis of the P.1203 model for YouTube, Vimeo, Amazon Instant Video and proprietary DASH-based streaming framework [104]. The authors found the overall performance of Mode 0 and Mode 1 to be quite accurate for H.264 codec configuration, except for the lower quality range where the predictions were found not to be so precise.…”

Section: ) (32)mentioning

confidence: 99%

QoE Modeling for HTTP Adaptive Video Streaming–A Survey and Open Challenges

2019

View full text Add to dashboard Cite

With the recent increased usage of video services, the focus has recently shifted from the traditional quality of service-based video delivery to quality of experience (QoE)-based video delivery. Over the past 15 years, many video quality assessment metrics have been proposed with the goal to predict the video quality as perceived by the end user. HTTP adaptive streaming (HAS) has recently gained much attention and is currently used by the majority of video streaming services, such as Netflix and YouTube. HAS, using reliable transport protocols, such as TCP, does not suffer from image artifacts due to packet losses, which are common in traditional streaming technologies. Hence, the QoE models developed for other streaming technologies alone are not sufficient. Recently, many works have focused on developing QoE models targeting HAS-based applications. Also, the recently published ITU-T Recommendation series P.1203 proposes a parametric bitstream-based model for the quality assessment of progressive download and adaptive audiovisual streaming services over a reliable transport. The main contribution of this paper is to present a comprehensive overview of recent and currently undergoing works in the field of QoE modeling for HAS. The HAS QoE models, influence factors, and subjective test methodologies are discussed, as well as existing challenges and shortcomings. The survey can serve as a guideline for researchers interested in QoE modeling for HAS and also discusses possible future work.INDEX TERMS HTTP adaptive streaming, QoE modeling, TCP, video quality assessment.

show abstract

P.1203 evaluation of real OTT video services

Cited by 10 publications

References 1 publication

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Integrating Visual and Network Data with Deep Learning for Streaming Video Quality Assessment

QoE Modeling for HTTP Adaptive Video Streaming–A Survey and Open Challenges

Contact Info

Product

Resources

About