Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Abstract

The paper presents a series of three new video quality model standards for the assessment of sequences of up to UHD/4K resolution. They were developed in a competition within the International Telecommunication Union (ITU-T), Study Group 12, in collaboration with the Video Quality Experts Group (VQEG), over a period of more than two years. A large video quality test set with a total of 26 individual databases was created, with 13 used for training and 13 for validation and selection of the winning models. For each database, video quality laboratory tests were run with at least 24 subjects each. The 5-point Absolute Category Rating (ACR) scale was used for rating, calculating Mean Opinion Scores (MOS) as ground-truth. To represent today's commonly applied HTTP-based adaptive streaming context, the test sequences comprise a variety of encoding settings, bitrates, resolutions and framerates for the three codecs H.264/AVC, H.265/HEVC and VP9, applied to a wide range of source sequences of around 8 s duration. Processing was carried out with an FFmpeg-based processing chain developed specifically for the competition, and via upload and encoding through exemplary online streaming services. The resulting data represents the largest, lab-test-based dataset used for video quality model development to date, with a total of around 5,000 test sequences. The paper addresses the three models ultimately standardized in the P.1204 Recommendation series, resulting in different model types and for different applications: (i) Rec. P.1204.3, no-reference bitstream-based, with access to encoded bitstream information; (ii) P.1204.4, pixel-based, using information from the reference and the processed video, and (iii) P.1204.5, no-reference hybrid, using both bitstream-and pixel-information without knowledge of the reference. The paper outlines the development process and provides holistic details about the statistical evaluation, test databases, model algorithms and validation results, as well as a performance comparison with state-of-the-art models.INDEX TERMS bitstream, full reference, http adaptive streaming (HAS), hybrid, pixel-based, QoE, reduced reference, video quality.

AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications

“…In general, based on the input information used for quality assessment, video quality models can be classified into several categories [17,18,19,20,21,22], for example, metadata-based, pixel-based, bitstream-based, or hybrid models. For the first type, namely, metadata-based models, the input information available is limited to video resolution, video bitrate, video framerate, and video codec.…”

“…The third type of models is referred to as bitstreambased, which are usually NR models that rely only on the encoded bitstream, typically without a full decoding, to estimate visual quality. Based on the extent to which bitstream information is used, bitstream models can be categorized into the following modes of operation [22]: Mode 0 (e.g. ITU P.1203.1 Mode 0 [26]), Mode 1 (e.g.…”

“…To cover newer videos codecs and higher resolutions, newer video quality short-term models were poposed in a subsequent standardization competition in ITU-T Study Group 12 (SG12). One of the standardized models is the high-accuracy, bitstream-based video quality model P.1204.3 (Mode 3) [27,39,22]. It is the candidate by the authors of this paper winning the bitstream-model competition in ITU-T SG12.…”

Bitstream-Quality-Estimation Model for Tile-Based VR Video Streaming Services

“…The FR pixelbased model takes the original and degraded images as input and can evaluate the impact of video codec and contents on the quality because it uses both original and decoded images [12][13][14]. Since the FR pixel-based model uses the video signals, the model does not need to be optimized/trained per codec [15]. However, the FR pixel-based model needs a large amount of computational power because original and de-graded videos are used.…”

“…However, NR pixel-based models have not been standardized due to the lack of quality-estimation accuracy [18] [19]. In addition, because of the absence of source information, NR pixel-based models are usually less accurate than the corresponding FR and RR counterparts [15]. On the one hand, NR bitstream-quality-estimation models have been standardized with a good quality-estimation accuracy [20] [21].…”