Subjective video quality assessments for 4K UHDTV

Sotelo, Rafael; Joskowicz, José; Anedda, Matteo; Murroni, Maurizio; Giusto, Daniele D.

doi:10.1109/bmsb.2017.7986225

Cited by 19 publications

(14 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When aiming for a sensitive assessment of encoding quality for high resolutions such as 4K UHD (3840 × 2160 pixels), laboratory tests with a controlled and 4K-appropriate viewing distance of 1.5 to 1.6 times the height of the screen ("1.5H" or "1.6H") are recommended, see [26,27]. As was shown in a number of studies, even in laboratory tests with highquality screens and controlled viewing conditions that follow recommendations such as those in [27,28], in many cases video quality can hardly be distinguished between HD and 4K UHD resolution, specifically depending on the initial quality of the source content used [29,30,31,32,33]. On the other hand, test contents in video quality tests 1 often are rather artificial and not representative of actual target applications such as VoD or live streaming.…”

Section: Introductionmentioning

confidence: 99%

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Raake

Borer²,

Satti³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The paper presents a series of three new video quality model standards for the assessment of sequences of up to UHD/4K resolution. They were developed in a competition within the International Telecommunication Union (ITU-T), Study Group 12, in collaboration with the Video Quality Experts Group (VQEG), over a period of more than two years. A large video quality test set with a total of 26 individual databases was created, with 13 used for training and 13 for validation and selection of the winning models. For each database, video quality laboratory tests were run with at least 24 subjects each. The 5-point Absolute Category Rating (ACR) scale was used for rating, calculating Mean Opinion Scores (MOS) as ground-truth. To represent today's commonly applied HTTP-based adaptive streaming context, the test sequences comprise a variety of encoding settings, bitrates, resolutions and framerates for the three codecs H.264/AVC, H.265/HEVC and VP9, applied to a wide range of source sequences of around 8 s duration. Processing was carried out with an FFmpeg-based processing chain developed specifically for the competition, and via upload and encoding through exemplary online streaming services. The resulting data represents the largest, lab-test-based dataset used for video quality model development to date, with a total of around 5,000 test sequences. The paper addresses the three models ultimately standardized in the P.1204 Recommendation series, resulting in different model types and for different applications: (i) Rec. P.1204.3, no-reference bitstream-based, with access to encoded bitstream information; (ii) P.1204.4, pixel-based, using information from the reference and the processed video, and (iii) P.1204.5, no-reference hybrid, using both bitstream-and pixel-information without knowledge of the reference. The paper outlines the development process and provides holistic details about the statistical evaluation, test databases, model algorithms and validation results, as well as a performance comparison with state-of-the-art models.INDEX TERMS bitstream, full reference, http adaptive streaming (HAS), hybrid, pixel-based, QoE, reduced reference, video quality.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Raake

Borer²,

Satti³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Sotelo et al [13] gave an overview of different subjective studies related to video compression and UHD video and also conducted their own test. They noted the limited availability of high-quality UHD video content for test purposes.…”

Section: Appearance Of Uhd In Qoe Studiesmentioning

confidence: 99%

Comparison of HD and UHD video quality with and without the influence of the labeling effect

et al. 2019

View full text Add to dashboard Cite

The past decades have shown a steady rise in the pixel resolution of digital visualization technologies. New TVs and computer displays available on the consumer market are commonly UHD/4K, and 8K recently started appearing in the commercial world. The added value of such higher resolution is not only indicated by the purported level of visual performance, but also by the labels that emphasize the devices' properties, especially the resolution. However, the genuinely perceived difference between two resolutions, such as HD and UHD, may not have the same effect on user experience as the labels attached to the devices or sequences themselves. In this paper we present four subjective studies in which we investigated the influence of the labeling effect in the context of HD and UHD video. Subjects were shown pairs of either HD or UHD video sequences and had to determine whether there was a difference in quality between the sequences. Two of the subjective tests involved labels in these paired comparisons, indicating to the subject which sequence was which, while the other two tests excluded these labels. For both test sets with and without labels, one test used a 3-point comparison scale for the subjective assessment of visual quality, while the other used a more fine-grained 7-point scale. Our findings show that the sole inclusion of quality labels can strongly impact subjective rating behavior and the overall opinion on UHD quality; also, visual differences between HD and UHD video were rarely noticeable by the subjects. Standards and recommendationsAs with most technologies finding widespread use among the consumers, there are standards that govern how a technology is to be developed, evaluated, and integrated with other technologies. Standards or international recommendations are provided for that interoperability. Among the most relevant international standards on the topic of UHD are documents from the International Telecommunication Union's telecommunication and broadcasting sectors (ITU-T and ITU-R).While ITU-R Rec. BT.709 addresses HDTV (i.e., TV up to 1080p resolution), Rec. BT.2020 [3] covers UHD and the corresponding specifications of dynamic range, color gamut and primaries, bit depths, frame rates, and pixel resolutions. Additional recommendations include Rec. BT.1769 which specifies parameter values for large screen digital imagery and how to design a system that gives viewers visual experiences of a high-sensation of reality-which UHD was also developed for.Subjective quality assessment tests are typically carried out in a rigorous fashion: users are placed in a dedicated testing room with specific lighting conditions and a certain viewing distance to the screen. Guidelines in ITU-T Recommendations P.910, P.911, and P.913 as well as ITU-R Rec. BT.500-13 may be applied in those tests. When subjectively evaluating the quality of UHD systems, the aspect of viewing distance plays a crucial role. Typically, for HDTV applications, human testers are seated at a distance of about 3 H to the TV, where H is the ...

show abstract

“…Milovanovic et al [ 52 ] subjectively compared the coding efficiency of three video coding standards (MPEG-H HEVC, H.264/MPEG-4 AVC, and H.262/MPEG-2). Sotelo et al [ 53 ] presented a subjective quality assessment of HEVC/H.265 compressed 4K Ultra-High-Definition (UHD) videos in a laboratory viewing environment. Kufa et al [ 54 ] explored coding efficiency performance of High Efficiency Video Coding (HEVC) and VP9 compression formats on video content in Full HD and UHD resolutions.…”

Section: Introductionmentioning

confidence: 99%

Impact of Scene Content on High Resolution Video Quality

Uhrina

Holesova

Bienik

et al. 2021

Sensors

View full text Add to dashboard Cite

This paper deals with the impact of content on the perceived video quality evaluated using the subjective Absolute Category Rating (ACR) method. The assessment was conducted on eight types of video sequences with diverse content obtained from the SJTU dataset. The sequences were encoded at 5 different constant bitrates in two widely video compression standards H.264/AVC and H.265/HEVC at Full HD and Ultra HD resolutions, which means 160 annotated video sequences were created. The length of Group of Pictures (GOP) was set to half the framerate value, as is typical for video intended for transmission over a noisy communication channel. The evaluation was performed in two laboratories: one situated at the University of Zilina, and the second at the VSB—Technical University in Ostrava. The results acquired in both laboratories reached/showed a high correlation. Notwithstanding the fact that the sequences with low Spatial Information (SI) and Temporal Information (TI) values reached better Mean Opinion Score (MOS) score than the sequences with higher SI and TI values, these two parameters are not sufficient for scene description, and this domain should be the subject of further research. The evaluation results led us to the conclusion that it is unnecessary to use the H.265/HEVC codec for compression of Full HD sequences and the compression efficiency of the H.265 codec by the Ultra HD resolution reaches the compression efficiency of both codecs by the Full HD resolution. This paper also includes the recommendations for minimum bitrate thresholds at which the video sequences at both resolutions retain good and fair subjectively perceived quality.

show abstract

Subjective video quality assessments for 4K UHDTV

Cited by 19 publications

References 8 publications

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204

Comparison of HD and UHD video quality with and without the influence of the labeling effect

Impact of Scene Content on High Resolution Video Quality

Contact Info

Product

Resources

About