Perceptual Video Quality Prediction Emphasizing Chroma Distortions

Abstract

Measuring the quality of digital videos viewed by human observers has become a common practice in numerous multimedia applications, such as adaptive video streaming, quality monitoring, and other digital TV applications. Here we explore a significant, yet relatively unexplored problem: measuring perceptual quality on videos arising from both luma and chroma distortions from compression. Toward investigating this problem, it is important to understand the kinds of chroma distortions that arise, how they relate to luma compression distortions, and how they can affect perceived quality. We designed and carried out a subjective experiment to measure subjective video quality on both luma and chroma distortions, introduced both in isolation as well as together. Specifically, the new subjective dataset comprises a total of 210 videos afflicted by distortions caused by varying levels of luma quantization commingled with different amounts of chroma quantization. The subjective scores were evaluated by 34 subjects in a controlled environmental setting. Using the newly collected subjective data, we were able to demonstrate important shortcomings of existing video quality models, especially in regards to chroma distortions. Further, we designed an objective video quality model which builds on existing video quality algorithms, by considering the fidelity of chroma channels in a principled way. We also found that this quality analysis implies that there is room for reducing bitrate consumption in modern video codecs by creatively increasing the compression factor on chroma channels. We believe that this work will both encourage further research in this direction, as well as advance progress on the ultimate goal of jointly optimizing luma and chroma compression in modern video encoders.

RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content

“…Another intriguing and more practical approach to integrating temporal features into BVQA models is to design separable spatial-temporal statistics [4], [48], [54], [55]. Spatial features can be modified to capture temporal effects within BIQA models like BRISQUE, whereby simple frame-differences or spatially displaced frame-differences are deployed [4], [48], [56], [57].…”

“…Most previous BVQA models have overlooked the importance of chromatic features, whereas recent work [6], [10], [55], [66] has shown the efficacy of color components for UGC video quality prediction. Previous efforts on the chromatic statistics of quality models involve opponent color spaces such as YIQ/YUV [15], [71], [72], LMS [12], [72], perceptual color spaces like CIELAB [10], [12], [73], HSI [12], [74], Yellow color [12], and "colorfulness" features [6], [13], [75].…”

“…Hence, we extract NSS-34 features from the four luma feature maps (Y, GM, LoG, DoG) at two scales: the original image scale and a reduced (by a factor two) resolution. However, we only extract NSS-34 features at the half scale on the twelve chroma feature maps, since frames are often compressed in YUV420 format, which already contain chroma information in reduced scale; additionally, it has also been observed that humans are more sensitive to luma distortions than chroma distortions [55]. To sum up, the entire collection of spatial features is collected by applying two-scales NSS-34 to the luma feature maps and single-scale NSS-34 to the chromatic maps, yielding a total of 34 × 2 × 4 + 34 × 1 × 12 = 680 features.…”

RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content

“…Another intriguing and more practical approach to integrating temporal features into BVQA models is to design separable spatial-temporal statistics [4], [48], [54], [55]. Spatial features can be modified to capture temporal effects within BIQA models like BRISQUE, whereby simple frame-differences or spatially displaced frame-differences are deployed [4], [48], [56], [57].…”

A Temporal Statistics Model For UGC Video Quality Prediction

“…Another intriguing and more practical approach is to design spatial-temporal separable statistics [15,[36][37][38]. Spatial features can be devised upon leading BIQA models such as BRISQUE, whereby temporal features are extended from simple frame-differences to the spatially displaced framedifferencing domains [15,36,39,40], which have been shown to be simple and effective temporal quality indicators.…”