Predicting the Quality of Compressed Videos with Pre-Existing Distortions

Abstract

Over the past decade, the online video industry has greatly expanded the volume of visual data that is streamed and shared over the Internet. Moreover, because of the increasing ease of video capture, many millions of consumers create and upload large volumes of User-Generated-Content (UGC) videos. Unlike streaming television or cinematic content produced by professional videographers and cinemagraphers, UGC videos are most commonly captured by naive users having limited skills and imperfect technique, and often are afflicted by highly diverse and mixed in-capture distortions. These UGC videos are then often uploaded for sharing onto cloud servers, where they further compressed for storage and transmission. Our paper tackles the highly practical problem of predicting the quality of compressed videos (perhaps during the process of compression, to help guide it), with only (possibly severely) distorted UGC videos as references. To address this problem, we have developed a novel Video Quality Assessment (VQA) framework that we call 1stepVQA (to distinguish it from two-step methods that we discuss). 1stepVQA overcomes limitations of Full-Reference, Reduced-Reference and No-Reference VQA models by exploiting the statistical regularities of both natural videos and distorted videos. We show that 1stepVQA is able to more accurately predict the quality of compressed videos, given imperfect reference videos. We also describe a new dedicated video database which includes (typically distorted) UGC reference videos, and a large number of compressed versions of them. We show that the 1stepVQA model outperforms other VQA models in this scenario. We are providing the dedicated new database free of charge at https://live.ece.utexas.edu/research/onestep/index.html

RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content

“…Another intriguing and more practical approach to integrating temporal features into BVQA models is to design separable spatial-temporal statistics [4], [48], [54], [55]. Spatial features can be modified to capture temporal effects within BIQA models like BRISQUE, whereby simple frame-differences or spatially displaced frame-differences are deployed [4], [48], [56], [57].…”

“…Prior BVQA methods accounting for temporal distortions, however, either rely on expensive motion estimation [9], [13], or underperform on UGC videos by only accounting for simple frame-difference statistics [4], [47], [48], even including complex CNN models [19], [26]. Here we attempt to exploit more general temporal scene statistics of natural videos to develop and improve BVQA models.…”

“…While full-reference (FR) VQA research is gradually maturing and several algorithms [2], [3] are quite widely deployed, recent attention has shifted more towards creating better no-reference (NR) VQA models that can be used to predict and monitor the quality of authentically distorted UGC videos. One intriguing property of UGC videos, from the data compression aspect, is that the original videos to be compressed often already suffer from artifacts or distortions, making it difficult to decide the compression settings [4]. Similarly, it is of great interest Z. Tu, X. Yu, and A. C. Bovik are with Laboratory for Image and Video Engineering (LIVE), Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, 78712, USA (emails: zhengzhong.tu@utexas.edu, yuxiangxu@utexas.edu, bovik@utexas.edu).…”

“…Many blind video quality models have been proposed to solve the UGC-VQA problem [4], [6]- [19]. Among these, BRISQUE [8], GM-LOG [11], FRIQUEE [12], V-BLIINDS [9], and VIDEVAL [6] have leveraged different sets of natural scene statistics (NSS)-based quality-aware features, using them to train shallow regressors to predict subjective quality scores.…”

Regression or classification? New methods to evaluate no-reference picture and video quality models

“…It has been observed that the empirical distribution of MOSs on a UGC-QA dataset usually follows a unimodal [31][32][33] or multimodal [15,27,[34][35][36] distribution, and the authors of [33] have also shown that the score distribution at different distortion levels is often roughly normally distributed. Therefore, we assume that the distributions of quality scores can be represented by a Gaussian mixture model (GMM) with N components as:…”