A Subjective and Objective Study of Space-Time Subsampled Video Quality

Lee, Dae Yeol; Paul, Somdyuti; Bampis, Christos G.; Ko, Hyunsuk; Kim, Jong-Ho; Jeong, Se Yoon; Homan, Blake; Bovik, Alan C.

doi:10.1109/tip.2021.3137658

Cited by 23 publications

(11 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To analyze the generalizability of the proposed design, we conducted performance comparison of the compared models on traditional VQA databases containing a variety of spatial and temporal artifacts, but single, shared frame rates. For evaluation purposes, we used 5 such legacy VQA datasets, LIVE-VQA [18], LIVE-mobile [21], CSIQ-VQA [19], ETRI-LIVE STSVQ [22] and AVT-VQDB-UHD-1 [23]. These datasets span a variety of spatial resolutions, contain various types of distortions such as scaling, compression, white noise etc., and employ multiple compression schemes such as MPEG, H264, H265, VP9 etc.…”

Section: Performance Comparison On Other Vqa Databasesmentioning

confidence: 99%

Making Video Quality Assessment Models Sensitive to Frame Rate Distortions

Madhusudana,

Birkbeck,

Wang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the problem of capturing distortions arising from changes in frame rate as part of Video Quality Assessment (VQA). Variable frame rate (VFR) videos have become much more common, and streamed videos commonly range from 30 frames per second (fps) up to 120 fps. VFR-VQA offers unique challenges in terms of distortion types as well as in making non-uniform comparisons of reference and distorted videos having different frame rates. The majority of current VQA models require compared videos to be of the same frame rate, but are unable to adequately account for frame rate artifacts. The recently proposed Generalized Entropic Difference (GREED) VQA model succeeds at this task, using natural video statistics models of entropic differences of temporal band-pass coefficients, delivering superior performance on predicting video quality changes arising from frame rate distortions. Here we propose a simple fusion framework, whereby temporal features from GREED are combined with existing VQA models, towards improving model sensitivity towards frame rate distortions. We find through extensive experiments that this feature fusion significantly boosts model performance on both HFR/VFR datasets as well as fixed frame rate (FFR) VQA databases. Our results suggest that employing efficient temporal representations can result much more robust and accurate VQA models when frame rate variations can occur.

show abstract

Section: Performance Comparison On Other Vqa Databasesmentioning

confidence: 99%

Making Video Quality Assessment Models Sensitive to Frame Rate Distortions

Madhusudana,

Birkbeck,

Wang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The performance of CONVIQT on videos modified by synthetic distortions is analyzed in Table IV. CONVIQT obtains superior correlations on recent VQA datasets such as ETRI-LIVE STSVQ [12] and AVT-VQDB-UHD-1 [11], while its performance is slightly lower on legacy datasets like LIVE-VQA [3] and CSIQ-VQA [4]. This may be attributed to the type of distortions present.…”

Section: B Correlation Against Human Judgmentsmentioning

confidence: 99%

“…Advancements in capture and display technologies has led to the widespread adoption of videos of 4K/ultra high definition (UHD) resolution, high dynamic range (HDR) and high frame rate (HFR). This has led to the creation of datasets like AVT-VQDB-UHD-1 [11], ETRI-LIVE STSVQ [12], BVI-HFR [13] and LIVE-YT-HFR [14] which now make it possible to analyze the perceptual implications of these new developments. From an NR algorithm design standpoint, having a single reliable model that is able to accurately quantify both synthetic and authentic artifacts would be quite advantageous, since it could be used in wide ranges of scenario presenting highly diverse of the distortions and combinations of them.…”

Section: Introductionmentioning

confidence: 99%

CONVIQT: Contrastive Video Quality Estimator

Madhusudana¹,

Birkbeck²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Distortion type identification and degradation level determination is employed as an auxiliary task to train a deep learning model containing a deep Convolutional Neural Network (CNN) that extracts spatial features, as well as a recurrent unit that captures temporal information. The model is trained using a contrastive loss and we therefore refer to this training framework and resulting model as CONtrastive VIdeo Quality EstimaTor (CONVIQT). During testing, the weights of the trained model are frozen, and a linear regressor maps the learned features to quality scores in a no-reference (NR) setting. We conduct comprehensive evaluations of the proposed model on multiple VQA databases by analyzing the correlations between model predictions and groundtruth quality ratings, and achieve competitive performance when compared to state-of-the-art NR-VQA models, even though it is not trained on those databases. Our ablation experiments demonstrate that the learned representations are highly robust and generalize well across synthetic and realistic distortions. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning. The implementations used in this work have been made available at https://github.com/pavancm/CONVIQT.

show abstract

“…We refer the reader to [11], [12] for a comprehensive analysis and collection of publicly available databases through 2012. In recent years, more video quality databases [13]- [18] have been proposed to address different scenarios. Although these databases have undoubtedly accelerated the development of VQA algorithms, generally distortions of the chroma channels were not specifically designed.…”

Section: A Subjective Video/picture Quality Databasesmentioning

confidence: 99%

Perceptual Video Quality Prediction Emphasizing Chroma Distortions

Chen

Bampis

et al. 2021

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

Measuring the quality of digital videos viewed by human observers has become a common practice in numerous multimedia applications, such as adaptive video streaming, quality monitoring, and other digital TV applications. Here we explore a significant, yet relatively unexplored problem: measuring perceptual quality on videos arising from both luma and chroma distortions from compression. Toward investigating this problem, it is important to understand the kinds of chroma distortions that arise, how they relate to luma compression distortions, and how they can affect perceived quality. We designed and carried out a subjective experiment to measure subjective video quality on both luma and chroma distortions, introduced both in isolation as well as together. Specifically, the new subjective dataset comprises a total of 210 videos afflicted by distortions caused by varying levels of luma quantization commingled with different amounts of chroma quantization. The subjective scores were evaluated by 34 subjects in a controlled environmental setting. Using the newly collected subjective data, we were able to demonstrate important shortcomings of existing video quality models, especially in regards to chroma distortions. Further, we designed an objective video quality model which builds on existing video quality algorithms, by considering the fidelity of chroma channels in a principled way. We also found that this quality analysis implies that there is room for reducing bitrate consumption in modern video codecs by creatively increasing the compression factor on chroma channels. We believe that this work will both encourage further research in this direction, as well as advance progress on the ultimate goal of jointly optimizing luma and chroma compression in modern video encoders.

show abstract

A Subjective and Objective Study of Space-Time Subsampled Video Quality

Cited by 23 publications

References 32 publications

Making Video Quality Assessment Models Sensitive to Frame Rate Distortions

Making Video Quality Assessment Models Sensitive to Frame Rate Distortions

CONVIQT: Contrastive Video Quality Estimator

Perceptual Video Quality Prediction Emphasizing Chroma Distortions

Contact Info

Product

Resources

About