Subjective and Objective Quality Assessment of 2D and 3D Foveated Video Compression in Virtual Reality

Jin, Ya‐Qiu; Chen, Meixu; Goodall, Todd; Patney, Anjul; Bovik, Alan C.

doi:10.1109/tip.2021.3087322

Cited by 26 publications

(23 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithms were tested and compared on the LIVE-FBT-FCVR databases [8], where subjective opinion scores (MOS) and difference MOS (DMOS) were collected for 180 foveated / compressed images and 10 reference (190 total) 8K immersive videos.…”

Section: A Evaluation Frameworkmentioning

confidence: 99%

“…The foveation distortions in the subjective study of [8] were created in real time. During the playback, the system reads three immersive (equirectangular) videos, each video uniformly pre-compressed with one of 5 QP levels, and creates the foveation distortion by combining three videos with descending quality order from the fovea to the periphery, given gazing data obtained from the eye tracker.…”

Section: A Evaluation Frameworkmentioning

confidence: 99%

“…However, although user / subjective evaluations are the most reliable method of quality assessment, they are also known to be expensive and time consuming, making it necessary to devise objective foveated quality assessment algorithms. Towards the development of objective algorithms, the authors of [7], [8] developed 2D and 3D foveated / compressed video quality databases in Virtual Reality, and evaluated a wide variety of quality assessment (QA) algorithms on the databases. Among available full reference (FR) foveated algorithms, Foveated Wavelet Quality Index (FWQI) [9] weights the error between the reference image and the foveated / distorted image in the wavelet domain, using a contrast sensitivity function (CSF) model [4], and a visually detectable noise threshold model [10].…”

Section: Introductionmentioning

confidence: 99%

“…Then the entropy differences are calculated and averaged across subbands. We have found that the proposed Foveated Entropic Differencing (FED) algorithm achieves state-of-theart performance on both the 2D and 3D LIVE-FBT-FCVR databases [8] when applied on a frame-by-frame basis.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Evaluating Foveated Video Quality Using Entropic Differencing

Jin

Patney

Bovik

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Virtual Reality is regaining attention due to recent advancements in hardware technology. Immersive images / videos are becoming widely adopted to carry omnidirectional visual information. However, due to the requirements for higher spatial and temporal resolution of real video data, immersive videos require significantly larger bandwidth consumption. To reduce stresses on bandwidth, foveated video compression is regaining popularity, whereby the space-variant spatial resolution of the retina is exploited. Towards advancing the progress of foveated video compression, we propose a full reference (FR) foveated image quality assessment algorithm, which we call foveated entropic differencing (FED), which employs the natural scene statistics of bandpass responses by applying differences of local entropies weighted by a foveation-based error sensitivity function. We evaluate the proposed algorithm by measuring the correlations of the predictions that FED makes against human judgements on the newly created 2D and 3D LIVE-FBT-FCVR databases for Virtual Reality (VR). The performance of the proposed algorithm yields state-of-the-art as compared with other existing full reference algorithms.

show abstract

Section: A Evaluation Frameworkmentioning

confidence: 99%

Section: A Evaluation Frameworkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Evaluating Foveated Video Quality Using Entropic Differencing

Jin

Patney

Bovik

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, the authors of [1], [4] conducted user studies to measure the quality of foveated compression / streaming algorithms they proposed. Towards creating a more generally applicable tools capable of predicting the perceptual quality of foveated / compressed contents, we recently designed 2D and 3D VR foveated video quality databases (LIVE-FBT-FCVR) [5] containing a wide spectrum of foveated and compressed distortions, on which we conducted extensive subjective studies of perceived quality. This new resource is intended to help escalate the development of accurate and efficient objective foveated video quality assessment (FVQA) models, which in turn can be used to help advance the development of improved foveated video compression techniques.…”

Section: Introductionmentioning

confidence: 99%

FOVQA: Blind Foveated Video Quality Assessment

Jin,

Patney,

Webb

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Previous blind or No Reference (NR) video quality assessment (VQA) models largely rely on features drawn from natural scene statistics (NSS), but under the assumption that the image statistics are stationary in the spatial domain. Several of these models are quite successful on standard pictures. However, in Virtual Reality (VR) applications, foveated video compression is regaining attention, and the concept of space-variant quality assessment is of interest, given the availability of increasingly high spatial and temporal resolution contents and practical ways of measuring gaze direction. Distortions from foveated video compression increase with increased eccentricity, implying that the natural scene statistics are space-variant. Towards advancing the development of foveated compression / streaming algorithms, we have devised a no-reference (NR) foveated video quality assessment model, called FOVQA, which is based on new models of space-variant natural scene statistics (NSS) and natural video statistics (NVS). Specifically, we deploy a space-variant generalized Gaussian distribution (SV-GGD) model and a spacevariant asynchronous generalized Gaussian distribution (SV-AGGD) model of mean subtracted contrast normalized (MSCN) coefficients and products of neighboring MSCN coefficients, respectively. We devise a foveated video quality predictor that extracts radial basis features, and other features that capture perceptually annoying rapid quality fall-offs. We find that FOVQA achieves state-of-the-art (SOTA) performance on the new 2D LIVE-FBT-FCVR database, as compared with other leading FIQA / VQA models. we have made our implementation of FOVQA available at: http://live.ece.utexas.edu/research/Quality/ FOVQA.zip.

show abstract

The Role of Artificial Intelligence and Robotic Solution Technologies in Metaverse Design

Gokce Narin

2023

Studies in Big Data

View full text Add to dashboard Cite

Subjective and Objective Quality Assessment of 2D and 3D Foveated Video Compression in Virtual Reality

Cited by 26 publications

References 68 publications

Evaluating Foveated Video Quality Using Entropic Differencing

Evaluating Foveated Video Quality Using Entropic Differencing

FOVQA: Blind Foveated Video Quality Assessment

The Role of Artificial Intelligence and Robotic Solution Technologies in Metaverse Design

Contact Info

Product

Resources

About