The history of video quality model validation

Pinson, Margaret H.; Staelens, Nicolas; Webster, A.

doi:10.1109/mmsp.2013.6659332

Cited by 10 publications

(3 citation statements)

References 6 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A multitude of subjective studies were conducted by various research groups such as for example the Video Quality Experts Group (VQEG) 1 with some of these being overviewed in [11]. Moreover, the results of many studies have been made accessible to other researchers as public VQA databases [6].…”

Section: A Subjective Vqa Methodsmentioning

confidence: 99%

A novel methodology for mapping objective video quality metrics to the subjective MOS scale

Moldovan

Ghergulescu

Muntean

2014

2014 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting

View full text Add to dashboard Cite

Abstract-With the rapid growth in video-based services, and as users are becoming increasingly quality-aware, the reliable estimation of video quality has become extremely important. While a multitude of objective Video Quality Assessment (VQA) metrics with various performance and complexity have been proposed, the nonlinearity of video quality and the lack of clear interpretations of the metrics make difficult to understand how the objective metric values reflect the video quality as perceived subjectively in terms of Mean Opinion Scores (MOS). This paper proposes and evaluates a methodology for mapping objective VQA metric values to subjective MOS scores based on publicly available VQA databases. Three different databases were used for comparing the performance of various objective metrics and evaluating the proposed methodology.Index Terms-Video quality assessment (VQA), mean opinion score (MOS), subjective methods, objective metrics, video quality mapping, performance evaluation, public video quality databases.

show abstract

Section: A Subjective Vqa Methodsmentioning

confidence: 99%

A novel methodology for mapping objective video quality metrics to the subjective MOS scale

Moldovan

Ghergulescu

Muntean

2014

2014 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting

View full text Add to dashboard Cite

show abstract

“…Examples of databases include: VQEG FRTV Phase I and HDTV that contains SD and HD television sequences [46], [47]; LIVE database analysed MPEG-2 and H.264 compression, and simulated transmissions over IP wired and wireless networks [48]; LIVE Mobile database contains H.264 compressed videos with distortions such as packet loss, frame freeze, and rate adaptation, that were evaluated on smartphone and tablet mobile devices [49], [50]; ECVQ and EVVQ databases contain MPEG-4 and H.264 compressed videos at CIF and VGA resolution [51], [52]; MMSP SVD database contains sequences encoded using scalable video coding [53]; SAVAM database contains eye-tracking data for video sequences at HD and UHD resolution [54].…”

Section: Public Vqa Databasesmentioning

confidence: 99%

VQAMap: A Novel Mechanism for Mapping Objective Video Quality Metrics to Subjective MOS Scale

Moldovan

Ghergulescu²,

Muntean

2016

IEEE Trans. on Broadcast.

View full text Add to dashboard Cite

The reliable estimation of video quality has become increasingly important with the proliferation of online video services and users becoming more quality aware. A multitude of objective video quality assessment (VQA) metrics with various performance and complexity have been proposed. However, their applicability in real-world scenarios is limited by the lack of clear interpretations of how the metric values reflect the subjective user-perceived video quality. This paper proposes a novel mechanism called VQAMap, that uses data from public VQA databases and enables to automatically create generic rules for mapping the values of objective VQA metrics to the subjective MOS scale (i.e., 1 -bad, 2 -poor, 3 -fair, 4 -good, and 5 -excellent). An extensive evaluation study of VQAMap was conducted using data from three public VQA databases, considering six objective VQA metrics. The results analysis has shown that VQAMap provides mapping rules with quality estimation accuracy as high as 95%, while the variation in performance being caused by the varying accuracy of the different objective metrics.Index Terms-Video quality assessment (VQA), mean opinion score (MOS), subjective methods, objective metrics, video quality mapping, performance evaluation, public video quality databases.

show abstract

“…Efforts to create objective metrics for digital video that correlate better with subjective assessments have been made since the early days of digital transmission [14,2]. The Alliance for Telecommunications Industry Solutions (ATIS) performed the first large scale validation test of such metrics in 1994-95, and since the late 1990s the Video Quality Experts Group (VQEG) has performed several similar tests resulting in various standards [8].…”

Section: Introductionmentioning

confidence: 99%

OpenVQ

Skarseth

Bjørlo²,

Halvorsen

et al. 2016

Proceedings of the 24th ACM International Conference on Multimedia

View full text Add to dashboard Cite

This paper presents an open source video quality assessment tool called OpenVQ and OPVQ, an objective quality metric based on work of the Video Quality Experts Group. It was primarily developed to satisfy the need of the community for a freely accessible and easy-to-use quality assessment tool, which would remove the need for using throughly outdated methods that is still prevalent because only those are easily available.OpenVQ is implemented as a toolbox of full-reference quality assessment methods, which can be extended with improved methods in the future. The primary method provided by OpenVQ at this time is OPVQ, the Open Perceptual Video Quality metric. It is very much inspired by the formal model of ITU-T Rec. J.247 Annex B, however with many modifications to deal with errors and inconsistencies in formulas and pseudo code, and restrictions to avoid violating known patents. OPVQ has been validated with several IRRCyN datasets and was found to yield results similar to those reported in ITU.T J.247.OpenVQ implements also PSNR (Peak signal-to-noise-ratio), because it is in high demand in spite of its bad performance, and SSIM (Structural similarity index), an image quality metric that is known to perform quite well in many cases.

show abstract

The history of video quality model validation

Cited by 10 publications

References 6 publications

A novel methodology for mapping objective video quality metrics to the subjective MOS scale

A novel methodology for mapping objective video quality metrics to the subjective MOS scale

VQAMap: A Novel Mechanism for Mapping Objective Video Quality Metrics to Subjective MOS Scale

OpenVQ

Contact Info

Product

Resources

About