Video Quality Experts Group: current results and future directions

Abstract

Subjective assessment methods have been used reliably for many years to evaluate video quality. They continue to provide the most reliable assessments compared to objective methods. Some issues that arise with subjective assessment include the cost of conducting the evaluations and the fact that these methods cannot easily be used to monitor video quality in real time. Furthermore, traditional, analog objective methods, while still necessary, are not sufficient to measure the quality of digitally compressed video systems. Thus, there is a need to develop new objective methods utilizing the characteristics of the human visual system. While several new objective methods have been developed, there is to date no internationally standardized method.The Video Quality Experts Group (VQEG) was formed in October 1997 to address video quality issues. The group is composed of e xperts from various backgrounds and affiliations, including participants from several internationally recognized organizations working in the field of video quality assessment. The majority of participants are active in the International Telecommunications Union (ITU) and VQEG combines the expertise and resources found in several ITU Study Groups to work towards a common goal. The first task undertaken by VQEG was to provide a validation of objective video quality measurement methods leading to Recommendations in both the Telecommunications (ITU-T) and Radiocommunication (ITU-R) sectors of the ITU. To this end, VQEG designed and executed a test program to compare subjective video quality evaluations to the predictions of a number of proposed objective measurement methods for video quality in the bit rate range of 768 kb/s to 50 Mb/s. The results of this test show that there is no objective measurement system that is currently able to replace subjective testing. Depending on the metric used for evaluation, the performance of eight or nine models was found to be statistically equivalent, leading to the conclusion that no single model outperforms the others in all cases. The greatest achievement of this first validation effort is the unique data set assembled to help future development of objective models.

No-Reference Video Quality Assessment Based on Artifact Measurement and Statistical Analysis

“…The simplest one is a linear function. There are also more sophisticated monotonic functions, such as a third-order polynomial function with monotonicity constraints [27], an S-shaped function [28], a four-parameter logistic function [29], and a five-parameter logistic function [30], [31]. Parameters of these functions are estimated by regression analysis between MOS and the corresponding quality measure in a database.…”

Recent developments in visual quality monitoring by key performance indicators

“…Testers rated video sequences affected by different degradation levels, obtained using the proposed exposure generation model. Similarly to the CONTENT 2009 experimental set-up (described in Section 5.1), four test sequences, provided by the VQEG [19], were used: "Betes", "Autumn", "Football", and "Susie". The video sequences reflected two different content characteristics (i.e.…”

Statistical evaluation of image quality measures

“…13 However, despite these oft cited criticisms of MSE-based quality measures there has been a recent resurgence of SNR/PSNR metrics. 17,18 For example, studies of the video quality expert Group ͑VQEG͒ 17 have shown that the PSNR measure is a very good indicator of subjective preference in video coding.…”

“…Obviously such quality monitoring can only be realized with objective methods. 17,18 It must be pointed out, however, that subjective assessment, albeit costly and time consuming, if not impractical, is accurate. Objective methods, on the other hand, can at best try to emulate the performance of subjective methods, and utilize knowledge of the human visual system.…”