On The Benefit of Parameter-Driven Approaches for the Modeling and the Prediction of Satisfied User Ratio for Compressed Video

Abstract

The human eye cannot perceive small pixel changes in images or videos until a certain threshold of distortion. In the context of video compression, Just Noticeable Difference (JND) is the smallest distortion level from which the human eye can perceive the difference between reference video and the distorted/compressed one. Satisfied-User-Ratio (SUR) curve is the complementary cumulative distribution function of the individual JNDs of a viewer group. However, most of the previous works predict each point in SUR curve by using features both from source video and from compressed videos with assumption that the group-based JND annotations follow Gaussian distribution, which is neither practical nor accurate. In this work, we firstly compared various common functions for SUR curve modeling. Afterwards, we proposed a novel parameter-driven method to predict the video-wise SUR from video features. Besides, we compared the prediction results of source-only features based (SRC-based) models and source plus compressed videos features (SRC+PVS-based) models.

Modeling Subject Scoring Behaviors in Subjective Experiments Based on a Discrete Quality Scale

“…For instance, for subjects who tend to provide lower (larger) scores that are accurate, i.e., correlated with the ground truth quality of the stimuli under evaluation, the bias weights corresponding to low (high) opinion scores on the quality scale are significantly greater than those of the others. As a consequence, from Eq (12), we conclude that low (or high) opinion scores are more attractive for this type of subject. Therefore, since this type of subject is not inconsistent, i.e., β does not tend to 0, our scoring model in Eq (23) simply indicates that they have a high probability of choosing a lower (or higher) opinion score when evaluating the quality of any stimulus.…”

“…In any case, if a subject is not inconsistent, the β parameter does not tend to 0; therefore, the probability of choosing a certain opinion score (see Eq (23)) is mainly determined by the total attractiveness (see Eq (12)) of that opinion score for that subject.…”

“…We compared the proposed RMLE approach to four different state-of-the-art approaches to estimate the subjective quality from noisy raw individual ratings: the MOS, the ITU-T Rec BT.500, the approach proposed in [8], [11] as implemented in the publicly available Netflix SUREAL software [30] and a very recent quality recovery approach called ZREC proposed in [12].…”

“…It is important to highlight that the AP algorithm utilized by the SUREAL software was endorsed by the ITU in 2021 as the most comprehensive method for subjective quality recovery (as per Section 12.6 of ITU-R P.913 [22]). As the latest standardized approach, the AP algorithm has recently served as the primary benchmark for evaluating newly proposed methods by various authors [12], [31]. Consequently, in the results section, we also consider SUREAL software as the primary benchmarking approach.…”

“…In several related studies [7], [8], [11], [12], the authors argue that the subject behavior can be reasonably captured by two main characteristics, i.e., subject bias and subject inconsistency. Subject bias is defined as the systematic tendency of a subject to assign lower (negative bias) or greater (positive bias) quality scores than the actual quality scores.…”