Generalized Score Distribution: A Two-Parameter Discrete Distribution Accurately Describing Responses From Quality of Experience Subjective Experiments

Nawała, Jakub; Janowski, Lucjan; Ćmiel, Bogdan; Rusek, Krzysztof; Pérez, Pablo

doi:10.1109/tmm.2022.3205444

Cited by 4 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Definition: An experiment precision comparison method allows assessing whether there is a statistically significant difference in terms of experiment precision between a pair of subjective experiments. 2 Readers interested in learning more about the differences between the GSD and Li2020 models are encouraged to take a look at [11].…”

Section: Experiments Precision Comparison Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Experiment Precision Measures and Methods for Experiment Comparisons

Janowski

Nawała

Hoßfeld

et al. 2023

2023 15th International Conference on Quality of Multimedia Experience (QoMEX)

Self Cite

View full text Add to dashboard Cite

The notion of experiment precision quantifies the variance of user ratings in a subjective experiment. Although there exist measures that assess subjective experiment precision, to the best of our knowledge, there is no systematic framework in the Multimedia Quality Assessment (MQA) field for comparing subjective experiments in terms of their precision. Therefore, the main idea of this paper is to propose a framework for comparing subjective experiments in the field of MQA based on appropriate experiment precision measures. We present three experiment precision measures and three related experiment precision comparison methods. We analyze the performance of the measures by using data from real-world Quality of Experience (QoE) subjective experiments. We believe our experiment precision assessment framework will help compare different subjective experiment methodologies. For example, it may help decide which methodology results in more precise user ratings. This may potentially inform future standardization activities.

show abstract

Section: Experiments Precision Comparison Methodsmentioning

confidence: 99%

“…The GSD model and its application to subjective MQA data are explored in [10] and [11]. For the concise description of the model, we refer the reader to [11].…”

Section: B Gsd Based Measure Gmentioning

confidence: 99%

Experiment Precision Measures and Methods for Experiment Comparisons

Janowski

Nawała

Hoßfeld

et al. 2023

2023 15th International Conference on Quality of Multimedia Experience (QoMEX)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Very few datasets provide individual subjective scores which can be used to understand better the subjects' rating behaviour, design better novel approaches for post-experimental screening of subjects, and accurately describe responses from subjective experiments. [14][15][16][17][18] 3. Demographics Information: Demographics of test participants might have an influence on the QoE.…”

Section: Individual Opinion Scoresmentioning

confidence: 99%

Assessing objective video quality in multi-screen video delivery

Barman,

Reznik,

Martini

2023

Applications of Digital Image Processing XLVI

View full text Add to dashboard Cite

In today’s online video delivery systems, videos are streamed and displayed on various devices with different screen sizes, from large-screen UHD and HDTVs to smaller-screen devices such as mobile phones and tablets. A video will be perceived differently depending on the device’s screen size, pixel density, and viewing distance when viewed on different devices. Quality models which can estimate the relative differences in perceptual quality of a video on different devices can be used to understand the end-user QoE, design optimal encoding ladders for a multi-screen delivery environment, and better rate-adaptation algorithms. We previously presented a BC-KU Multi-Screen dataset1 consisting of subjective scores for different contents encoded in different resolution-bitrate pairs when viewed on three different devices. This paper presents several contributions extending the earlier dataset, which is of interest to the multimedia quality of experience (QoE) community. We first present an in-depth statistical data analysis on the previously unpublished individual subjective ratings of the Multi-Screen dataset. To better understand the relative differences in MOS scores, we present and analyze various demographic information about the test participants. We then evaluate the performance of twelve quality metrics based on five different performance measures. Individual subjective ratings, analysis scripts, and results are available as an open-source dataset. We believe the newly contributed results, files, and scripts will help analyze and design improved, low-complexity parametric models for multi-screen video delivery systems.

show abstract

Generalised score distribution: underdispersed continuation of the beta-binomial distribution

et al. 2023

View full text Add to dashboard Cite

Consider a class of discrete probability distributions with a limited support. A typical example of such support is some variant of a Likert scale, with a response mapped to either the $$\{1, 2, \ldots , 5\}$$ { 1 , 2 , … , 5 } or $$\{-3, -2, \ldots , 2, 3\}$$ { - 3 , - 2 , … , 2 , 3 } set. Such type of data is common for Multimedia Quality Assessment but can also be found in many other research fields. For modelling such data a latent variable approach is usually used (e.g., Ordered Probit). In many cases it is convenient or even necessary to avoid latent variable approach (e.g., when dealing with too small sample size). To avoid it the proper class of discrete distributions is needed. The main idea of this paper is to propose a family of discrete probability distributions with only two parameters that play the same role as the parameters of the normal distribution. We call the new class the Generalised Score Distribution (GSD). The proposed GSD class covers the entire set of possible means and variances, for any fixed and finite support. Furthermore, the GSD class can be treated as an underdispersed continuation of a reparametrized beta-binomial distribution. The GSD class parameters are intuitive and can be easily estimated by the method of moments. We also offer a Maximum Likelihood Estimation (MLE) algorithm for the GSD class and evidence that the class properly describes response distributions coming from 24 Multimedia Quality Assessment experiments. At last, we show that the GSD class can be represented as a sum of dichotomous zero–one random variables, which points to an interesting interpretation of the class.

show abstract

Generalized Score Distribution: A Two-Parameter Discrete Distribution Accurately Describing Responses From Quality of Experience Subjective Experiments

Cited by 4 publications

References 31 publications

Experiment Precision Measures and Methods for Experiment Comparisons

Experiment Precision Measures and Methods for Experiment Comparisons

Assessing objective video quality in multi-screen video delivery

Generalised score distribution: underdispersed continuation of the beta-binomial distribution

Contact Info

Product

Resources

About