Video quality subjective assessment considering cognitive criteria and user preferences on video content

Abstract

There are several subjective test methodologies for assessing video quality, being that each of these methods was developed by considering specific degradation factors in video quality. Therefore, for video streaming services that use TCP as transport protocol different evaluation criteria should be considered as compared to current methods, because the impairment are only pauses. In this paper, the concepts of the mechanisms that integrate perceptual and cognitive processes are studied, relating them to a test environment for subjective evaluation of video, which is limited to streaming video service transmitted via HTTP / TCP. For this reason, some recommendations to perform subjective assessment of streaming video service are proposed, taking into account perceptual and cognitive criteria. In this work, subjective tests are conducted, evaluators with different content preferences assess 3 different video content. Results show that the assessors' QoE depends strongly on video content preferences. Thus, assessors who are interested in the video content are more drastic in the ratings. In the case of sport videos, the Mean Opinion Score (MOS) value of assesors with explicit preference in sports reachs only 54% of the MOS obtained by assessors without interest in sports. Furthermore, the roll of the instructor during the test is also evaluated.

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

“…Thus, real-life scenarios and QoE definition [12], [56] provide arguments supporting the use of longer sequences that can attract the interest of the participants in the tests and immerse them in the narrative [57]- [60]. Other research questions motivate the use of stimuli longer in duration than is typical in standard tests allowing to study a more realistic experience and deeper user engagement [61] or higher-order cognitive processes [62]. Additionally, users of 360°videos may spend some time at the beginning of the sequence to explore the whole scene before focusing on the assessments [63], [64].…”