Linking distortion perception and visual saliency in H.264/AVC coded video containing packet loss

Abstract

In this paper, distortions caused by packet loss during video transmission are evaluated with respect to their perceived annoyance. In this respect, the impact of visual saliency on the level of annoyance is of particular interest, as regions and objects in a video frame are typically not of equal importance to the viewer. For this purpose, gaze patterns from a task free eye tracking experiment were utilised to identify salient regions in a number of videos. Packet loss was then introduced into the bit stream such as that the corresponding distortions appear either in a salient region or in a non-salient region. A subjective experiment was then conducted in which human observers rated the annoyance of the distortions in the videos. The outcomes show a strong tendency that distortions in a salient region are indeed perceived as much more annoying as compared to distortions in the non-salient region. The saliency of the distorted image content was further found to have a larger impact on the perceived annoyance as compared to the distortion duration. The findings of this work are considered to be of great use to improve prediction performance of video quality metrics in the context of transmission errors.

Attention driven visual QoE: Mechanism and methodologies

“…Naturally, if visual attention mechanism is taken into account, the packets representing attentive objects can be assigned high priority and protected during transmission through error-prone networks. A subjective quality experiment in which video packets were forced to be lost in either salient or non-salient regions has demonstrated that distortions in salient regions are significantly annoying when compared to distortions in non-salient regions [16]. Thus, the overall QoE of received erroneous signals can be increased by applying the attention mechanism in error protection strategies, when compared with the equal error protection (EEP) approach without assigning high priority to attended objects.…”

Psychophysiology-Based QoE Assessment: A Survey

“…In Engelke et al [51], the impact of content saliency relative to distortion location was investigated for H.264 coded video with localised packet loss distortions. It was shown that distortions located in salient regions have a significantly higher impact on quality perception as compared to distortions in non-salient regions.…”

“…Ninassi et al [55] integrated FDMs into several quality metrics to predict JPEG and JPEG2000 distortions, but no improvements were found for the considered metrics. Based on the FDMs and positive outcomes in [51], Engelke et al [56] integrated spatial saliency weighting into a video quality metric, TetraVQM [57], successfully improving the metric's prediction accuracy. The task given to the observer is also known to influence gaze behaviour and it is generally agreed that the integration of task-free eye tracking data into quality prediction models is more successful than when using eye tracking data obtained during quality assessment task [58], [59].…”

Quantifying the importance of preserving video quality in visually important regions at the expense of background content

“…Research in [19] has actually shown that packet-loss artifacts, which are strongly localized in space and time, are more annoying when appearing in the ROI than when appearing in the background of the same video. The extent to which distortions in the ROI would be more annoying than those in the background was, however, not quantified, nor was the difference in annoyance proven to be true for diffused artifacts such as blockiness or blur.…”