Stereoscopic Video Quality Assessment Using Binocular Energy

Abstract

Abstract-Stereoscopic imaging is becoming increasingly popular. However, to ensure the best quality of experience, there is a need to develop more robust and accurate objective metrics for stereoscopic content quality assessment. Existing stereoscopic image and video metrics are either extensions of conventional 2D metrics (with added depth or disparity information) or are based on relatively simple perceptual models. Consequently, they tend to lack the accuracy and robustness required for stereoscopic content quality assessment. This paper introduces full-reference stereoscopic image and video quality metrics based on a Human Visual System (HVS) model incorporating important physiological findings on binocular vision. The proposed approach is based on the following three contributions. First, it introduces a novel HVS model extending previous models to include the phenomena of binocular suppression and recurrent excitation. Second, an image quality metric based on the novel HVS model is proposed. Finally, an optimised temporal pooling strategy is introduced to extend the metric to the video domain. Both image and video quality metrics are obtained via a training procedure to establish a relationship between subjective scores and objective measures of the HVS model. The metrics are evaluated using publicly available stereoscopic image/video databases as well as a new stereoscopic video database. An extensive experimental evaluation demonstrates the robustness of the proposed quality metrics. This indicates a considerable improvement with respect to the state-of-the-art with average correlations with subjective scores of 0.86 for the proposed stereoscopic image metric and 0.89 and 0.91 for the proposed stereoscopic video metrics.

Full-Reference Stereoscopic Video Quality Assessment Using a Motion Sensitive HVS Model

“…In contrast to previously reported research, this paper introduces a novel HVS model which directly encodes temporal complexity to mimic the spatiotemporal characteristics of human stereoscopic perception. The proposed HVS model generalises our previous model [8] which was also based on the HVS but excluded influence of motion sensitivity. The novel model is applied here to stereoscopic video quality assessment thereby demonstrating the importance of incorporating motion sensitivity in perceptual tasks.…”

“…Simple cells and complex cells are the main cell types in the primary visual cortex that are responsible for binocular vision in the HVS. Several physiological models have been proposed to mimic their properties [12], [13] and have been used to build image and video quality metrics [2], [8]. These models are based on the computation of binocular signals using analytical methods to estimate the perceptual quality of stereoscopic images and videos.…”

“…This results in two types of complex cells: non-motion sensitive and motion sensitive complex cells. Non-motion sensitive complex cells respond to spatial orientation regardless of whether motion is present or not, similarly to the complex cells introduced in [8]; in contrast, motion sensitive complex cells respond to spatial orientation only in the presence of motion [14], [15]. Different velocity response functions are investigated to model the behaviour of these cells as a function of the amplitude of the motion at a given orientation and scale, taking into account minimum velocity requirements.…”

Stereoscopic video quality assessment based on 3D convolutional neural networks

“…Taking the temporal characteristics of video and binocular perception in HVS into account, a RR SVQA method was proposed by Yu et al [18] . In [19] , Galkandage et al presented a FR SVQA method built on a HVS model incorporating the phenomena of binocular suppression and recurrent excitation. Despite of these achievements, all of the above methods are FR or NR methods without practical value and few NR methods were presented.…”

No-Reference Stereoscopic Video Quality Assessment Algorithm Using Joint Motion and Depth Statistics

“…They computed the JND value between reference and distorted depth maps to measure the quality of an S3D video. Galkandage et al [34] proposed S3D FR IQA and VQA metrics based on an HVS model and temporal features. They computed the Energy Quality Metric (EBEQM) scores to measure the spatial quality and finally pooled these scores by using empirical methods to estimate the overall quality score of an S3D video.…”