SC-IQA: Shift compensation based image quality assessment for DIBR-synthesized views

Abstract

Depth-image-based-rendering (DIBR) has been used to generate the virtual views for Multi-view videos and Freeviewpoint videos. However, the quality assessment of DIBRsynthesized views is very challenging owing to the new types of distortions induced by inaccurate depth maps, dis-occlusions and image inpainting methods. There exist a large number of object shifts and geometric distortions in the synthesized view which the traditional 2D quality metrics may fail to assess. In this paper, we propose a shift compensation based image quality assessment metric (SC-IQA) for DIBR-synthesized views. Firstly, the global geometric shift is compensated roughly by an SURF + RANSAC homography approach. Then, a multi-resolution block matching method, which performs a more accurate matching, is used to precisely compensate the shift and penalize the local geometric distortion as well. In addition, a visual saliency map is also used as a weighting function. To calculate the final overall quality scores, only the worst blocks are utilized since the biggest distortions have the most effects on the overall perceptual quality. The results show that the proposed metric significantly outperforms the state-of-the-art synthesized view dedicated metrics and the conventional 2D IQA metrics.

No-Reference Quality Assessment for 3D Synthesized Images Based on Visual-Entropy-Guided Multi-Layer Features Analysis

“…Specifically, the PSNR and SSIM [ 11 ] are traditional IQA metrics. The VSQA [ 16 ], 3DSwIM [ 17 ], ST-SIAQ [ 18 ], EM-IQA [ 19 ], MW-PSNR [ 20 ], MP-PSNR [ 21 ], SC-IQA [ 22 ], and IDEA [ 23 ] are FR IQA metrics for 3D synthesized images. The APT [ 24 ], MNSS [ 25 ], OUT [ 26 ], NR-MWT [ 28 ], SET [ 30 ], NIQSV [ 31 ], NIQSV+ [ 32 ], CLGM [ 33 ], GANs-NRM [ 34 ], and Wang [ 35 ] are NR metrics designed for 3D synthesized images.…”

“…With this concern, some researchers have proposed IQA metrics targeting 3D synthesized images. These methods are mainly divided into two categories, full-reference (FR) [ 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ] and no-reference (NR) [ 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ].…”

“…Sandić-Stanković et al designed two IQA metrics, i.e., morphological wavelet peak signal-to-noise ratio (MW-PSNR) [ 20 ] and morphological pyramid peak signal-to-noise ratio (MP-PSNR) [ 21 ], in order to evaluate the quality of synthesized geometric distortions in a transform domain. Tian et al matched the horizontal displacement between the reference and synthesized images to devise a shift-compensation-based IQA (SC-IQA) metric [ 22 ]. Li et al presented an FR quality metric for visual views by simultaneously measuring local instance degradation and global appearance (IDEA), in which local distortions were detected by discrete orthogonal moments and global sharpness was measured by super-pixel representation [ 23 ].…”

Perceptual Quality Assessment of DIBR Synthesized Views Using Saliency Based Deep Features

“…The scores are mapped to the ground truth of each view pair using the non-linearity equation as given in [3]. [8] NR IQA 0.6881 0.6261 3 SC-IQA [14] FR IQA 0.6620 0.5960 4 DSCB [9] NR IQA 0.6030 0.5571 5 LOGS [15] FR IQA 0.6280 0.6160 6 MP-PSNR [16] FR IQA 0.6190 0.5809 7 MW-PSNR [17] FR IQA 0.5389 0.4875 8 Wang's [18] NR IQA 0.4338 0.4254 9 APT [3] NR IQA 0.4329 0.4164 10 Jakhetiya [19] NR IQA 0.2715 0.2352 11 OMIQA [20] NR IQA 0.2705 0.2331 12 NIQSV+ [21] NR IQA 0.2324 0.1545…”

“…Also, these results will motivate future researchers to use cosine similarity instead of MSE when distortions are not uniformly distributed across the images. Further,in order to show that the proposed algorithm is performing better than the existing ones, we compared the it with the recently proposed five FR (SSPD [10], SC-IQA [14], LOGS [15], MP-PSNR [16], MW-PSNR [17]) and seven NR quality assessment algorithms (Yan's [8], DSCB [9], Wang's [18], APT [3], Jakhetiya's [19], OMIQA [20], NIQSV+ [21]). As depicted in Table 3, the proposed method outperforms the contemporary IQA metrics developed for DIBR synthesized views.…”