Segmented Spherical Projection-Based Blind Omnidirectional Image Quality Assessment

Abstract

In contrast with traditional images, omnidirectional image (OI) has a higher resolution and provides the user with an interactive wide field of view. OI with equirectangular projection (ERP) format, as the default for encoding and transmitting omnidirectional visual contents, is not suitable for quality assessment of OI because of serious geometric distortion in the bipolar regions, especially for blind image quality assessment. In this paper, a segmented spherical projection (SSP) based blind omnidirectional image quality assessment (SSP-BOIQA) method is proposed. The OI with ERP format is first converted into that with SSP format, so as to solve the problem of stretching distortion in the bipolar regions of ERP format, but retain the equatorial region of ERP format. On the one hand, considering that the bipolar regions of the SSP format are circular, a local/global perceptual features extraction scheme with fan-shaped window is proposed for estimating the distortion in the bipolar regions of OI. On the other hand, the perceptual features of the equatorial region are extracted with heat map as weighting factor to reflect users' visual behavior. Then, the features extracted from the OI's bipolar and equatorial regions are pooled to predict the quality of distorted OIs. The experiments on two databases, namely CVIQD2018 and MVAQD databases, demonstrate that the proposed SSP-BOIQA method outperforms the state-of-the-art blind quality assessment methods, and is more consistent with human visual perception. INDEX TERMS Omnidirectional image, blind quality assessment, segmented spherical projection, user's visual behavior, fan-shaped window. HAO JIANG received the M.S. degree from Ningbo University, China, in 2008, where he is currently pursuing the Ph.D. degree in signal and information processing. He is also a Researcher with the Ningbo Key Laboratory of Intelligent Household Appliances, Zhejiang Business Technology Institute, China. His research interests mainly include low complexity video processing and omni-vision quality assessment.

Transfer Learning from Vision Transformers or ConvNets for 360-Degree Images Quality Assessmentƒ

“…Datasets: This study is carried out on three 360-IQA datasets namely OIQA [15], CVIQ [1], and MVAQD [16]. OIQA contains 320 impaired images generated from 16 pristine ones using five levels of JPEG compression (JPEG), JPEG2000 compression (JP2K), Gaussian blur (BLUR) and Gaussian white noise (WN).…”

“…Motivated by the lack of insights on the use of ViTs for 360-IQA, and with a focus on their performances, we conduct a study on transfer learning from pre-trained ViTs and ConvNets. Towards this end, two versions of pre-trained ViTs namely B16 and L16 corresponding to the base and large architectures, respectively, are analyzed on three 360-degree datasets, namely OIQA [15], CVIQ [1], and MVAQD [16]. The performances of the selected ViTs are compared to ResNet-50 [2] and EfficientNet-B3 [17] (considered among the best models for 360-IQA) in terms of (i) prediction performances, (ii) computational complexity, and (iii) training behavior by analyzing the evolution of training losses.…”

No-Reference Image Quality Assessment Based on Dual-Domain Feature Fusion

“…Each decision tree will give its own classification results, and the final regression score will be obtained by averaging the classification results of all decision trees. Many studies have shown that RF has higher prediction accuracy and is less prone to over-fitting [ 47 , 48 , 49 ], which is better than SVR in predicting the color images. Therefore, RF is used in this paper to learn the mapping relationship between feature vectors and Mean Opinion Score (MOS), so as to obtain the final quality score.…”

Client-Oriented Blind Quality Metric for High Dynamic Range Stereoscopic Omnidirectional Vision Systems

“…For OIQA, initially, some full-reference OIQA metrics based on PSNR and SSIM were proposed. After that, starting from representations of OI, Zheng et al [ 24 ] proposed a segmented spherical projection-based blind OIQA metric (called SSP-OIQA), in which the bipolar and equatorial regions of OI are obtained by the segmented spherical projection, and different feature extraction schemes are designed for evaluating distorted OI. Jiang et al [ 3 ] proposed a perception-driven blind OIQA framework based on cubemap projection (CMP).…”