Deep Activation Pooling for Blind Image Quality Assessment

Abstract

Driven by the rapid development of digital imaging and network technologies, the opinion-unaware blind image quality assessment (BIQA) method has become an important yet very challenging task. In this paper, we design an effective novel scheme for opinion-unaware BIQA. We first utilize the convolutional maps to select high-contrast patches, and then we utilize these selected patches of pristine images to train a pristine multivariate Gaussian (PMVG) model. In the test stage, each high-contrast patch is fitted by a test MVG (TMVG) model, and the local quality score is obtained by comparing with the PMVG. Finally, we propose the deep activation pooling (DAP) to automatically emphasize the more important scores and suppress the less important ones so as to obtain the overall image quality score. We verify the proposed method on two widely used databases, that is, the computational and subjective image quality (CSIQ) and the laboratory for image and video engineering (LIVE) databases, and the experimental results demonstrate that the proposed method achieves better results than the state-of-the-art methods.

No-reference Automatic Quality Assessment for Colorfulness-Adjusted, Contrast-Adjusted, and Sharpness-Adjusted Images Using High-Dynamic-Range-Derived Features

“…Spearman's rank ordered a correlation coefficient (SROCC) and Pearson's linear correlation coefficient (PLCC), which were used to compare the performances. We used the Spearman and Pearson coefficients methods for performance evaluation because these methods have been widely used for correlation metrics between image quality assessment and subjective scores [21][22][23][24][25].…”

“…However, other machine learning metrics such as the support vector machine can be used in the training process. Because SVM-based quality evaluation methods have already been used in the detection on artifacts caused by compression [21][22][23][24][25], it is possible to use the SVM method for our proposed method. In addition, if there are additional features including the six features, neural network methods can be used for the evaluation of adjusted images.…”

“…Accordingly, objective assessment methods that automatically evaluate no-reference images have been extensively researched, which resulted in developments such as just-noticeable difference (JND)-based techniques [19,20]. Furthermore, natural scene statistic (NSS)-based methods have been developed to improve the correlation with subjective evaluations considering HVS properties [21][22][23][24][25]. The natural image-quality index (NIQE) [24] method uses the statistical difference between the fitted Gaussian functions of the high-quality and low-quality images in spatial domains.…”

Multi-Pooled Inception Features for No-Reference Image Quality Assessment

“…Similarly, Ji et al [23] utilized a CNN and an LSTM for NR-IQA, but the deep features were extracted from the convolutional layers of a VGG16 [24] network. In contrast to other algorithms, Zhang et al [25] proposed an opinion-unaware deep method. Namely, high-contrast image patches were selected using deep convolutional maps from pristine images which were used to train a multi-variate Gaussian model.…”

Potential of Deep Features for Opinion-Unaware, Distortion-Unaware, No-Reference Image Quality Assessment

“…Very recent methods in the opinion/distortion unaware domain use (IL)NIQE features, but consider activations in pre-trained deep neural networks to select salient patches. They assign more weight to scores from those patches over others during score aggregation [17]. (IL)NIQE features can also reliably predict quality of multi-spectral images [18].…”