Blind Image Quality Assessment Based on High Order Statistics Aggregation

Abstract

Blind image quality assessment (BIQA) research aims to develop a perceptual model to evaluate the quality of distorted images automatically and accurately without access to the non-distorted reference images. The state-of-the-art general purpose BIQA methods can be classified into two categories according to the types of features used. The first includes handcrafted features which rely on the statistical regularities of natural images. These, however, are not suitable for images containing text and artificial graphics. The second includes learning-based features which invariably require large codebook or supervised codebook updating procedures to obtain satisfactory performance. These are time-consuming and not applicable in practice. In this paper, we propose a novel general purpose BIQA method based on high order statistics aggregation (HOSA), requiring only a small codebook. HOSA consists of three steps. First, local normalized image patches are extracted as local features through a regular grid, and a codebook containing 100 codewords is constructed by K-means clustering. In addition to the mean of each cluster, the diagonal covariance and coskewness (i.e., dimension-wise variance and skewness) of clusters are also calculated. Second, each local feature is softly assigned to several nearest clusters and the differences of high order statistics (mean, variance and skewness) between local features and corresponding clusters are softly aggregated to build the global quality aware image representation. Finally, support vector regression is adopted to learn the mapping between perceptual features and subjective opinion scores. The proposed method has been extensively evaluated on ten image databases with both simulated and realistic image distortions, and shows highly competitive performance to the state-of-the-art BIQA methods.

On the use of deep learning for blind image quality assessment

“…DIIVINE [34] 0.90 0.88 BRISQUE [31] 0.93 0.91 BLIINDS-II [39] 0.93 0.91 Low Level Features [21] 0.94 0.94 Multi-task CNN [20] 0.93 0.94 HOSA [49] 0.95 0.93 DeepBIQ 0.97 0.96 [31] 0.93 0.91 BLIINDS-II [39] 0.92 0.90 MGMSD [1] 0.88 0.89 Low Level Features [21] 0.89 0.88 Multi-task CNN [20] 0.90 0.91 Shallow CNN [19] 0.90 0.92 DeepBIQ 0.95 0.95 Table 9 Median LCC and median SROCC across 100 trainval-test random splits of the TID2013.…”

NIMA: Neural Image Assessment

“…Perceptual tuning could be quite expensive and time consuming, especially when human opinion is required. In this section, our proposed models are (a) 6.38 (7.16) (b) 6.24 (6.79) (c) 6 Kim et al [16] 0.80 0.80 ---Moorthy et al [39] 0.89 0.88 ---Mittal et al [40] 0.92 0.89 ---Saad et al [41] 0.91 0.88 ---Kottayil et al [42] 0.89 0.88 ---Xu et al [35] 0.96 0.95 ---Bianco et al [7] 0 used to tune a tone enhancement method [43], and an image denoiser [44]. A more detailed treatment is presented in [23].…”

Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning

“…To validate our approach, we conduct extensive evaluations, where ten state-of-the-art NR-IQA methods are compared. We follow the experimental protocol used in three most recent algorithms (i.e., HOSA [44], BIECON [20], and RankIQA [25]), where the reference images are randomly divided into two subsets with 80% for training and 20% for testing, and the corresponding distorted images are divided in the same way to ensure there is no overlap image content between the two sets. All the experiments are under ten times random train-test splitting operation, and the median SROCC and LCC values are reported as final statistics.…”

“…IQA algorithms could be classified into three categories: full-reference IQA (FR-IQA) [50,24,19], reducedreference IQA (RR-IQA) [11], and general purpose noreference IQA (NR-IQA) [46,17,42,51,44,20,25]. Al- the Ground-truth Reference image which is undistorted.…”