No-Reference Image Quality Assessment by Hallucinating Pristine Features

Abstract

Mapping images to deep feature space for comparisons has been wildly adopted in recent learning-based full-reference image quality assessment (FR-IQA) models. Analogous to the classical classification task, the ideal mapping space for quality regression should possess both inter-class separability and intra-class compactness. The inter-class separability that focuses on the discrimination of images with different quality levels has been highly emphasized in existing models. However, the intra-class compactness that maintains small objective quality variance of images with the same or indistinguishable quality escapes the research attention, potentially leading to the perception-biased measures. In this paper, we reveal that such bias is mainly caused by the unsuitable subspace that the features are projected and compared in. To account for this, we develop the Debiased Mapping based quality Measure (DMM), which relies on the orthonormal bases of deep learning features formed by singular value decomposition (SVD). The SVD in deep learning feature domain, which overwhelmingly separates the quality variations with singular values and projection bases, facilitates the quality inference with dedicatedly designed distance measure. Experiments on different IQA databases demonstrate the mapping method is able to efficiently mitigate the perception bias, and the superior performance on quality prediction verifies the effectiveness of our method. The implementation will be publicly available.

Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks

“…TReS (Golestaneh, Dadsetan, and Kitani 2022) proposes to compute local features with CNN and non-local features with self-attention, introduces a per-batch loss for correct ranking and a self-supervision loss between reference and flipped images. FPR (Chen et al 2022) hallucinates pseudo-reference features from the distorted image using mutual learning on reference and distorted images with triplet loss. Attention maps are predicted to aggregate scores over patches.…”

Learning from mixed datasets: A monotonic image quality assessment model

“…Typical NR‐IQA models rely on the natural scene statistics (NSS) construction [4–6] with the assumption that the image quality can be estimated by measuring the destruction level of the NSS. In recent years, many pioneers resort to deep learning technology for both FR‐IQA [7–11] and NR‐IQA [8, 12–15], endowing the IQA models with a strong capability for quality‐aware feature extraction. In particular, a shallow ConvNet was adopted for patch‐based NR‐IQA learning in [12].…”

“…It contains two main components: dataset-shared quality regressor and dataset-specific quality transformer. In the testing phase, only the quality regressor is used no matter which dataset the testing image is from cent years, many pioneers resort to deep learning technology for both FR-IQA [7][8][9][10][11] and NR-IQA [8,[12][13][14][15], endowing the IQA models with a strong capability for quality-aware feature extraction. In particular, a shallow ConvNet was adopted for patch-based NR-IQA learning in [12].…”

Unsupervised blind image quality assessment via joint spatial and transform features

“…Pan et al 22 introduced a distortion aware module in CNN to perform BIQA on different distortions. Chen et al 23 propose an NR-IQA method via feature level pseudo-reference hallucination. Pan et al 24 proposed a multi-branch convolutional neural network to perform NR-IQA.…”