End-to-End Blind Image Quality Assessment Using Deep Neural Networks

Abstract

We propose a multi-task end-to-end optimized deep neural network (MEON) for blind image quality assessment (BIQA). MEON consists of two sub-networks-a distortion identification network and a quality prediction network-sharing the early layers. Unlike traditional methods used for training multi-task networks, our training process is performed in two steps. In the first step, we train a distortion type identification sub-network, for which large-scale training samples are readily available. In the second step, starting from the pre-trained early layers and the outputs of the first sub-network, we train a quality prediction sub-network using a variant of the stochastic gradient descent method. Different from most deep neural networks, we choose biologically inspired generalized divisive normalization (GDN) instead of rectified linear unit as the activation function. We empirically demonstrate that GDN is effective at reducing model parameters/layers while achieving similar quality prediction performance. With modest model complexity, the proposed MEON index achieves state-of-the-art performance on four publicly available benchmarks. Moreover, we demonstrate the strong competitiveness of MEON against state-of-the-art BIQA models using the group maximum differentiation competition methodology.

Subjective and Objective De-Raining Quality Assessment Towards Authentic Rain Image

“…To evaluate the performance of the proposed B-FEN model, we conduct the experiments on our IVIPC-DQA database. Due to the absence of specific quality metric for de-rained image, we compare the proposed B-FEN model with our previous B-GFN [16] and some representative general-purpose image quality assessment models, which include 10 opinion-aware (OA) metrics (i.e., BIQI [18], BLIINDS II [41], BRISQUE [22], DIIVINE [21], M3 [42], NFERM [43], TCLT [19], MEON [31], DB-CNN [32] and WaDIQaM [33]), and 4 opinion-unaware (OU) metrics (i.e., NIQE [44], ILNIQE [45], QAC [46], and LPSI [47]). Meanwhile, two popular unidirectional feature embedding networks, i.e., DenseNet-161 [37] and ResNet-152 [48] are also involved in our comparison, which are categorized as OA metric in the following section.…”

“…After building the IVIPC-DQA database, we further develop an efficient objective model to predict the human perception towards the de-rained image. Recently, many deep learning based NR-IQA models [31]- [33] have explored various efficient network structures for evaluating the uniform distortions, which achieve state-of-the-art quality prediction accuracy via a common unidirectional feature embedding (UFE) architecture as shown in Fig. 9 (a).…”

“…9 (b). More specifically, the forward feature embedding aims to extract the perceptual quality related global information, which is similar to existing methods [31]- [33]. By contrast, the backward feature embedding attempts to incorporate the global information into multiple local features, which could be captured from the intermediate layers of a convolutional neural network (CNN) [34]- [36].…”

“…All OA metrics need training process to determine the parameters of quality assessment model. Following the setup of [31]- [33], we randomly separate the IVIPC-DQA database into the non-overlapped training and testing sets, which include 80% and 20% images respectively.…”

Deep Activation Pooling for Blind Image Quality Assessment

“…The blind IQA (BIQA) is a new trend of NR-IQA research, and it does not need to know the distortion types. Most existing BIQA methods are opinion-aware [10][11][12][13], which trains a regression model from numerous training images (source and distortion images) with the homologous human subjective scores. Specifically, the features are first obtained from the training images, and then both feature vectors and corresponding human subjective scores are employed to train a regression model.…”