Blindly Assess Image Quality in the Wild Guided by a Self-Adaptive Hyper Network

Su, Shaolin; Yan, Qingsen; Zhu, Yu; Zhang, Cheng; Ge, Xin; Sun, Jinping; Zhang, Yanning

doi:10.1109/cvpr42600.2020.00372

Cited by 336 publications

(204 citation statements)

References 36 publications

Supporting

Mentioning

203

Contrasting

Order By: Relevance

“…For example, Kang et al [7][8] proposed a multi-task shallow CNN to learn both the distortion type and the quality score; Kim and Lee [9] applied state-of-the-art FR-IQA methods to provide proxy quality scores for each image patch as the ground truth label in the pre-training stage, and the proposed network was fine-tuned by the Subjective annotations. Similarly, Da Pan et al [10] employed the U-Net to learn the local quality predicting scores previously calculated by Full-Reference IQA methods, several Dense layers were then incorporated to pool the local quality predicting scores into an overall perceptual quality score; Liang et al [11] tried to utilize similar scene as reference to provide more prior information for the IQA model; Liu et al [12] proposed to use RankNet to learn the quality rank information of image pairs in the training set, and then used the output of the second last layer to predict the quality score; Yee et al [13] tried to learn the corresponding unknown reference image from the distorted one by resorting the Generative Adversarial Networks, and to assess the perceptual quality by comparing the hallucinated reference image and the distorted image; Chiu et al [1] proposed a new IQA framework and corresponding dataset that links the IQA issue to two practical vision tasks which are image captioning and visual question answering respectively; Su et al [14] employed self-adaptive hyper network whose parameters could adjust according to image contents; Zhu et al [15] leveraged meta-learning to learn a general-purpose BIQA model from training set of several specific distortion types.…”

Section: Related Workmentioning

confidence: 99%

“…In order to accelerate the time-consuming calculating procedure of IQA-curve for each channel, a self-adaptive hyper network is designed inspired by [14], which could precisely predict the IQA-curve in one-shot, and get rid of M × N times of JPEG compression and IQA prediction. The overall procedure of our proposed group quality optimized framework for multi-channel JPEG image encoding system is shown as Fig.…”

Section: A Problem Formulationmentioning

confidence: 99%

“…Inspired by [14], a self-adaptive hyper network is designed to accurately predict the IQA-curve, and its pipeline is illustrated as Fig. 3.…”

Section: B Hyper Network For Predicting Iqa-curvementioning

confidence: 99%

“…Rather than directly fed the multi-scale features into traditional regression networks whose parameters are fixed for all images, our work employs a content understanding hyper network to generate customized regression parameters for each test image. The reasons we employ the content understanding hyper network are summarized as follow: Firstly, such model has been applied for blind predict IQA score in [14] and achieved satisfactory performance, the predicting of IQA score and IQA-curve are related; Secondly, different image contents yield discriminative patterns of IQA-curves, e.g., the firstorder derivative of IQA-curve obtained from different source images exhibit quite unsimilar patterns; Thirdly, based on our experience, using fixed parameters for all test images to predict a multi-dimension curve often leads to overfitting problem, which could be avoided by generating customized parameters from the hyper networks.…”

Section: B Hyper Network For Predicting Iqa-curvementioning

confidence: 99%

“…As described in Fig. 3, the overall pipeline of our IQAcurve prediction model is similar with [14], but it is comprised of two parallel dataflows, the upper one is employed for predicting the transmitting resource distribution whilst the lower one is employed to for predicting the perceptual quality distribution. The overall structures of the two data streams are similar, so the illustration is focused on the lower one that employed for predicting the perceptual quality distribution.…”

Section: B Hyper Network For Predicting Iqa-curvementioning

confidence: 99%

See 4 more Smart Citations

Group Perceptual Quality Optimization for Multi-Channel Image Encoding Systems Based on Adaptive Hyper Networks

Chen

Chen³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Images and short videos that produced by social networks surge in recent years. Image/Video encoders, such as JPEG and H.264, are indispensably involved to reduce the transmitting bandwidth. However, based on our observation, the encoding parameters and their candidates are often preset to fixed values (or fixed candidate values) in real-world scenarios, which might not be the optimal bandwidth allocation strategy. Considering that, we propose an efficient group quality optimization (GQO) framework for multichannel image/video encoding systems in which the encoding parameters are configured in a perceptualquality-driven manner. The GQO framework employs adaptive hyper network to predict the relationships between encoding parameters, transmitting resources, and perceptual qualities, i.e., just taking the pristine image as input, the adaptive hyper network could accurately yield a global overview of perceptual quality and transmitting resource varied along encoding parameters. A step-by-step optimization procedure is then employed to search the optimal encoding parameter for each channel so that overall perceptual quality could be maximized under limited transmitting resource. Experimental results demonstrate the proposed GQO framework could achieve higher perceptual quality whilst maintain the same bandwidth compared to traditional allocation strategies where encoding parameters are preset.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: A Problem Formulationmentioning

confidence: 99%

“…Inspired by [14], a self-adaptive hyper network is designed to accurately predict the IQA-curve, and its pipeline is illustrated as Fig. 3.…”

Section: B Hyper Network For Predicting Iqa-curvementioning

confidence: 99%

Section: B Hyper Network For Predicting Iqa-curvementioning

confidence: 99%

Section: B Hyper Network For Predicting Iqa-curvementioning

confidence: 99%

See 3 more Smart Citations

Group Perceptual Quality Optimization for Multi-Channel Image Encoding Systems Based on Adaptive Hyper Networks

Chen

Chen³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

Blind image quality assessment based on the multiscale and dual‐domains features fusion

Ning

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

Image quality assessment is to simulate subjective human visual perception and realize image quality inference automatically. Although deep neural networks have achieved great success, the majority of them do not fully consider perception characteristics. Therefore, according to the human visual scale characteristics, we proposed an image quality assessment algorithm based on multiscale and dual domains fusion. Firstly, the original image and its phase congruency respectively input into two branches, feature pyramid and channel attention mechanism are adopted to extract multiscale features. After that, bilinear pool is used to aggregate the spatial and frequency domain characteristics of the corresponding scales, and allows arbitrary scale input to ensure that the features are extracted from the inherent quality images. Finally, the single quality score is obtained through learned weights of each scale. Comparative experiments between our approach and state-of-the-art are conducted on five public databases, the results demonstrate that the proposed algorithm is not only robust to different types and across database, but also sensitive to scale. K E Y W O R D Simage quality assessment, multiscale features, original scale input, phase congruency INTRODUCTIONPeople have higher and higher requirements for image quality, so it is an important research topic to automatically and accurately assess image quality before display 1 to ensure human visual perception. Image quality assessment (IQA) can be used to optimize the process of image acquisition, transmission, and storage, thereby minimizing possible image quality degradation. It can also be used to guide image restoration, image deblurring, image super-resolution reconstruction, and other tasks to improve image quality. 2According to the needs of reference information, IQA methods can be divided into full reference (FR), reduced reference (RR), and no reference (NR). FR-IQA calculates the difference between the reference image and the distorted image to obtain the quality score of the distorted image. The classic algorithms are structural similarity (SSIM) 3 and feature similarity (FSIM). 4 The Human Visual System (HVS) is suitable for the extraction of structural information, and image distortion can also change the perception of structural information. Therefore, SSIM calculates the illuminance, contrast and structural similarity between the reference image and the test image to obtain quality index. However, when pooling into a single image quality score, the weights of different local regions are not considered. FSIM proposed that HVS is mainly based on low-level features for image understanding. Therefore, the low-level feature similarity is proposed to obtain the quality index. Specifically, the phase consistency (PC) and the gradient magnitude (GM) are complementary, and the PC is also used as a weight to obtain the similarity score.

show abstract

CapsNet meets ORB: A deformation‐tolerant baseline for recognizing distorted targets

Lin

Gao

Jia

et al. 2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

show abstract

Blindly Assess Image Quality in the Wild Guided by a Self-Adaptive Hyper Network

Cited by 336 publications

References 36 publications

Group Perceptual Quality Optimization for Multi-Channel Image Encoding Systems Based on Adaptive Hyper Networks

Group Perceptual Quality Optimization for Multi-Channel Image Encoding Systems Based on Adaptive Hyper Networks

Blind image quality assessment based on the multiscale and dual‐domains features fusion

CapsNet meets ORB: A deformation‐tolerant baseline for recognizing distorted targets

Contact Info

Product

Resources

About