Perceptual Image Quality Assessment with Transformers

Cheon, Manri; Yoon, Sungjun; Kang, Byungyeon; Lee, Junwoo

doi:10.1109/cvprw53098.2021.00054

Cited by 100 publications

(49 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To verify the effectiveness of our method, we utilize LPIPS [30], FID [17], KID [5], PieAPP [27], DISTS [11] and IQT [10] to guide the evalution of reconstructions. The combination of these scores is consistent with MOS to some degree.…”

Section: Quantitative Resultsmentioning

confidence: 99%

PO-ELIC: Perception-Oriented Efficient Learned Image Coding

He¹,

Yang²,

Hou³

et al. 2022

Preprint

View full text Add to dashboard Cite

In the past years, learned image compression (LIC) has achieved remarkable performance. The recent LIC methods outperform VVC in both PSNR and MS-SSIM. However, the low bit-rate reconstructions of LIC suffer from artifacts such as blurring, color drifting and texture missing. Moreover, those varied artifacts make image quality metrics correlate badly with human perceptual quality. In this paper, we propose PO-ELIC, i.e., Perception-Oriented Efficient Learned Image Coding. To be specific, we adapt ELIC, one of the state-of-the-art LIC models, with adversarial training techniques. We apply a mixture of losses including hinge-form adversarial loss, Charbonnier loss, and style loss, to finetune the model towards better perceptual quality. Experimental results demonstrate that our method achieves comparable perceptual quality with HiFiC with much lower bitrate.

show abstract

Section: Quantitative Resultsmentioning

confidence: 99%

PO-ELIC: Perception-Oriented Efficient Learned Image Coding

He¹,

Yang²,

Hou³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Cheon et al [3] adapt Vision transformers [7] for perceptual IQA. They have used a pretrained Inception-Resnet-v2 network [25] as feature extraction backbone and transformer encoder-decoder architecture to obtain quality score predidction.…”

Section: Related Workmentioning

confidence: 99%

Can No-reference features help in Full-reference image quality estimation?

Dutta¹,

Das²,

Shah³

2022

Preprint

View full text Add to dashboard Cite

Development of perceptual image quality assessment (IQA) metrics has been of significant interest to computer vision community. The aim of these metrics is to model quality of an image as perceived by humans. Recent works in Full-reference IQA research perform pixelwise comparison between deep features corresponding to query and reference images for quality prediction. However, pixelwise feature comparison may not be meaningful if distortion present in query image is severe. In this context, we explore utilization of no-reference features in Full-reference IQA task. Our model consists of both full-reference and noreference branches. Full-reference branches use both distorted and reference images, whereas No-reference branch only uses distorted image. Our experiments show that use of no-reference features boosts performance of image quality assessment. Our model achieves higher SRCC and KRCC scores than a number of state-of-the-art algorithms on KADID-10K and PIPAL datasets.

show abstract

“…Recently, learning-based FR-IQA methods (Prashnani et al 2018;Ding et al 2021) have achieved significant improvement. The most current IQT model (Cheon et al 2021) involves the visual transformer with extra quality and position embeddings to achieve the best performance for the FR-IQA task. Different from FR-IQA, the RR-IQA method (Rehman and Wang 2012) utilizes only parts of the FR image information.…”

Section: Related Workmentioning

confidence: 99%

“…1(a)) only use LQ images as input to directly measure image quality. FR/RR-IQA methods (Rehman and Wang 2012;Cheon et al 2021) (Fig. 1(b)) utilize the complete or partial information of the pixel-aligned HQ reference images.…”

Section: Introductionmentioning

confidence: 99%

Content-Variant Reference Image Quality Assessment via Knowledge Distillation

Yin

Wang

Yuan

et al. 2022

AAAI

View full text Add to dashboard Cite

Generally, humans are more skilled at perceiving differences between high-quality (HQ) and low-quality (LQ) images than directly judging the quality of a single LQ image. This situation also applies to image quality assessment (IQA). Although recent no-reference (NR-IQA) methods have made great progress to predict image quality free from the reference image, they still have the potential to achieve better performance since HQ image information is not fully exploited. In contrast, full-reference (FR-IQA) methods tend to provide more reliable quality evaluation, but its practicability is affected by the requirement for pixel-level aligned reference images. To address this, we firstly propose the content-variant reference method via knowledge distillation (CVRKD-IQA). Specifically, we use non-aligned reference (NAR) images to introduce various prior distributions of high-quality images. The comparisons of distribution differences between HQ and LQ images can help our model better assess the image quality. Further, the knowledge distillation transfers more HQ-LQ distribution difference information from the FR-teacher to the NAR-student and stabilizing CVRKD-IQA performance. Moreover, to fully mine the local-global combined information, while achieving faster inference speed, our model directly processes multiple image patches from the input with the MLP-mixer. Cross-dataset experiments verify that our model can outperform all NAR/NR-IQA SOTAs, even reach comparable performance than FR-IQA methods on some occasions. Since the content-variant and non-aligned reference HQ images are easy to obtain, our model can support more IQA applications with its robustness to content variations. Our code is available: https://github.com/guanghaoyin/CVRKD-IQA.

show abstract

Perceptual Image Quality Assessment with Transformers

Cited by 100 publications

References 37 publications

PO-ELIC: Perception-Oriented Efficient Learned Image Coding

PO-ELIC: Perception-Oriented Efficient Learned Image Coding

Can No-reference features help in Full-reference image quality estimation?

Content-Variant Reference Image Quality Assessment via Knowledge Distillation

Contact Info

Product

Resources

About