Super-resolved image perceptual quality improvement via multifeature discriminators

Abstract

Generative adversarial network (GAN) for image super-resolution (SR) has attracted enormous interests in recent years. However, the GAN-based SR methods only use image discriminator to distinguish SR images and high-resolution (HR) images. Image discriminator fails to discriminate images accurately since image features cannot be fully expressed. In this paper, we design a new GAN-based SR framework GAN-IMC which includes generator, image discriminator, morphological component discriminator and color discriminator. The combination of multiple feature discriminators improves the accuracy of image discrimination. Adversarial training between the generator and multi-feature discriminators forces SR images to converge with HR images in terms of data and features distribution. Moreover, in some cases, feature enhancement of salient regions is also worth considering. GAN-IMC is further optimized by weighted content loss (GAN-IMCW), which effectively restores and enhances salient regions in SR images. The effectiveness and robustness of our method are confirmed by extensive experiments on public datasets. Compared with state-of-the-art methods, the proposed method not only achieves competitive Perceptual Index (PI) and Natural Image Quality Evaluator (NIQE) values but also obtains pleasant visual perception in image edge, texture, color and salient regions. ).2 reconstruction quality compared with state-of-the-art methods at that time due to the close correlation between pixel loss function and Power Signal-to-Noise Ratio (PSNR). This research is a milestone for SR. Subsequently, considerable researches 10,13,14,15,16 that minimized pixel loss function to train CNNs had been conducted and PSNR values had been dramatically improved. However, the studies 17-19 pointed out that SR results with good visual quality reflected by PSNR values were inconsistent with or even contrary to the subjective evaluation of human observers. Blurry edges and over-smooth textures were shown in SR results while having a high PSNR value. Both Perceptual Index (PI) 25 and Natural Image Quality Evaluator (NIQE) 28 are brought up to evaluate SR results in terms of perceptual quality.In order to improve SR images visual quality, researchers have introduced different loss functions to optimize SR networks. In 2017, Ledge et al. 17 presented a generative adversarial network (GAN) 8 composed of a generator and an image discriminator for SR. The generator is used to generate SR results. The discriminator is used to determine SR images and HR images. Adversarial learning between generator and discriminator encourages some or several types of data of SR images to be similar to that of HR images. Park et al. 22 proposed a feature discriminator that distinguishes SR image from HR image by feature maps to produce high-frequency details.Considering that morphological component and color are highly correlated with image visual quality, a natural idea is to introduce morphological component discriminator and color discriminator to identify images. In this pa...

Generative Adversarial Network for Pansharpening With Spectral and Spatial Discriminators

“…This multi-discriminator framework is mainly used for the super-resolution problem. For example, Zhu et al [20] consider three discriminators. The first one is a pixel discriminator, commonly used in GAN, the second one compares colors with the low-resolution images and the third one compares edges and textures by considering the grayscale images.…”

Usefulness of a No-Reference Metric for Evaluation of Images in Nuclear Medicine -A Comparative Study with Visual Assessment

“…In recent years, diagnostic imaging support has adapted to various methods using arti cial intelligence, among which the generative adversarial network (GAN), an unsupervised learning method, has been drawing attention [6][7][8]. In a study using GAN, Xuan et al pointed out that it is di cult to evaluate images using peak signal to noise ratio (PSNR) based on the differences from the reference images [9]. They recommend using a no-reference metric, which utilizes the perceptual index as a metric for image evaluation [9].…”

“…In a study using GAN, Xuan et al pointed out that it is di cult to evaluate images using peak signal to noise ratio (PSNR) based on the differences from the reference images [9]. They recommend using a no-reference metric, which utilizes the perceptual index as a metric for image evaluation [9].…”