Generative Adversarial Network-Based Image Super-Resolution Using Perceptual Content Losses

Cheon, Manri; Kim, Jun Hyuk; Choi, Jun Ho; Lee, Jong Seok

doi:10.1007/978-3-030-11021-5_4

Cited by 33 publications

(18 citation statements)

References 30 publications

(57 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ledge et al 17 applied adversarial loss to optimize SR network. Cheon et al 26 proposed perceptual image content loss that measures the difference between images after applying Discrete Cosine Transform (DCT) and differential operation on SR images and HR images. Sajjadi et al 19 used texture loss 21 to ensure the consistent style between images.…”

Section: Super-resolutionmentioning

confidence: 99%

Super-resolved image perceptual quality improvement via multifeature discriminators

Zhu¹,

Cheng²,

Peng³

et al. 2020

J. Electron. Imag.

View full text Add to dashboard Cite

Generative adversarial network (GAN) for image super-resolution (SR) has attracted enormous interests in recent years. However, the GAN-based SR methods only use image discriminator to distinguish SR images and high-resolution (HR) images. Image discriminator fails to discriminate images accurately since image features cannot be fully expressed. In this paper, we design a new GAN-based SR framework GAN-IMC which includes generator, image discriminator, morphological component discriminator and color discriminator. The combination of multiple feature discriminators improves the accuracy of image discrimination. Adversarial training between the generator and multi-feature discriminators forces SR images to converge with HR images in terms of data and features distribution. Moreover, in some cases, feature enhancement of salient regions is also worth considering. GAN-IMC is further optimized by weighted content loss (GAN-IMCW), which effectively restores and enhances salient regions in SR images. The effectiveness and robustness of our method are confirmed by extensive experiments on public datasets. Compared with state-of-the-art methods, the proposed method not only achieves competitive Perceptual Index (PI) and Natural Image Quality Evaluator (NIQE) values but also obtains pleasant visual perception in image edge, texture, color and salient regions. ).2 reconstruction quality compared with state-of-the-art methods at that time due to the close correlation between pixel loss function and Power Signal-to-Noise Ratio (PSNR). This research is a milestone for SR. Subsequently, considerable researches 10,13,14,15,16 that minimized pixel loss function to train CNNs had been conducted and PSNR values had been dramatically improved. However, the studies 17-19 pointed out that SR results with good visual quality reflected by PSNR values were inconsistent with or even contrary to the subjective evaluation of human observers. Blurry edges and over-smooth textures were shown in SR results while having a high PSNR value. Both Perceptual Index (PI) 25 and Natural Image Quality Evaluator (NIQE) 28 are brought up to evaluate SR results in terms of perceptual quality.In order to improve SR images visual quality, researchers have introduced different loss functions to optimize SR networks. In 2017, Ledge et al. 17 presented a generative adversarial network (GAN) 8 composed of a generator and an image discriminator for SR. The generator is used to generate SR results. The discriminator is used to determine SR images and HR images. Adversarial learning between generator and discriminator encourages some or several types of data of SR images to be similar to that of HR images. Park et al. 22 proposed a feature discriminator that distinguishes SR image from HR image by feature maps to produce high-frequency details.Considering that morphological component and color are highly correlated with image visual quality, a natural idea is to introduce morphological component discriminator and color discriminator to identify images. In this pa...

show abstract

Section: Super-resolutionmentioning

confidence: 99%

Super-resolved image perceptual quality improvement via multifeature discriminators

Zhu¹,

Cheng²,

Peng³

et al. 2020

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Although [19] shows a more natural HF than [18], it still has a lower PSNR. Reference [20] also proposes a novel generator loss function with the EUSR network [24]. However, instead of MSE loss and VGG loss, it uses content loss and differential content loss, which both use L1-norm.…”

Section: Related Workmentioning

confidence: 99%

“…Including the VGG-based discriminator described in [21], we evaluate the various combinations of discriminators in order to confirm the effect of each discriminator as listed in Table 1. The DCT loss for the generator in [20] is also compared with the same network.…”

Section: A Experimentsmentioning

confidence: 99%

Multi-Perspective Discriminators-Based Generative Adversarial Network for Image Super Resolution

2019

View full text Add to dashboard Cite

Recently, generative adversarial network-based image super resolution has been investigated, and it has been shown to lead to overwhelming improvements in subjective quality. However, it also leads to checkerboard artifacts and the unpleasing high-frequency (HF) components. In this paper, we propose a multi-discriminators-based image super resolution method that distinguishes those artifacts from various perspectives. First, the DCT perspective discriminator is proposed because the checkerboard artifacts are easily separated on the frequency domain. Second, the gradient perspective discriminator is proposed, because the unpleasing HF components can be discriminated on the gradient magnitude distribution. These proposed multi-perspective discriminators can easily identify artifacts, and they can help the generator reproduce artifact-less SR images. The experimental results show that the proposed SR-GAN with multiperspective discriminators achieves objective and subjective quality improvements in terms of PSNR, SSIM, PI and MOS, as compared to the conventional SR-GAN by reducing the aforementioned artifacts. INDEX TERMS Image super-resolution, deep learning for super resolution, SR GAN, multi-discriminators.

show abstract

“…Although its PSNR and SSIM index are slightly lower than those of the other SR techniques that employ only MSE, SRGAN employs a sophisticated discriminator network to generate realistic images that can fool humans. This idea has succeeded to more complicated structures with additional loss functions [31], [32]. Voynov et al [33] adopted perceptual loss from depth images to recover texture information.…”

Section: B Image Super-resolutionmentioning

confidence: 99%

Object Recognition in Very Low Resolution Images Using Deep Collaborative Learning

Seo

Park

2019

IEEE Access

View full text Add to dashboard Cite

Although recent studies on object recognition using deep neural networks have reported remarkable performance, they have usually assumed that adequate object size and image resolution are available, which may not be guaranteed in real applications. This paper proposes a framework for recognizing objects in very low resolution images through the collaborative learning of two deep neural networks: image enhancement network and object recognition network. The proposed image enhancement network attempts to enhance extremely low resolution images into sharper and more informative images with the use of collaborative learning signals from the object recognition network. The object recognition network with trained weights for high resolution images actively participates in the learning of the image enhancement network. It also utilizes the output from the image enhancement network as augmented learning data to boost its recognition performance on very low resolution objects. Through experiments on various low resolution image benchmark datasets, we verified that the proposed method can improve the image reconstruction and classification performance. INDEX TERMS Machine learning, object recognition, very low resolution recognition, image enhancement, deep neural networks, collaborative learning. HYEYOUNG PARK received the B.S. degree (summa cum laude) and the M.S. and Ph.D.

show abstract

Generative Adversarial Network-Based Image Super-Resolution Using Perceptual Content Losses

Cited by 33 publications

References 30 publications

Super-resolved image perceptual quality improvement via multifeature discriminators

Super-resolved image perceptual quality improvement via multifeature discriminators

Multi-Perspective Discriminators-Based Generative Adversarial Network for Image Super Resolution

Object Recognition in Very Low Resolution Images Using Deep Collaborative Learning

Contact Info

Product

Resources

About