The 2018 PIRM Challenge on Perceptual Image Super-Resolution

Blau, Yochai; Mechrez, Roey; Timofte, Radu; Michaeli, Tomer; Zelnik-Manor, Lihi

doi:10.1007/978-3-030-11021-5_21

Cited by 408 publications

(346 citation statements)

References 54 publications

Supporting

Mentioning

318

Contrasting

Order By: Relevance

“…Interestingly, it is comparable to the externally trained SR-GAN method [30], despite having been exposed to only a single image. Following [4], we compare these 5 methods in Table 3 on the BSD100 dataset [35] in terms of distortion (RMSE) and perceptual quality (NIQE [40]), which are two fundamentally conflicting requirements [5]. As can be seen, SinGAN excels in perceptual quality; its NIQE score is only slightly inferior to SRGAN, and its RMSE is slightly better.…”

Section: Applicationsmentioning

confidence: 99%

SinGAN: Learning a Generative Model From a Single Natural Image

Shaham

Dekel

Michaeli

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

756

689

View full text Add to dashboard Cite

Random samples from a single image Single training image Figure 1: Image generation learned from a single training Image. We propose SinGAN-a new unconditional generative model trained on a single natural image. Our model learns the image's patch statistics across multiple scales, using a dedicated multi-scale adversarial training scheme; it can then be used to generate new realistic image samples that preserve the original patch distribution while creating new object configurations and structures. AbstractWe introduce SinGAN, an unconditional generative model that can be learned from a single natural image. Our model is trained to capture the internal distribution of patches within the image, and is then able to generate high quality, diverse samples that carry the same visual content as the image. SinGAN contains a pyramid of fully convolutional GANs, each responsible for learning the patch distribution at a different scale of the image. This allows generating new samples of arbitrary size and aspect ratio, that have significant variability, yet maintain both the global structure and the fine textures of the training image. In contrast to previous single image GAN schemes, our approach is not limited to texture images, and is not conditional (i.e. it generates samples from noise). User studies confirm that the generated samples are commonly confused to be real images. We illustrate the utility of SinGAN in a wide range of image manipulation tasks.

show abstract

Section: Applicationsmentioning

confidence: 99%

SinGAN: Learning a Generative Model From a Single Natural Image

Shaham

Dekel

Michaeli

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

756

689

View full text Add to dashboard Cite

show abstract

“…Although the recently proposed CNN-based SR solutions [37,40] provide state-of-the-art quantitative results in terms of peak signal-to-noise ratio (PSNR) when they optimize for reconstruction losses such as L1 or L2 in image space, the results are smooth without the fine details required for a good perceptual quality. This problem is more visible with the increase of the upscaling factor [22,5]. On top of that, the PSNR measure is unable to capture perceptually important differences between two images as it relies on the differences between pixel-level values at the same position [41,42,15].…”

Section: Introductionmentioning

confidence: 99%

“…On top of that, the PSNR measure is unable to capture perceptually important differences between two images as it relies on the differences between pixel-level values at the same position [41,42,15]. One way to introduce perceptually important features into the SR image is to use generative adversarial networks (GANs) [13,26,5]. These networks help to create realistic SR images that look like HR images, which are naturally sharper and contain fine details.…”

Section: Introductionmentioning

confidence: 99%

Exemplar Guided Face Image Super-Resolution Without Facial Landmarks

Dogan

Timofte

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Self Cite

View full text Add to dashboard Cite

Nowadays, due to the ubiquitous visual media there are vast amounts of already available high-resolution (HR) face images. Therefore, for super-resolving a given very lowresolution (LR) face image of a person it is very likely to find another HR face image of the same person which can be used to guide the process. In this paper, we propose a convolutional neural network (CNN)-based solution, namely GWAInet, which applies super-resolution (SR) by a factor 8× on face images guided by another unconstrained HR face image of the same person with possible differences in age, expression, pose or size. GWAInet is trained in an adversarial generative manner to produce the desired high quality perceptual image results. The utilization of the HR guiding image is realized via the use of a warper subnetwork that aligns its contents to the input image and the use of a feature fusion chain for the extracted features from the warped guiding image and the input image. In training, the identity loss further helps in preserving the identity related features by minimizing the distance between the embedding vectors of SR and HR ground truth images. Contrary to the current state-of-the-art in face super-resolution, our method does not require facial landmark points for its training, which helps its robustness and allows it to produce fine details also for the surrounding face region in a uniform manner. Our method GWAInet produces photo-realistic images in upscaling factor 8× and outperforms state-of-theart in quantitative terms and perceptual quality.

show abstract

“…Blurry edges and over-smooth textures were shown in SR results while having a high PSNR value. Both Perceptual Index (PI) 25 and Natural Image Quality Evaluator (NIQE) 28 are brought up to evaluate SR results in terms of perceptual quality.In order to improve SR images visual quality, researchers have introduced different loss functions to optimize SR networks. In 2017, Ledge et al 17 presented a generative adversarial network (GAN) 8 composed of a generator and an image discriminator for SR.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

Super-resolved image perceptual quality improvement via multifeature discriminators

Zhu¹,

Cheng²,

Peng³

et al. 2020

J. Electron. Imag.

View full text Add to dashboard Cite

Generative adversarial network (GAN) for image super-resolution (SR) has attracted enormous interests in recent years. However, the GAN-based SR methods only use image discriminator to distinguish SR images and high-resolution (HR) images. Image discriminator fails to discriminate images accurately since image features cannot be fully expressed. In this paper, we design a new GAN-based SR framework GAN-IMC which includes generator, image discriminator, morphological component discriminator and color discriminator. The combination of multiple feature discriminators improves the accuracy of image discrimination. Adversarial training between the generator and multi-feature discriminators forces SR images to converge with HR images in terms of data and features distribution. Moreover, in some cases, feature enhancement of salient regions is also worth considering. GAN-IMC is further optimized by weighted content loss (GAN-IMCW), which effectively restores and enhances salient regions in SR images. The effectiveness and robustness of our method are confirmed by extensive experiments on public datasets. Compared with state-of-the-art methods, the proposed method not only achieves competitive Perceptual Index (PI) and Natural Image Quality Evaluator (NIQE) values but also obtains pleasant visual perception in image edge, texture, color and salient regions. ).2 reconstruction quality compared with state-of-the-art methods at that time due to the close correlation between pixel loss function and Power Signal-to-Noise Ratio (PSNR). This research is a milestone for SR. Subsequently, considerable researches 10,13,14,15,16 that minimized pixel loss function to train CNNs had been conducted and PSNR values had been dramatically improved. However, the studies 17-19 pointed out that SR results with good visual quality reflected by PSNR values were inconsistent with or even contrary to the subjective evaluation of human observers. Blurry edges and over-smooth textures were shown in SR results while having a high PSNR value. Both Perceptual Index (PI) 25 and Natural Image Quality Evaluator (NIQE) 28 are brought up to evaluate SR results in terms of perceptual quality.In order to improve SR images visual quality, researchers have introduced different loss functions to optimize SR networks. In 2017, Ledge et al. 17 presented a generative adversarial network (GAN) 8 composed of a generator and an image discriminator for SR. The generator is used to generate SR results. The discriminator is used to determine SR images and HR images. Adversarial learning between generator and discriminator encourages some or several types of data of SR images to be similar to that of HR images. Park et al. 22 proposed a feature discriminator that distinguishes SR image from HR image by feature maps to produce high-frequency details.Considering that morphological component and color are highly correlated with image visual quality, a natural idea is to introduce morphological component discriminator and color discriminator to identify images. In this pa...

show abstract

The 2018 PIRM Challenge on Perceptual Image Super-Resolution

Cited by 408 publications

References 54 publications

SinGAN: Learning a Generative Model From a Single Natural Image

SinGAN: Learning a Generative Model From a Single Natural Image

Exemplar Guided Face Image Super-Resolution Without Facial Landmarks

Super-resolved image perceptual quality improvement via multifeature discriminators

Contact Info

Product

Resources

About