EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis

Abstract

Single image super-resolution is the task of inferring a high-resolution image from a single low-resolution input. Traditionally, the performance of algorithms for this task is measured using pixel-wise reconstruction measures such as peak signal-to-noise ratio (PSNR) which have been shown to correlate poorly with the human perception of image quality. As a result, algorithms minimizing these metrics tend to produce over-smoothed images that lack highfrequency textures and do not look natural despite yielding high PSNR values.We propose a novel application of automated texture synthesis in combination with a perceptual loss focusing on creating realistic textures rather than optimizing for a pixelaccurate reproduction of ground truth images during training. By using feed-forward fully convolutional neural networks in an adversarial training setting, we achieve a significant boost in image quality at high magnification ratios. Extensive experiments on a number of datasets show the effectiveness of our approach, yielding state-of-the-art results in both quantitative and qualitative benchmarks.

Improving resolution of MR images with an adversarial network incorporating images with different contrast

“…Since the choice of loss function guides learning process, defining an appropriate loss function is important in constructing a CNN framework. Pixel-wise loss functions such as L1 loss 14,36,39 or L2 loss 16,19,20 have been widely used for image super-resolution and denoising, but blurred image quality is recently considered as one major issue in image generation using CNNs. 15,17,19 As one potential way to overcome the problem, adversarial loss has been proposed.…”

“…Another recent technique for better perceptual image quality is inspired by a generative adversarial network (GAN), composed of a generator CNN and a discriminator CNN. 16,19,24,25 While the generator was trained to generate HR images close to the ground truths, the discriminator was simultaneously trained to distinguish the generated HR images from the ground truths. Since the discriminator error is back-propagated to the generator, the errors of the discriminator and the generator are adversarial, yielding an adversarial loss.…”

“…Usage of the adversarial loss improves perceptual image quality. 16,19 Recently, the adversarial loss has been applied to medical imaging research for noise reduction in low-dose CT 26 and de-aliasing of CS-MRI. 27 In addition, the incorporation of anatomical information from differently weighted MR images can improve MRI reconstruction, because high frequency contents are not significantly dependent on the choice of MRI pulse sequences.…”

“…Since the discriminator error is back‐propagated to the generator, the errors of the discriminator and the generator are adversarial, yielding an adversarial loss. Usage of the adversarial loss improves perceptual image quality . Recently, the adversarial loss has been applied to medical imaging research for noise reduction in low‐dose CT and de‐aliasing of CS‐MRI…”

“…Second, the L1/L2 loss functions typically used in CNNs for optimization have recently been proved to not provide perceptually better image qualities. [14][15][16] New strategies to overcome the issue have been proposed, [15][16][17][18][19] but are not investigated well in MR image reconstruction. Third, sparse sampling is not a prerequisite for CNN, because the image reconstruction is conducted in the feature space.…”