Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Ledig, Christian; Theis, Lucas; Huszár, Ferenc; Caballero, José; Cunningham, Andrew; Acosta, Alejandro; Aitken, Andrew P.; Tejani, Alykhan; Totz, Johannes; Wang, Zehan; Shi, Wenzhe

doi:10.48550/arxiv.1609.04802

Cited by 344 publications

(439 citation statements)

References 47 publications

Supporting

Mentioning

396

Contrasting

Unclassified

Order By: Relevance

“…We used a multi-resolution PatchGAN discriminator, as was done in the original GauGAN implementation (Isola et al, 2017). According to this paper, a perceptual VGG loss (Ledig et al, 2016) and a discriminator featurematching loss (Salimans et al, 2016) are essential for good performance. Since these last two losses rely on paired examples, we trained the Painter separately from the Masker using images of floods.…”

Section: Paintermentioning

confidence: 99%

ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods

Schmidt¹,

Luccioni²,

Teng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Climate change is a major threat to humanity, and the actions required to prevent its catastrophic consequences include changes in both policy-making and individual behaviour. However, taking action requires understanding the effects of climate change, even though they may seem abstract and distant. Projecting the potential consequences of extreme climate events such as flooding in familiar places can help make the abstract impacts of climate change more concrete and encourage action. As part of a larger initiative to build a website that projects extreme climate events onto user-chosen photos, we present our solution to simulate photo-realistic floods on authentic images. To address this complex task in the absence of suitable training data, we propose ClimateGAN, a model that leverages both simulated and real data for unsupervised domain adaptation and conditional image generation. In this paper, we describe the details of our framework, thoroughly evaluate components of our architecture and demonstrate that our model is capable of robustly generating photo-realistic flooding.

show abstract

Section: Paintermentioning

confidence: 99%

ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods

Schmidt¹,

Luccioni²,

Teng³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Generative Adversarial Network. Generative adversarial networks (GANs) [23] have shown impressive results in image generation [22], [24], image translation [25], [26], [27], superresolution imaging [2], [28], and face image synthesis [29], [30]. These GAN models are trained to minimize the discrepancy between distributions of the training data and unobserved generations by adversarially learning between two neural networks.…”

Section: Related Workmentioning

confidence: 99%

A Domain Gap Aware Generative Adversarial Network for Multi-domain Image Translation

Xu,

Wang

2021

Preprint

View full text Add to dashboard Cite

Recent image-to-image translation models have shown great success in mapping local textures between two domains. Existing approaches rely on a cycle-consistency constraint that supervises the generators to learn an inverse mapping. However, learning the inverse mapping introduces extra trainable parameters and it is unable to learn the inverse mapping for some domains. As a result, they are ineffective in the scenarios where (i) multiple visual image domains are involved; (ii) both structure and texture transformations are required; and (iii) semantic consistency is preserved. To solve these challenges, the paper proposes a unified model to translate images across multiple domains with significant domain gaps. Unlike previous models that constrain the generators with the ubiquitous cycle-consistency constraint to achieve the content similarity, the proposed model employs a perceptual self-regularization constraint. With a single unified generator, the model can maintain consistency over the global shapes as well as the local texture information across multiple domains. Extensive qualitative and quantitative evaluations demonstrate the effectiveness and superior performance over state-of-the-art models. It is more effective in representing shape deformation in challenging mappings with significant dataset variation across multiple domains.

show abstract

“…Ledig et al [11] proposed a deep residual network which is able to recover photo-realistic textures from heavily downsampled images on public benchmarks. An extensive Mean-Opinion-Score (MOS) test shows significant gains in perceptual quality using SR based on Generative Adversarial Network (SRGAN).…”

Section: Super-resolution (Sr)mentioning

confidence: 99%

“…Detection Error Trade-off (DET) curves are included to show the performance and the efficiency of our proposal. All these new experiments are benchmarked with those previously obtained in [9,10,11].…”

Section: Introductionmentioning

confidence: 99%

Selfie Periocular Verification using an Efficient Super-Resolution Approach

Tapia¹,

Valenzuela²,

Lara³

et al. 2021

Preprint

View full text Add to dashboard Cite

Selfie-based biometrics has great potential for a wide range of applications from marketing to higher security environments like online banking. This is now especially relevant since e.g. periocular verification is contactless, and thereby safe to use in pandemics such as COVID-19. However, selfie-based biometrics faces some challenges since there is limited control over the data acquisition conditions. Therefore, super-resolution have to be used to increase the quality of the captured images. Most of the state of the art super-resolution methods use deep networks with large filters, thereby needing to train and store a correspondingly large number of parameters, and making their use difficult for mobile devices commonly used for selfie-based.In order to achieve an efficient super-resolution method, we propose an Efficient Single Image Super-Resolution (ESISR) algorithm, which takes into account a trade-off between the efficiency of the deep neural network and the size of its filters. To that end, the method implements a novel loss function based on the Sharpness metric. This metric turns out to be more suitable for increasing the quality of the eye images. Our method drastically reduces the number of parameters when compared with Deep CNNs with Skip Connection and Network (DCSCN): from 2,170,142 to 28,654 parameters when the image size is increased by a factor of x3. Furthermore, the proposed method keeps the sharp quality of the images, which is highly relevant for biometric recognition purposes. The results on remote verification systems with raw images reached an Equal Error Rate (EER) 8.7% for FaceNet and 10.05% for VGGFace. Where embedding vectors were used from periocular images the best results reached an EER of 8.9% (x3) for FaceNet and 9.90% (x4) for VGGFace.

show abstract

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Cited by 344 publications

References 47 publications

ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods

ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods

A Domain Gap Aware Generative Adversarial Network for Multi-domain Image Translation

Selfie Periocular Verification using an Efficient Super-Resolution Approach

Contact Info

Product

Resources

About