Unsupervised Learning for Real-World Super-Resolution

Lugmayr, Andreas; Danelljan, Martin; Timofte, Radu

doi:10.1109/iccvw.2019.00423

Cited by 171 publications

(118 citation statements)

References 40 publications

Supporting

Mentioning

114

Contrasting

Order By: Relevance

“…In this subsection, the DCR and DRM are removed respectively to form the Baseline-I and Baseline-II. Two Baselines are trained on the same training set provided by AIMRWSR challenge [21,20]. It is verified that the proposed DCR and DRM can be used to improve the visual and objective quality of the reconstructed image.…”

Section: Ablation Studymentioning

confidence: 84%

“…As shown in Fig. 1, the proposed method can provide more fine details of the reconstructed image based on training set provided by [21,20]. In summary, the main contribution of the proposed method is two-fold:…”

Section: Introductionmentioning

confidence: 92%

“…Therefore, we select the trained model at iteration 80, 000 as our final ESRGAN model. The proposed SR model is trained based on AIMRWSR training set (source domain) [21,20]. The scaling factor is 4.…”

Section: Experiments Settingsmentioning

confidence: 99%

See 2 more Smart Citations

Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution

Hsu

Lin

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Deep learning-based single image super-resolution enables very fast and high-visual-quality reconstruction. Recently, an enhanced super-resolution based on generative adversarial network (ESRGAN) has achieved excellent performance in terms of both qualitative and quantitative quality of the reconstructed high-resolution image. In this paper, we propose to add one more shortcut between two dense-blocks, as well as add shortcut between two convolution layers inside a dense-block. With this simple strategy of adding more shortcuts in the proposed network, it enables a faster learning process as the gradient information can be back-propagated more easily. Based on the improved ESRGAN, the dual reconstruction is proposed to learn different aspects of the super-resolved image for judiciously enhancing the quality of the reconstructed image. In practice, the super-resolution model is pre-trained solely based on pixel distance, followed by fine-tuning the parameters in the model based on adversarial loss and perceptual loss. Finally, we fuse two different models by weighted-summing their parameters to obtain the final super-resolution model. Experimental results demonstrated that the proposed method achieves excellent performance in the real-world image super-resolution challenge. We have also verified that the proposed dual reconstruction does further improve the quality of the reconstructed image in terms of both PSNR and SSIM. Figure 1. Visual quality comparison between the (a) ground truth and the super-resolved results by using (b) Bicubic interpolation, (c) ESRGAN [26], and (d) proposed method.

show abstract

Section: Ablation Studymentioning

confidence: 84%

Section: Introductionmentioning

confidence: 92%

See 1 more Smart Citation

Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution

Hsu

Lin

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…The corruptions in the source data are artificial but unknown. The validation and test set contain 100 images each and have the same corruptions as the source data [26].…”

Section: Esrgan and Esrgan-fsmentioning

confidence: 99%

Frequency Separation for Real-World Super-Resolution

Fritsche

Timofte

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

190

187

View full text Add to dashboard Cite

Most of the recent literature on image super-resolution (SR) assumes the availability of training data in the form of paired low resolution (LR) and high resolution (HR) images or the knowledge of the downgrading operator (usually bicubic downscaling). While the proposed methods perform well on standard benchmarks, they often fail to produce convincing results in real-world settings. This is because real-world images can be subject to corruptions such as sensor noise, which are severely altered by bicubic downscaling. Therefore, the models never see a real-world image during training, which limits their generalization capabilities. Moreover, it is cumbersome to collect paired LR and HR images in the same source domain.To address this problem, we propose DSGAN to introduce natural image characteristics in bicubically downscaled images. It can be trained in an unsupervised fashion on HR images, thereby generating LR images with the same characteristics as the original images. We then use the generated data to train a SR model, which greatly improves its performance on real-world images. Furthermore, we propose to separate the low and high image frequencies and treat them differently during training. Since the low frequencies are preserved by downsampling operations, we only require adversarial training to modify the high frequencies. This idea is applied to our DSGAN model as well as the SR model. We demonstrate the effectiveness of our method in several experiments through quantitative and qualitative analysis. Our solution is the winner of the AIM Challenge on Real World SR at ICCV 2019. arXiv:1911.07850v1 [eess.IV] 18 Nov 2019 1 Our code and models are publicly available at https://github. com/ManuelFritsche/real-world-sr

show abstract

“…Moreover, it cannot be applied to old photo content. This challenge therefore focuses on the fully unsupervised super-resolution case, similar to the setting employed in many recent works [32,18,5,23], Where no reference high-resolution images are available for training.…”

Section: Introductionmentioning

confidence: 99%

AIM 2019 Challenge on Real-World Image Super-Resolution: Methods and Results

Lugmayr

Joon

Won

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

104

View full text Add to dashboard Cite

This paper reviews the AIM 2019 challenge on real world super-resolution. It focuses on the participating methods and final results. The challenge addresses the real world setting, where paired true high and low-resolution images are unavailable. For training, only one set of source input images is therefore provided in the challenge. In Track 1: Source Domain the aim is to super-resolve such images while preserving the low level image characteristics of the source input domain. In Track 2: Target Domain a set of high-quality images is also provided for training, that defines the output domain and desired quality of the superresolved images. To allow for quantitative evaluation, the source input images in both tracks are constructed using artificial, but realistic, image degradations. The challenge is the first of its kind, aiming to advance the state-of-the-art and provide a standard benchmark for this newly emerging task. In total 7 teams competed in the final testing phase, demonstrating new and innovative solutions to the problem.

show abstract

Unsupervised Learning for Real-World Super-Resolution

Cited by 171 publications

References 40 publications

Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution

Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution

Frequency Separation for Real-World Super-Resolution

AIM 2019 Challenge on Real-World Image Super-Resolution: Methods and Results

Contact Info

Product

Resources

About