NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results

Zhang, Kai; Gu, Shuhang; Timofte, Radu; Shang, Taizhang; Dai, Qiuju; Zhu, Shengchen; Yang, Tong; Guo, Yandong; Jo, Yeon Hwa; Yang, Sejong; Kim, Seon Joo; Zha, Lin; Jiang, Joe‐Air; Gao, Xinbo; Lu, Wen; Liu, Jing; Yoon, Kang Sup; Jeon, Taegyun; Akita, Kazutoshi; Ooba, Takeru; Ukita, Norimichi; Luo, Zhipeng; Yao, Yuehan; Xu, Zhenyu; He, Dongliang; Wu, Wenhao; Ding, Yukang; Li, Chao; Fu, Li; Wen, Shifeng; Li, Jianwei; Yang, Fuzhi; Yang, Huan; Fu, Jianlong; Kim, Byung–Hoon; Baek, Jehyun; Ye, Jong Chul; Yi, Fan; Huang, Thomas S.; Lee, Junyeop; Lee, Bokyeung; Min, Jungki; Kim, Gwantae; Lee, Kanghyu; Park, Jaihyun; Mykhailych, Mykola; Zhong, Haoyu; Shi, Yukai; Yang, Xuemin; Yang, Zhijing; Li, Lin; Zhao, Tongtong; Peng, Jinjia; Wang, Huibing; Jin, Zhi; Wu, Jiahao; Chen, Yifu; Shang, Chenming; Zhang, Huanrong; Min, Jeong‐Ki; Hrishikesh, P. S.; Puthussery, Densen; Jiji, C. V.

doi:10.1109/cvprw50498.2020.00254

Cited by 50 publications

(25 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since real-world SR is challenging due to unknown degradation and various noise [34], [61], [74]- [79], we also validate the effectiveness of our method for compressed inputs in Figure 19. Unlike previous experiments, FxSR and SRGAN [20] are re-trained using LR images compressed with JPEG quality factor 90, called FxSR-CA (compression artifacts) and SRGAN-CA.…”

Section: E Compressed Lr Image Restorationmentioning

confidence: 71%

Flexible Style Image Super-Resolution Using Conditional Objective

2022

View full text Add to dashboard Cite

Recent studies have significantly enhanced the performance of single-image super-resolution (SR) using convolutional neural networks (CNNs). While there can be many high-resolution (HR) solutions for a given input, most existing CNN-based methods do not explore alternative solutions during the inference. A typical approach to obtaining alternative SR results is to train multiple SR models with different loss weightings and exploit the combination of these models. Instead of using multiple models, we present a more efficient method to train a single adjustable SR model on various combinations of losses by taking advantage of multi-task learning. Specifically, we optimize an SR model with a conditional objective during training, where the objective is a weighted sum of multiple perceptual losses at different feature levels. The weights vary according to given conditions, and the set of weights is defined as a style controller. Also, we present an architecture appropriate for this training scheme, which is the Residual-in-Residual Dense Block equipped with spatial feature transformation layers. At the inference phase, our trained model can generate locally different outputs conditioned on the style control map. Extensive experiments show that the proposed SR model produces various desirable reconstructions without artifacts and yields comparable quantitative performance to state-of-the-art SR methods. Code and trained models are available at https://github.com/seungho-snu/FxSR.

show abstract

Section: E Compressed Lr Image Restorationmentioning

confidence: 71%

Flexible Style Image Super-Resolution Using Conditional Objective

2022

View full text Add to dashboard Cite

show abstract

“…The learned perceptual image patch similarity (LPIPS) score [4] was proposed to measure the perceptual quality on various computer vision tasks. According to [2], the LPIPS score reliably coincides with human perception for assessing super-resolved images. We use the LPIPS score as an indicator of perceptual image quality in our experiments.…”

Section: Related Workmentioning

confidence: 89%

One-to-many Approach for Improving Super-Resolution

Park¹,

Lee²

2021

Preprint

View full text Add to dashboard Cite

Super-resolution (SR) is a one-to-many task with multiple possible solutions. However, previous works were not concerned about this characteristic. For a oneto-many pipeline, the generator should be able to generate multiple estimates of the reconstruction, and not be penalized for generating similar and equally realistic images. To achieve this, we propose adding weighted pixel-wise noise after every Residual-in-Residual Dense Block (RRDB) to enable the generator to generate various images. We modify the strict content loss to not penalize the stochastic variation in reconstructed images as long as it has consistent content. Additionally, we observe that there are out-of-focus regions in the DIV2K, DIV8K datasets that provide unhelpful guidelines. We filter blurry regions in the training data using the method of [10]. Finally, we modify the discriminator to receive the lowresolution image as a reference image along with the target image to provide better feedback to the generator. Using our proposed methods, we were able to improve the performance of ESRGAN in ×4 perceptual SR and achieve the state-of-the-art LPIPS score in ×16 perceptual extreme SR.Preprint. Under review.

show abstract

“…The review for super-resolution can be referred to [174] [175]. The review for GAN based superresolution can be referred to [176] [177].…”

Section: A Super-resolutionmentioning

confidence: 99%

A Review on Remote Sensing Data Fusion with Generative Adversarial Networks (GAN)

Liu¹

2021

Preprint

View full text Add to dashboard Cite

In the past decades, remote sensing (RS) data fusion has always been an active research community. A large number of algorithms and models have been developed. Generative Adversarial Networks (GAN), as an important branch of deep learning, show promising performances in variety of RS image fusions. This review provides an introduction to GAN for remote sensing data fusion. We briefly review the frequently-used architecture and characteristics of GAN in data fusion and comprehensively discuss how to use GAN to realize fusion for homogeneous RS data, heterogeneous RS data, and RS and ground observation data. We also analyzed some typical applications with GAN-based RS image fusion. This review takes insight into how to make GAN adapt to different types of fusion tasks and summarizes the advantages and disadvantages of GAN-based RS data fusion. Finally, we discuss the promising future research directions and make a prediction on its trends.

show abstract

NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results

Cited by 50 publications

References 35 publications

Flexible Style Image Super-Resolution Using Conditional Objective

Flexible Style Image Super-Resolution Using Conditional Objective

One-to-many Approach for Improving Super-Resolution

A Review on Remote Sensing Data Fusion with Generative Adversarial Networks (GAN)

Contact Info

Product

Resources

About