Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model

Abstract

Most of the existing learning-based single image super-resolution (SISR) methods are trained and evaluated on simulated datasets, where the low-resolution (LR) images are generated by applying a simple and uniform degradation (i.e., bicubic downsampling) to their high-resolution (HR) counterparts. However, the degradations in real-world LR images are far more complicated. As a consequence, the SISR models trained on simulated data become less effective when applied to practical scenarios. In this paper, we build a real-world super-resolution (RealSR) dataset where paired LR-HR images on the same scene are captured by adjusting the focal length of a digital camera. An image registration algorithm is developed to progressively align the image pairs at different resolutions. Considering that the degradation kernels are naturally non-uniform in our dataset, we present a Laplacian pyramid based kernel prediction network (LP-KPN), which efficiently learns per-pixel kernels to recover the HR image. Our extensive experiments demonstrate that SISR models trained on our RealSR dataset deliver better visual quality with sharper edges and finer textures on real-world scenes than those trained on simulated datasets. Though our RealSR dataset is built by using only two cameras (Canon 5D3 and Nikon D810), the trained model generalizes well to other camera devices such as Sony a7II and mobile phones.

DDet: Dual-Path Dynamic Enhancement Network for Real-World Image Super-Resolution

“…where K Gauss is a Gaussian blur operator and ∂ denotes the degradation process. ∂ usually adopts a Bicubic downscale algorithm in traditional image SR. Real-SR dataset [20] is proposed to address real-world degeneration metrics by capturing I HR and I LR with different quality optical sensors and resolution settings. The data collection manner inherently make the image pair has different resolution property.…”

“…EXPERIMENT Dataset and Baselines. We employ real-world single image super-resolution dataset(RealSR) [20] [17], our method exhibit a 0.61 dB gain. It notes that EDSR gets into overfitting as the training sample number is limited.…”

“…DFN [18] opens up a new solution to turn convolutional features into dynamic filters for feature aggregation. Similar idea [19], [20] is addressed in the image restoration task yet. KPN [20] proposes a dynamic prediction mechanism by employing the high-dimensional feature into convolutional kernels.…”

“…Similar idea [19], [20] is addressed in the image restoration task yet. KPN [20] proposes a dynamic prediction mechanism by employing the high-dimensional feature into convolutional kernels. LP-KPN [20] further conducts the dynamic kernels on multi-scale feature for image restoration.…”

“…Compared with the traditional convolutional kernel, our spatial attention kernel is learned in a content-adaptive fashion, which is more flexible. Compared with dynamic filter pipelines [25], [20], our model uses multiple dynamic kernels from the different receptive field without information loss, which make our method enjoys rich diversity. We also conduct a lightweight coupled detail manipulation(CDM) to relax pixel displacement in real image SR.…”