Semi‐supervised image super‐resolution with attention CycleGAN

Abstract

Single-Image Super-Resolution (SISR) has always been an important topic in the field of image processing, which attempts to improve the image resolution and is of great significance in practice. Recently, SISR has made substantial progress aided by deep learning (DL), which has demonstrated impressive potential in many low-level tasks. In the current DL-based SISR approaches, most of them are based on supervised learning. However, in the real world, only low-resolution (LR) images with unknown degradation are provided, which limit the application of current supervised models. To mitigate this problem, in this paper, a two-stage semi-supervised SISR method called SRAttentionGAN, is proposed. First, an upsampling network SRResNet, which is pre-trained in a supervised manner, is employed to scale the LR image to the desired size. Then, the upsampled results are fed into our improved unsupervised CycleGAN framework, which does not need paired samples, to obtain sharper and more realistic super-resolution (SR) images. Specifically, in the improved CycleGAN part, an attention-guided generator is proposed to perceive the discriminative semantic parts between the source and target images, to avoid the impact from low-level information. It also prevents the overall color tone from being changed. A multiscale discriminator is also adopted to further rich texture details. The effectiveness of the proposed SRAttentionGAN experiments is validated using four benchmarks (Set5, Set14, Urban100, and BSDS100) in both quantitative and qualitative aspects. Compared with the state-of-the-arts, the results are visually promising and show competitive performance in perceptual metrics, Natural Image Quality Evaluator (NIQE) and Perception Index (PI), which have better agreement with the human visual perception. INTRODUCTIONRecently, due to the rapid development of network communication and high-end display devices, single image super-resolution (SISR), which is one of the most important low-level tasks, has drawn increasing attention in both academia and industry. The goal of SISR is to reconstruct a high-resolution (HR) image from its corresponding low-resolution (LR) image. In recent decade, many SISR methods based on deep learning (DL) have emerged and achieved significant improvements in Peak Signal to Noise Ratio (PSNR)-based evaluations [1-7], or in perceptual image quality evaluations [8-10] over traditional methods. However, most of these methods belong to supervised learning, which means that a large number of paired HR and LR images are required and it is difficult to collect such a dataset This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Semi-Supervised Super-Resolution

“…Cycle-in-Cycle GAN (CinCGAN) [24] is one work that focuses on accomplishing unsupervised SR by implementing a network consisting of three generators and two discriminators. Recently proposed [25] makes use of a pre-trained SR network and the CycleGAN architecture consisting of two generators and two discriminators for super-resolution, making it exceptionally computationally expensive to train. On the other hand, the proposed architecture in this paper consists of only one generator and one discriminator, which drastically reduces the number of parameters and is consequently easier to train.…”

Novel Deep Learning Technique to Improve Resolution of Low-Quality Finger Print Image for Bigdata Applications

“…Gan based SR model requires training for both generator and discriminator separately and it takes longer time for reconstructing a betterquality image compared to other models, hence it is practically not feasible for employing to realtime applications. Mingzheng Hou et al [25] employed GAN based network for generating good quality image. They applied SRResnet for upsampling the input image of appropriate size.…”