Saliency-Guided Remote Sensing Image Super-Resolution

Abstract

Deep learning has recently attracted extensive attention and developed significantly in remote sensing image super-resolution. Although remote sensing images are composed of various scenes, most existing methods consider each part equally. These methods ignore the salient objects (e.g., buildings, airplanes, and vehicles) that have more complex structures and require more attention in recovery processing. This paper proposes a saliency-guided remote sensing image super-resolution (SG-GAN) method to alleviate the above issue while maintaining the merits of GAN-based methods for the generation of perceptual-pleasant details. More specifically, we exploit the salient maps of images to guide the recovery in two aspects: On the one hand, the saliency detection network in SG-GAN learns more high-resolution saliency maps to provide additional structure priors. On the other hand, the well-designed saliency loss imposes a second-order restriction on the super-resolution process, which helps SG-GAN concentrate more on the salient objects of remote sensing images. Experimental results show that SG-GAN achieves competitive PSNR and SSIM compared with the advanced super-resolution methods. Visual results demonstrate our superiority in restoring structures while generating remote sensing super-resolution images.

DAE2GAN: Image super-resolution for remote sensing based on an improved edge-enhanced generative adversarial network with double-end attention mechanism

“…Many people propose their work by modifying the network structure and designed some novel loss functions for SRGAN: Xiong et al 28 used a Wasserstein distance to replace the KL and JS divergence as well as modify part of network structure; Zhang et al 29 designed an encoder-decoder structure for unsupervised SR and used a robust loss function based on perceptual loss; Xu et al 30 designed residual generators with self-attention mechanisms and weight normalization and combined multiple losses to optimize the training process. Liu et al 31 proposed an SG-GAN network based on saliency guidance, focusing on more salient parts of complex structures while maintaining rich edges in details. Dong et al 32 proposed a reference-based RRSGAN, which has better generalization ability in different scenarios by using gradient-assisted feature alignment and incorporating the relevant attention module (RAM).…”

“…designed residual generators with self-attention mechanisms and weight normalization and combined multiple losses to optimize the training process. Liu et al 31 . proposed an SG-GAN network based on saliency guidance, focusing on more salient parts of complex structures while maintaining rich edges in details.…”

Resolution-Agnostic Remote Sensing Scene Classification With Implicit Neural Representations

“…Lei et al [44] propose coupled adversarial training with a well-designed discriminator to learn a better discrimination between the super-resolved image and the corresponding ground truth. Liu et al [45] design a saliency-guided GAN method to improve visual results with additional salient priors. Some researchers focus on the SR of remote sensing satellite videos.…”

Exploiting Digital Surface Models for Inferring Super-Resolution for Remotely Sensed Images

“…In this direction, Gong et al [31] proposed the Enlighten-GAN model that uses a self-supervised hierarchical perceptual loss. Liu et al [32] exploited the salient maps of images to learn additional structure priors and to make the model focus more on the salient objects. Huan et al [33] proposed a multi-scale residual network with hierarchical feature fusion and multiscale dilation residual blocks.…”

“…In this paper, the authors exploit the best practices derived from state-of-the-art experimentation up to date, suggesting partly keeping the training principles (content and adversarial losses) of the highly successful ESRGAN while modifying the way the perceptual > REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) < loss is conceived in the context of super-resolution in general and in remote sensing specifically. The key idea of the proposed approach lies in the observation that single-image superresolution is an ill-posed problem in the sense that for any LR image exist numerous HR images that could correspond to it [23], [31], [32]. Thus, for any successful model to achieve superior performance, it must derive significant pixel-level knowledge during the training.…”