Generative Adversarial Network for Pansharpening With Spectral and Spatial Discriminators

Abstract

The pansharpening problem amounts to fusing a high-resolution panchromatic image with a lowresolution multispectral image so as to obtain a high-resolution multispectral image. So the preservation of the spatial resolution of the panchromatic image and the spectral resolution of the multispectral image are of key importance for the pansharpening problem. To cope with it, we propose a new method based on a bi-discriminator in a Generative Adversarial Network (GAN) framework. The first discriminator is optimized to preserve textures of images by taking as input the luminance and the near infrared band of images, and the second discriminator preserves the color by comparing the chroma components Cb and Cr. Thus, this method allows to train two discriminators, each one with a different and complementary task. Moreover, to enhance these aspects, the proposed method based on bi-discriminator, and called MDSSC-GAN SAM, considers a spatial and a spectral constraints in the loss function of the generator. We show the advantages of this new method on experiments carried out on Pléiades and World View 3 satellite images.

Multimodal 4DVarNets for the reconstruction of sea surface dynamics from SST-SSH synergies

“…We may point out however that in such schemes, there is no guarantee for the pre-trained operators to be fully-relevant for the considered inversion task. End-to-end learning approaches may address these limitations as one may learn an inverse model using some reconstruction performance metrics in the training loss [2,38,24]. Deep learning methods for space-time inpainting issues [31] do not apply directly given the very high missing data rates to be accounted for with ocean remote sensing data.…”

“…Physics-driven learning schemes naturally arise as appealing approaches to benefit from prior physical knowledge on sea surface dynamics and associated satellite-derived observations. While one may complement classic end-to-end neural architecture with physics-informed training losses as illustrated in [24] for pan-sharpening applications, we here explore neural approaches which explicitly rely on a variational formulation similar to (2) [20,33]. Such approaches make explicit the exploitation of an underlying state-space formulation.…”

Pansharpening via Triplet Attention Network With Information Interaction

“…In this section, we conduct comparison experiments on reduced-scale and full-scale datasets. The methods used for comparison are BDSD [51], IHS [7], Indusion [50], P+XS [52], PNN [30], PanNet [31], PSGAN [38], and MDSSC-GAN [53]. In addition, extensive experiments of the proposed method are presented to provide a comprehensive analysis.…”

Triple‐loss driven generative adversarial network for pansharpening

“…TFNet [26] Convolutional neural network 0.8585 GTP-PNet [14] Convolutional neural network 0.8768 LDP-Net [16] Convolutional neural network 0.9182 MDSSC-GAN [27] Generative adversarial network 0.9289 PSGAN [28] Generative adversarial network 0.9316 DI-GAN [29] Generative adversarial network 0.9379 discriminator can circumvent the restriction of input scales and provide significant guidance for the generator. • Extensive comparative and ablation experiments are conducted to verify the effectiveness of the proposed method and each module.…”

“…In recent years, deep learning (DL)-based methods have emerged and gained significant attention in the field of pansharpening. The most representative ones are the convolutional neural network (CNN)-based methods [13,15,16,26,31] and the generative adversarial network (GAN)-based methods [27][28][29]32]. These DL-based methods have achieved remarkable performance improvements compared to traditional pansharpening techniques.…”