Cloud Removal of Optical Remote Sensing Imagery with Multitemporal Sar-Optical Data Using X-Mtgan

Xia, Yu; Zhang, Hongyan; Zhang, Liangpei; Fan, Zhiyu

doi:10.1109/igarss.2019.8899105

Cited by 11 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When it comes to the medium-resolution images such as Sentinel-1 and 2 images, the results may not be satisfying [31]. To deal with medium-resolution remote sensing images, some reconstruction methods [34][35][36] make use of multitemporal optical and SAR data as reference. These methods aim to train the network G with the concatenation of S 1 , S 2 and O 1 as input and O 2 as target to relieve the difficulty of direct transformation from SAR to optical images.…”

Section: Problem Formulationmentioning

confidence: 99%

“…To relieve the difficulty of direct transformation, some studies [34][35][36][37] proposed heterochronic SAR-based methods by further adding multitemporal SAR and optical images as reference. With pix2pix model, the authors of [34] synthesized the corrupted optical image with multitemporal SAR and optical images.…”

Section: Introductionmentioning

confidence: 99%

“…With pix2pix model, the authors of [34] synthesized the corrupted optical image with multitemporal SAR and optical images. Based on [34], in [36] the authors further made use of multitemporal SAR images as a constraint to optimize the network. In [35], the authors analyzed the results of transformation with multitemporal SAR and optical images and found that heterochronic reconstruction with SAR data outperforms isochronic SAR-based methods under the same model.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sentinel-2 Cloud Removal Considering Ground Changes by Fusing Multitemporal SAR and Optical Images

Gao¹,

Yi²,

Tang³

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Publicly available optical remote sensing images from platforms such as Sentinel-2 satellites contribute much to the Earth observation and research tasks. However, information loss caused by clouds largely decreases the availability of usable optical images so reconstructing the missing information is important. Existing reconstruction methods can hardly reflect the real-time information because they mainly make use of multitemporal optical images as reference. To capture the real-time information in the cloud removal process, Synthetic Aperture Radar (SAR) images can serve as the reference images due to the cloud penetrability of SAR imaging. Nevertheless, large datasets are necessary because existing SAR-based cloud removal methods depend on network training. In this paper, we integrate the merits of multitemporal optical images and SAR images to the cloud removal process, the results of which can reflect the ground information change, in a simple convolution neural network. Although the proposed method is based on deep neural network, it can directly operate on the target image without training datasets. We conduct several simulation and real data experiments of cloud removal in Sentinel-2 images with multitemporal Sentinel-1 SAR images and Sentinel-2 optical images. Experiment results show that the proposed method outperforms those state-of-the-art multitemporal-based methods and overcomes the constraint of datasets of those SAR-based methods.

show abstract

Section: Problem Formulationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sentinel-2 Cloud Removal Considering Ground Changes by Fusing Multitemporal SAR and Optical Images

Gao¹,

Yi²,

Tang³

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…The Generative Adversarial Network (GAN) [15] is a DL based method widely used in countless applications such as image inpainting [16] and domain translation [17], with outstanding results. The so-called image-to-image translation methods exploit GANs' ability to generate samples of complex probability distributions to synthesize optical images from SAR data over cloud-covered areas [11,13,[18][19][20].…”

Section: Introductionmentioning

confidence: 99%

“…Thanks to its success, pix2pix was later adapted to synthesize cloud-free multispectral optical images from a corresponding SAR image (e.g., [13]). Later works proposed improvements to that model, like new terms in the objective loss function [11,18], new training protocols [19], and also the use of temporal [20] and spectral [18] data. In all cases, the basic architectures comprising a generator and a discriminator network was maintained.…”

Section: Introductionmentioning

confidence: 99%

Atrous Cgan for Sar to Optical Image Translation

TURNES¹

View full text Add to dashboard Cite

The capture of land cover scenes with optical satellite sensors is often constrained by the presence of clouds that corrupt the collected images. Among the methods for recovering satellite optical images corrupted by clouds, several image to image translation approaches using Generative Adversarial Networks (GANs) have emerged with profitable results, managing to create realistic optical images from Synthetic Aperture Radar (SAR) data. Conditional GAN (cGAN) based methods proposed so far for SAR-to-optical image synthesis tend to produce noisy and unsharp optical outcomes. In this work, we propose the atrous-cGAN, a novel cGAN architecture that improves the SAR-to-optical image translation. The proposed generator and discriminator networks rely on atrous convolutions and incorporate the Atrous Spatial Pyramid Pooling (ASPP) module to enhance fine details in the generated optical image by exploiting spatial context at multiple scales. This work reports experiments carried out to assess the performance of atrous-cGAN for the synthesis of Landsat images from Sentinel-1A data based on four public datasets. The experimental analysis indicated that the atrous-cGAN overcomes the classical pix2pix model as a feature learning tool for semantic segmentation. The proposal also generates higher visual quality images, in general with higher similarity with the true optical image.

show abstract