Filmy Cloud Removal on Satellite Imagery with Multispectral Conditional Generative Adversarial Nets

Enomoto, Kenji; Satoh, Ken; Wang, Weimin; Fukui, Hiroshi; Matsuoka, M.; Nakamura, Ryosuke; Kawaguchi, Nobuo

doi:10.1109/cvprw.2017.197

Cited by 115 publications

(99 citation statements)

References 16 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In total, we use 24720 Sentinel-1/Sentinel-2 patch-pairs for training, and 1117 patch-pairs for testing. In extension to what [4] proposed for synthetic corruption of RGB channels, we synthesized cloud-and haze-corrupted input data by adding Perlin noise to all Sentinel-2 channels in an adaptive manner.…”

Section: Test Datasetmentioning

confidence: 99%

“…Among different GAN rationales, conditional GANs (cGANs) have attracted considerable interest in the remote sensing community, as they allow to generate desired artificial data based on a specified target output [7]. As an example related to the content of this paper, an approach proposed in [4] uses the cGAN concept to generate cloud-free RGB images from combined cloud-affected RGB and cloudfree near infrared (NIR) measurements. The problem with that approach is that the majority of types of clouds are impenetrable not only to visible but also to infrared light [8].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper we overcome the above-outlined problem by advancing the idea of [4] in that we use co-registered SAR instead of NIR data as auxiliary input information about the ground covered by clouds. Furthermore, our implementation allows for MS input images with more than 3 bands, it incorporates deeper adversarial networks, and supports a large array of remote sensing data formats and bit-depths.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Conditional Generative Adversarial Network to Fuse Sar And Multispectral Optical Data For Cloud Removal From Sentinel-2 Images

Grohnfeldi

Schmitt

Zhu

2018

IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium

119

View full text Add to dashboard Cite

In this paper, we present the first conditional generative adversarial network (cGAN) architecture that is specifically designed to fuse synthetic aperture radar (SAR) and optical multi-spectral (MS) image data to generate cloud-and hazefree MS optical data from a cloud-corrupted MS input and an auxiliary SAR image. Experiments on Sentinel-2 MS and Sentinel-1 SAR data confirm that our extended SAR-Opt-cGAN model utilizes the auxiliary SAR information to better reconstruct MS images than an equivalent model which uses the same architecture but only single-sensor MS data as input.

show abstract

Section: Test Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Conditional Generative Adversarial Network to Fuse Sar And Multispectral Optical Data For Cloud Removal From Sentinel-2 Images

Grohnfeldi

Schmitt

Zhu

2018

IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium

119

View full text Add to dashboard Cite

show abstract

“…O N AVERAGE about 55% of the Earth's land surface is covered by clouds [1], impacting the aim of missions, such as Copernicus, to reliably provide noise-free observations at a high frequency, a prerequisite for applications relying on temporally seamless monitoring of our environment, such as change detection or monitoring [2]- [5]. The need for cloud-free Earth observations, hence, gave rise to a rapidly growing number of cloud removal methods [6]- [12]. While the aforementioned contributions share the common aim of dehazing and declouding optical imagery, the majority of methods are evaluated on narrowly defined and geospatially distinct regions of interest (ROIs).…”

Section: Introductionmentioning

confidence: 99%

“…While optical imagery is affected by bad weather conditions and lack of daylight, sensors based on synthetic aperture radar (SAR) as mounted on Sentinel-1 (S1) satellites are not [14] and, thus, provide a valuable source of complementary information. Recent advances in cloud removal combine multimodal data with deep neural networks recovering the affected areas [6], [7], [12], [15].…”

Section: Introductionmentioning

confidence: 99%

Multisensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

Ebel

Meraner

Schmitt

et al. 2021

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

The majority of optical observations acquired via spaceborne Earth imagery are affected by clouds. While there is numerous prior work on reconstructing cloud-covered information, previous studies are, oftentimes, confined to narrowly defined regions of interest, raising the question of whether an approach can generalize to a diverse set of observations acquired at variable cloud coverage or in different regions and seasons. We target the challenge of generalization by curating a large novel data set for training new cloud removal approaches and evaluate two recently proposed performance metrics of image quality and diversity. Our data set is the first publically available to contain a global sample of coregistered radar and optical observations, cloudy and cloud-free. Based on the observation that cloud coverage varies widely between clear skies and absolute coverage, we propose a novel model that can deal with either extreme and evaluate its performance on our proposed data set. Finally, we demonstrate the superiority of training models on real over synthetic data, underlining the need for a carefully curated data set of real observations. To facilitate future research, our data set is made available online.

show abstract

Research on cloud removal based on fusing Multi-temporal Remote Sensing Images

Zou

Tao

et al. 2021

Preprint

View full text Add to dashboard Cite

Optical satellite images are a critical data source; however, cloud cover often compromises their quality, hindering image applications and analysis. Consequently, effectively removing clouds from optical satellite images has emerged as a prominent research direction. While recent advancements in cloud removal primarily rely on generative adversarial networks, which may yield suboptimal image quality, diffusion models have demonstrated remarkable success in diverse image-generation tasks, showcasing their potential in addressing this challenge. This paper presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery. Specifically, we introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output. Moreover, we propose a novel and efficient time and condition fusion block within the cloud removal model to accurately simulate the correspondence between the appearance in the conditional image and the target image at a low computational cost. Extensive experimental evaluations on two commonly used benchmark datasets demonstrate that DiffCR consistently achieves state-of-the-art performance on all metrics, with parameter and computational complexities amounting to only 5.1% and 5.4%, respectively, of those previous best methods. The source code, pre-trained models, and all the experimental results will be publicly available at https://github.com/XavierJiezou/DiffCR upon the paper's acceptance of this work.

show abstract

Filmy Cloud Removal on Satellite Imagery with Multispectral Conditional Generative Adversarial Nets

Abstract: In this paper, we propose a method for cloud removal from visible light RGB satellite images

Cited by 115 publications

References 16 publications

A Conditional Generative Adversarial Network to Fuse Sar And Multispectral Optical Data For Cloud Removal From Sentinel-2 Images

A Conditional Generative Adversarial Network to Fuse Sar And Multispectral Optical Data For Cloud Removal From Sentinel-2 Images

Multisensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

Research on cloud removal based on fusing Multi-temporal Remote Sensing Images

Contact Info

Product

Resources

About