Cloud-Aware Generative Network: Removing Cloud From Optical Remote Sensing Images

Sun, Linjian; Zhang, Ye; Chang, Xuling; Wang, Yanjie; Xu, Jiajia

doi:10.1109/lgrs.2019.2928840

Cited by 21 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For homogenous data, Chen et al [34] proposed the CTS-CNN model based on GAN to reconstruct images in ZY-3 with small ratios through the content generation , texture generation, and the spectrum generation networks. Sun et al [35] proposed a cloud-aware generative network (CGAN) to restore the missing information from Google Earth satellite images in relatively complex scenes. Meraner et al [36] constructed a deep residual neural network based on GAN to reconstruct weakly textured scenes such as mountains, water, and forests in Sentinel-2 (10 m).…”

Section: B Reconstruction Of Temporal-based Methodsmentioning

confidence: 99%

DecRecNet: A Decoupling-Reconstruction Network for Restoring the Missing Information of Optical Remote Sensing Images

Liu,

Cui,

Jiang

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Temporal-based methods effectively improve the utilization rate of remote sensing images, but large ratios of missing information still need to be improved in the reconstruction models. In this paper, based on the imaging theory with the help of a radiation correction model, a decoupling-reconstruction network (DecRecNet) for image reconstruction is proposed. The network uses a ground content radiation (GCR) correction module and imaging environment radiation (IER) correction module and their corresponding loss functions to decouple the image radiation information into ground content radiation related to the ground objects and imaging environment radiation related to the imaging conditions and carry out targeted processing to achieve the purpose of missing information reconstruction. The ground content radiation consistency loss function is used to preserve the ground content information, and the imaging environment radiation consistency loss function and imaging environment radiation smoothness loss are used to coordinate the imaging environment. The radiation guiding module further performs targeted radiation adjustment on the foreground and background images to transfer the background imaging environment to the foreground and consistent radiation information of the same ground object. Compared with the classical U-Net, DeepLabV3+, RFR-Net, and STS-CNN methods, our model showed remarkable advantages in cloud occlusion and stripes of Landsat-8 (30 m), GaoFen-1 (2 m), and Landsat-7 (30 m) at various missing ratios, data sources, resolutions, and scenes, and achieving the goal of missing information reconstruction in large missing ratios.

show abstract

Section: B Reconstruction Of Temporal-based Methodsmentioning

confidence: 99%

DecRecNet: A Decoupling-Reconstruction Network for Restoring the Missing Information of Optical Remote Sensing Images

Liu,

Cui,

Jiang

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Furthermore, in sharp contrast to MSDA-CR [29] and CR-MSS [58] that utilize multispectral data as input, we mainly focus on visible (RGB) bands in our evaluation. This is because RGB images are more commonly available [11], [12], [28], [59]. However, we also perform supplement experiments to demonstrate that the proposed model can work well with multispectral data by exploiting both RGB and near-infrared (NIR) data.…”

Section: A Datasetsmentioning

confidence: 97%

Cloud-EGAN: Rethinking CycleGAN From a Feature Enhancement Perspective for Cloud Removal by Combining CNN and Transformer

Huang

Zhang

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Cloud cover presents a major challenge for geoscience research of remote sensing images with thick clouds causing complete obstruction with information loss while thin clouds blurring the ground objects. Deep learning (DL) methods based on convolutional neural networks (CNNs) have recently been introduced to the cloud removal task. However, their performance is hindered by their weak capabilities in contextual information extraction and aggregation. Unfortunately, such capabilities play a vital role in characterizing remote sensing images with complex ground objects. In this work, the conventional cycle-consistent generative adversarial network (CycleGAN) is revitalized from a feature enhancement perspective. More specifically, a saliency enhancement (SE) module is first designed to replace the original CNN module in CycleGAN to re-calibrate channel attention weights to capture detailed information for multi-level feature maps. Furthermore, a high-level feature enhancement (HFE) module is developed to generate contextualized cloud-free features while suppressing cloud components. In particular, HFE is composed of both CNN-and transformer-based modules. The former enhances the local high-level features by employing residual learning and multi-scale strategies, while the latter captures the long-range contextual dependencies with the Swin transformer module to exploit high-level information from a global perspective. Capitalizing on the SE and HFE modules, an effective Cloud-Enhancement GAN, namely Cloud-EGAN, is proposed to accomplish thin and thick cloud removal tasks. Extensive experiments on the RICE and the WHUS2-CR datasets confirm the impressive performance of Cloud-EGAN.

show abstract

“…Currently, deep learning-based methods are gaining considerable attention. They have the potential to solve many of the problems that arise in traditional cloud removal methods and achieve impressive results [33][34][35]. For example, Multispectral conditional Generative Adversarial Networks (McGANs), leveraging the remarkable generative capabilities of conditional Generative Adversarial Networks (cGANs), remove simulated clouds from Worldview-2 imagery by extending the input channels of cGANs to be compatible with multispectral input [25].…”

Section: B Algorithms For Cloud Removalmentioning

confidence: 99%

Multimodal and Multiresolution Data Fusion for High-Resolution Cloud Removal: A Novel Baseline and Benchmark

Xu,

Shi,

Ebel

et al. 2024

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Cloud removal is a significant and challenging problem in remote sensing, and in recent years, there have been notable advancements in this area. However, two major issues remain hindering the development of cloud removal: the unavailability of high-resolution imagery for existing datasets and the absence of evaluation regarding the semantic meaningfulness of the generated structures. In this paper, we introduce M3R-CR, a benchmark dataset for high-resolution Cloud Removal with Multi-Modal and Multi-Resolution data fusion. M3R-CR is the first public dataset for cloud removal to feature globally sampled high-resolution optical observations, paired with radar measurements and pixel-level land cover annotations. With this dataset, we consider the problem of cloud removal in highresolution optical remote sensing imagery by integrating multimodal and multi-resolution information. In this context, we have to take into account the alignment errors caused by the multi-resolution nature, along with the more pronounced misalignment issues in high-resolution images due to inherent imaging mechanism differences and other factors. Existing multimodal data fusion based methods, which assume the image pairs are aligned accurately at pixel-level, are thus not appropriate for this problem. To this end, we design a new baseline named Align-CR to perform the low-resolution SAR image guided highresolution optical image cloud removal. It gradually warps and fuses the features of the multi-modal and multi-resolution data during the reconstruction process, effectively mitigating concerns associated with misalignment. In the experiments, we evaluate the performance of cloud removal by analyzing the quality of visually pleasing textures using image reconstruction metrics and further analyze the generation of semantically meaningful structures using a well-established semantic segmentation task. The proposed Align-CR method is superior to other baseline methods in both areas. The project is available at https://github. com/zhu-xlab/M3R-CR.

show abstract

Cloud-Aware Generative Network: Removing Cloud From Optical Remote Sensing Images

Cited by 21 publications

References 13 publications

DecRecNet: A Decoupling-Reconstruction Network for Restoring the Missing Information of Optical Remote Sensing Images

DecRecNet: A Decoupling-Reconstruction Network for Restoring the Missing Information of Optical Remote Sensing Images

Cloud-EGAN: Rethinking CycleGAN From a Feature Enhancement Perspective for Cloud Removal by Combining CNN and Transformer

Multimodal and Multiresolution Data Fusion for High-Resolution Cloud Removal: A Novel Baseline and Benchmark

Contact Info

Product

Resources

About