UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series

Ebel, Patrick; Garnot, Vivien Sainte Fare; Schmitt, Michael Henry; Wegner, Jan Dirk; Zhu, Xiao Xiang

doi:10.1109/cvprw59228.2023.00202

Cited by 7 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table VI presents a quantitative comparison of cloud removal performance and efficiency between our proposed DiffCR and existing models, namely MCGAN [12], Pix2Pix [13], AE [22], STNet [23], DSen2-CR [25], STGAN [14], CTGAN [15], &ORXG\T CR-TS-Net [16], PMAA [24], UnCRtainTS [26], and DDPM-CR [27]. Except for DDPM-CR and DiffCR, which are based on diffusion methods, all others are based on either GAN or regression (without adversarial loss).…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Research on cloud removal based on fusing Multi-temporal Remote Sensing Images

Zou

Tao

et al. 2021

Preprint

View full text Add to dashboard Cite

Optical satellite images are a critical data source; however, cloud cover often compromises their quality, hindering image applications and analysis. Consequently, effectively removing clouds from optical satellite images has emerged as a prominent research direction. While recent advancements in cloud removal primarily rely on generative adversarial networks, which may yield suboptimal image quality, diffusion models have demonstrated remarkable success in diverse image-generation tasks, showcasing their potential in addressing this challenge. This paper presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery. Specifically, we introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output. Moreover, we propose a novel and efficient time and condition fusion block within the cloud removal model to accurately simulate the correspondence between the appearance in the conditional image and the target image at a low computational cost. Extensive experimental evaluations on two commonly used benchmark datasets demonstrate that DiffCR consistently achieves state-of-the-art performance on all metrics, with parameter and computational complexities amounting to only 5.1% and 5.4%, respectively, of those previous best methods. The source code, pre-trained models, and all the experimental results will be publicly available at https://github.com/XavierJiezou/DiffCR upon the paper's acceptance of this work.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…are also some works [16], [25], [26] that utilize radar imagery to assist in cloud removal, as well as a limited number of works [27], [28] that employ diffusion models. However, due to the limitations of GAN frameworks and adversarial learning mechanisms, existing GAN-based methods struggle to attain satisfactory image fidelity.…”

Section: A Cloud Removalmentioning

confidence: 99%

Research on cloud removal based on fusing Multi-temporal Remote Sensing Images

Zou

Tao

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…h) Include attention modules: Inspired by the success of attention modules [161], [185], [186]. The main focus of application in RS data has been to perform temporal attention [1], [15], [36], [74], [155], [157], [165], [182], [187], [188]. Nevertheless, in RS image-based applications, the use-cases have been extended as a way to enhance the information.…”

Section: Modeling Considerationsmentioning

confidence: 99%

“…Meanwhile, active-based views are the quintessential views chosen to complement (and improve by fusion) the optical in classification tasks with MV models, e.g. by using SAR view [1], [15], [33], [34], [36], [99], [113], [120], [124], [126], [143], [163], [173], [174], [188], [190], [193], [199]- [202] or LiDAR view [94], [139]. Furthermore, the DSM view has been widely used together with the optical view [17]- [19], [55], [58], [64], [65], [76], [80], [84], [85], [88], [89], [92], [104], [123], [184], where on some occasions is a LiDAR-derived DSM [32], [34], [35], [59], [62], [81], [86], [87], [93], …”

Section: A Which Views Are Most Used In Earth Observation?mentioning

confidence: 99%

“…The visible light of the spectrum (RGB bands concretely) have been shown to be more relevant than other spectral bands in the optical view when considering the predictive performance of NN models [16], [99], [204]. Besides, views with coarse resolutions are usually worst in predictive performance than finer-resolution views [106], [120], [201], and multi-temporal views have better predictive performance than static views [99], [106], [188], [191], [205]. Although RS-based views provide valuable and more accessible information for downstream tasks than ground-based views [30], [117], some works have used interesting data sources to complement RS-based views.…”

Section: A Which Views Are Most Used In Earth Observation?mentioning

confidence: 99%

See 1 more Smart Citation

Common Practices and Taxonomy in Deep Multiview Fusion for Remote Sensing Applications

Mena,

Arenas,

Nuske

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The advances in remote sensing technologies have boosted applications for Earth observation. These technologies provide multiple observations or views with different levels of information. They might contain static or temporary views with different levels of resolution, in addition to having different types and amounts of noise due to sensor calibration or deterioration. A great variety of deep learning models have been applied to fuse the information from these multiple views, known as deep multiview or multi-modal fusion learning. However, the approaches in the literature vary greatly since different terminology is used to refer to similar concepts or different illustrations are given to similar techniques. This article gathers works on multi-view fusion for Earth observation by focusing on the common practices and approaches used in the literature. We summarize and structure insights from several different publications concentrating on unifying points and ideas. In this manuscript, we provide a harmonized terminology while at the same time mentioning the various alternative terms that are used in literature. The topics covered by the works reviewed focus on supervised learning with the use of neural network models. We hope this review, with a long list of recent references, can support future research and lead to a unified advance in the area.

show abstract