Research on cloud removal based on fusing Multi-temporal Remote Sensing Images

Abstract

Optical satellite images are a critical data source; however, cloud cover often compromises their quality, hindering image applications and analysis. Consequently, effectively removing clouds from optical satellite images has emerged as a prominent research direction. While recent advancements in cloud removal primarily rely on generative adversarial networks, which may yield suboptimal image quality, diffusion models have demonstrated remarkable success in diverse image-generation tasks, showcasing their potential in addressing this challenge. This paper presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery. Specifically, we introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output. Moreover, we propose a novel and efficient time and condition fusion block within the cloud removal model to accurately simulate the correspondence between the appearance in the conditional image and the target image at a low computational cost. Extensive experimental evaluations on two commonly used benchmark datasets demonstrate that DiffCR consistently achieves state-of-the-art performance on all metrics, with parameter and computational complexities amounting to only 5.1% and 5.4%, respectively, of those previous best methods. The source code, pre-trained models, and all the experimental results will be publicly available at https://github.com/XavierJiezou/DiffCR upon the paper's acceptance of this work.

A Review of Image Inpainting Methods Based on Deep Learning

“…The demand for detailed information on the Earth's surface in remote sensing image analysis is expanding. Zhao et al [2,3] attempted to fill areas in remote sensing images that were obscured by clouds. Dong et al [4,5] restored and filled areas in remote sensing images where mountains and buildings were obscured.…”

“…Image processing technology is a crucial research direction of deep learning in the field of computer vision. Image inpainting, as a significant branch of image processing, has been widely applied in various industries, such as medical image processing [1], satellite remote sensing image processing [2][3][4][5], and image processing for film and artistic creation [6][7][8][9]. During image processing, factors such as a poor environment, excessive noise, unfavorable shooting conditions, and unstable network communications often result in image blurring and loss.…”

Characteristics in the Spatiotemporal Variation and Periodic Evolution of Groundwater in the Xining Area of China, Eastern Qinghai–Tibet Plateau

“…The attention mechanism plays a crucial role in the field of artificial intelligence, notably in areas such as computer vision [9,10,11,12], natural language processing, and speech processing [13,14]. SENet [15] introduced the mechanism of channel attention.…”