A Hybrid Algorithm with Swin Transformer and Convolution for Cloud Detection

Abstract

Cloud detection is critical in remote sensing image processing, and convolutional neural networks (CNNs) have significantly advanced this field. However, traditional CNNs primarily focus on extracting local features, which can be challenging for cloud detection due to the variability in the size, shape, and boundaries of clouds. To address this limitation, we propose a hybrid Swin transformer–CNN cloud detection (STCCD) network that combines the strengths of both architectures. The STCCD network employs a novel dual-stream encoder that integrates Swin transformer and CNN blocks. Swin transformers can capture global context features more effectively than traditional CNNs, while CNNs excel at extracting local features. The two streams are fused via a fusion coupling module (FCM) to produce a richer representation of the input image. To further enhance the network’s ability in extracting cloud features, we incorporate a feature fusion module based on the attention mechanism (FFMAM) and an aggregation multiscale feature module (AMSFM). The FFMAM selectively merges global and local features based on their importance, while the AMSFM aggregates feature maps from different spatial scales to obtain a more comprehensive representation of the cloud mask. We evaluated the STCCD network on three challenging cloud detection datasets (GF1-WHU, SPARCS, and AIR-CD), as well as the L8-Biome dataset to assess its generalization capability. The results show that the STCCD network outperformed other state-of-the-art methods on all datasets. Notably, the STCCD model, trained on only four bands (visible and near-infrared) of the GF1-WHU dataset, outperformed the official Landsat-8 Fmask algorithm in the L8-Biome dataset, which uses additional bands (shortwave infrared, cirrus, and thermal).

Spatial–Temporal Approach and Dataset for Enhancing Cloud Detection in Sentinel-2 Imagery: A Case Study in China

“…Recently, deep learning methods have been increasingly used in cloud detection, and they have achieved outstanding performance [24][25][26][27]. Deep-learning-based algorithms automatically learn spatial and semantic features directly from training data, avoiding the need for manual feature selection and reducing the reliance on subjective experience.…”

Radiation Feature Fusion Dual-Attention Cloud Segmentation Network

“…Employing AI technologies like deep learning to address meteorological issues has been a major research direction in recent years. Recent advancements in deep learning have provided new solutions for cloud segmentation in remote sensing imagery [21][22][23][24]. Deep neural networks [25] (DNNs) have been widely applied in fields like image recognition, object detection, and segmentation.…”

Methodology for Severe Convective Cloud Identification Using Lightweight Neural Network Model Ensembling

“…Cloud segmentation based on deep learning is a long-standing and ongoing area of research in the field of remote sensing. This includes general cloud segmentation without differentiating cloud types [5,11,14,[17][18][19][20][21][22], as well as segmentation of severe convective clouds [2,4,6,8,15,[23][24][25][26][27], which continue to be actively studied, underscoring the enduring significance of this problem. In practical applications, particularly where real-time processing of vast datasets is required, lightweight neural networks have garnered attention due to their lower computational demands and rapid processing capabilities.…”