A Semantic Segmentation Method for Remote Sensing Images Based on the Swin Transformer Fusion Gabor Filter

Abstract

Semantic segmentation of remote sensing images is increasingly important in urban planning, autonomous driving, disaster monitoring, and land cover classification. With the development of high-resolution remote sensing satellite technology, multilevel, large-scale, and high-precision segmentation has become the focus of current research. High-resolution remote sensing images have high intraclass diversity and low interclass separability, which pose challenges to the precision of the detailed representation of multiscale information. In this paper, a semantic segmentation method for remote sensing images based on Swin Transformer fusion with a Gabor filter is proposed. First, a Swin Transformer is used as the backbone network to extract image information at different levels. Then, the texture and edge features of the input image are extracted with a Gabor filter, and the multilevel features are merged by introducing a feature aggregation module (FAM) and an attentional embedding module (AEM). Finally, the segmentation result is optimized with the fully connected conditional random field (FC-CRF). Our proposed method, called Swin-S-GF, its mean Intersection over Union (mIoU) scored 80.14%, 66.50%, and 70.61% on the large-scale classification set, the fine land-cover classification set, and the "AI + Remote Sensing imaging dataset" (AI+RS dataset), respectively. Compared with DeepLabV3, mIoU increased by 0.67%, 3.43%, and 3.80%, respectively. Therefore, we believe that this model provides a good tool for the semantic segmentation of high-precision remote sensing images.

Deep learning-based semantic segmentation of remote sensing images: a review

“…Because self-attention can calculate within the window, its computational complexity increases linearly with the size of the picture rather than quadratic. Therefore, in RS semantic segmentation, it is widely used (Panboonyuen et al, 2021;Xu et al, 2021;Feng et al, 2022;Gu et al, 2022;Liu Y. et al, 2022;Meng et al, 2022;Xu Y. et al, 2022). ST first used the module to segment the data into many non-overlapping different patches.…”

A Robust Visual SLAM Method for Additive Manufacturing of Vehicular Parts Under Dynamic Scenes

“…The Detection Transformer (DEtection TRansformer, DE-TR) [40], [41] with an ensemble global loss that makes predictions through bilateral match and a classical encoderdecoder architecture, which containing three components: a CNN based backbone to extracte feature representations, a Transformer pretraining model to enhance features, and a simple feedforward network (FFN) for performing the object detection prediction.The detail structure is shown as Figure 3. Starting from an initial image x img ∈ R 3×H0×W0 (3 color channels, To batch the input images together with sufficient 0 padding to have the same dimension (H 0 ,W 0 ) as the largest image in same batch), a convolutional network then to generate a activation map f ∈ R C×H×W with lower resolution.…”

“…For a decoder, it transforms N embeddings with size d by multi-head attention mechanism. The authors in [40] adopted an auto-regressive model to predict one element of the output sequence at once. Because the decoder is also permutationindependent (order-independent), hence thedifferent results will be produced according to N input embeddings.…”

GODANet: an object detection model for remote sensing images fusing contextual information and dynamic convolution

“…Remote sensing image 1 – 4 object detection refers to the automatic identification and localization of target objects of interest in remote sensing images. It is a fundamental and crucial task in optical remote sensing image processing.…”