Maskformer with Improved Encoder-Decoder Module for Semantic Segmentation of Fine-Resolution Remote Sensing Images

Abstract

In 2021, the Transformer based models have demonstrated extraordinary achievement in the field of computer vision. Among which, Maskformer, a Transformer based model adopting the mask classification method, is an outstanding model in both semantic segmentation and instance segmentation. Considering the specific characteristics of semantic segmentation of remote sensing images (RSIs), we design CADA-MaskFormer(a Mask classification-based model with Cross-shaped window self-Attention and Densely connected feature Aggregation) based on Maskformer by improving its encoder and pixel decoder. Concretely, the mask classification that generates one or even more masks for specific category to perform the elaborate segmentation is especially suitable for handling the characteristic of large within-class and small between-class variance of RSIs. Furthermore, we apply the Cross-Shaped Window self-attention mechanism to model the long-range context information contained in RSIs at maximum extent without the increasing of computational complexity. In addition, the Densely Connected Feature Aggregation Module (DCFAM) is used as the pixel decoder to incorporate multi-level feature maps from the encoder to get a finer semantic segmentation map. Extensive experiments conducted on two remotely sensed semantic segmentation datasets Potsdam and Vaihingen achieves 91.88% and 91.01% in OA index respectively, outperforming most of competitive models designed for RSIs. The code is available from https://github.com/lqwrl542293/JL-Yang_ CV/tree/master/CADA_Maskformer

Panoptic SwiftNet: Pyramidal Fusion for Real-Time Panoptic Segmentation

“…Semantic segmentation of remote sensing images is a popular and active line of research [28][29][30][31]. On the other hand, only a few papers consider panoptic segmentation of remote sensing images [6,32,33].…”

Potential development of dairy production in Sichuan Province, China

“…For instance, DETR [20] has demonstrated superior performance for various vision tasks [41,7], owing to the Transformer's inherent capability to model long-range dependencies which is beneficial for handling complex scenes. More recently, MaskFormer [22], a simple and efficient Transformer-based method for instance segmentation, showed remarkable results in both 2D and 3D instance segmentation tasks. However, the extension of Transformers to 3D instance segmentation isn't straightforward and presents unique challenges.…”