Hybridizing Cross-Level Contextual and Attentive Representations for Remote Sensing Imagery Semantic Segmentation

Abstract

Semantic segmentation of remote sensing imagery is a fundamental task in intelligent interpretation. Since deep convolutional neural networks (DCNNs) performed considerable insight in learning implicit representations from data, numerous works in recent years have transferred the DCNN-based model to remote sensing data analysis. However, the wide-range observation areas, complex and diverse objects and illumination and imaging angle influence the pixels easily confused, leading to undesirable results. Therefore, a remote sensing imagery semantic segmentation neural network, named HCANet, is proposed to generate representative and discriminative representations for dense predictions. HCANet hybridizes cross-level contextual and attentive representations to emphasize the distinguishability of learned features. First of all, a cross-level contextual representation module (CCRM) is devised to exploit and harness the superpixel contextual information. Moreover, a hybrid representation enhancement module (HREM) is designed to fuse cross-level contextual and self-attentive representations flexibly. Furthermore, the decoder incorporates DUpsampling operation to boost the efficiency losslessly. The extensive experiments are implemented on the Vaihingen and Potsdam benchmarks. In addition, the results indicate that HCANet achieves excellent performance on overall accuracy and mean intersection over union. In addition, the ablation study further verifies the superiority of CCRM.

A Synergistical Attention Model for Semantic Segmentation of Remote Sensing Images

“…AFNet [61] designs a multi-path encoder structure, multi-path attentionfused module and refinement attention-fused block for semantic segmentation of very high resolution remote sensing images HMANet [38] hybridizes three-level representations to augment learnt features, including space, channel and category. Similarly, Li et al proposed HCANet [62] that extracts and aggregates cross-level contextual information from pixels, superpixels, and global space. In addition, an adaptive fusion network was designed by Li et al [63], enhancing the feature points from either low-level or high-level feature maps with the constructed attention modules.…”

MSAFNet: Multiscale Successive Attention Fusion Network for Water Body Extraction of Remote Sensing Images

“…The design of the attention mechanism is inspired by the human visual system. In application to the semantic segmentation of natural images, various attention mechanisms are introduced into models, which have achieved great performance in mining visual features [47,48]. In SENet, SE block was proposed to enhance the correlation of interchannel features using a global averaging pooling operation in the channel dimension [49].…”

FPIseg: Iterative segmentation network based on feature pyramid for few‐shot segmentation

“…In addition, the model's receptive field has a significant impact on its performance, and thus dilated convolution [14,15,17,34] and global pooling [35] have been introduced into the field of semantic segmentation, achieving significant performance improvement. At the same time, some models [13,32,36,37] leveraged attention mechanisms [16,31,[38][39][40][41][42] to capture long-distance dependency for semantic segmentation, which reach state-of-the-art performance. However, all of the aforementioned methods heavily rely on large-scale, accurate pixel-level labelled dataset.…”