Scale-Aware Neural Network for Semantic Segmentation of Multi-Resolution Remote Sensing Images

Abstract

Assigning geospatial objects with specific categories at the pixel level is a fundamental task in remote sensing image analysis. Along with the rapid development of sensor technologies, remotely sensed images can be captured at multiple spatial resolutions (MSR) with information content manifested at different scales. Extracting information from these MSR images represents huge opportunities for enhanced feature representation and characterisation. However, MSR images suffer from two critical issues: (1) increased scale variation of geo-objects and (2) loss of detailed information at coarse spatial resolutions. To bridge these gaps, in this paper, we propose a novel scale-aware neural network (SaNet) for the semantic segmentation of MSR remotely sensed imagery. SaNet deploys a densely connected feature network (DCFFM) module to capture high-quality multi-scale context, such that the scale variation is handled properly and the quality of segmentation is increased for both large and small objects. A spatial feature recalibration (SFRM) module was further incorporated into the network to learn intact semantic content with enhanced spatial relationships, where the negative effects of information loss are removed. The combination of DCFFM and SFRM allows SaNet to learn scale-aware feature representation, which outperforms the existing multi-scale feature representation. Extensive experiments on three semantic segmentation datasets demonstrated the effectiveness of the proposed SaNet in cross-resolution segmentation.

Few-Shot Rotation-Invariant Aerial Image Semantic Segmentation

“…Directly from an architecture perspective, Mitton et al [48] designs the rotation equivariant convolutional neural network model to efficiently perform the deforestation segmentation. Moreover, through the designed densely connected feature network and the spatial feature recalibration module, the SaNet [49] successfully extracts the scale-aware feature representation to enhance the performance of aerial image semantic segmentation. Similarly, Sci-Net [50] leverages UNet hierarchical representation and dense atrous spatial pyramid pooling to achieve scaleinvariant building segmentation from aerial imagery.…”

Context-Driven Feature-Focusing Network for Semantic Segmentation of High-Resolution Remote Sensing Images

“…ResNet's [40] complex residual modules give it a strong feature extraction capability. Numerous studies [2,13,41] have shown that a pre-trained ResNet backbone performs well in segmentation tasks. ResNet is used as an encoder in the suggested framework, albeit without fully connected (FC) layers.…”

“…The semantic segmentation of HRRSIs is to categorize each pixel in remote sensing (RS) images. With the rapid development of RS acquisition technology, semantic segmentation applications of HRRSIs have become more widespread [1][2][3]. The semantic segmentation's robustness and the generalization ability of RS images are crucial for applications such as mapping, land use, earth observation, and land cover.…”

Gaussian Dynamic Convolution for Semantic Segmentation in Remote Sensing Images

“…The framework consists of a double branch, partly used to suppress the large-scale background and partly used to activate the features of small objects. Some recent methods [5,27,28] try to incorporate modules that are effective in the field of general segmentation into remote sensing image segmentation networks, such as the well-known transformer or attention mechanisms, which are effective in improving the accuracy of the networks to some extent. However, these methods mainly target special application scenarios and are not effective in solving problems of semantic segmentation for high-resolution remote sensing images, such as multi-scale variation of objects and loss of foreground details.…”