MACU-Net for Semantic Segmentation of Fine-Resolution Remotely Sensed Images

Abstract

Semantic segmentation of remotely sensed images plays an important role in land resource management, yield estimation, and economic assessment. U-Net, a deep encoderdecoder architecture, has been used frequently for image segmentation with high accuracy. In this Letter, we incorporate multi-scale features generated by different layers of U-Net and design a multi-scale skip connected and asymmetric-convolutionbased U-Net (MACU-Net), for segmentation using fine-resolution remotely sensed images. Our design has the following advantages: (1) The multi-scale skip connections combine and realign semantic features contained in both low-level and high-level feature maps; (2) the asymmetric convolution block strengthens the feature representation and feature extraction capability of a standard convolution layer. Experiments conducted on two remotely sensed datasets captured by different satellite sensors demonstrate that the proposed MACU-Net transcends the U-Net, U-NetPPL, U-Net 3+, amongst other benchmark approaches.

Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images

“…In DilatedFCNs [12,14,[37][38][39][40]61], dilate or atrous convolutions are harnessed to retain the receptive field-of-view, and a multi-scale context module is utilized to cope with highlevel feature maps. Alternatively, EncoderDecoders [34,35,[62][63][64][65][66][67][68] utilize an encoder to capture multi-level feature maps, which are then incorporated into the final prediction using a decoder.…”

“…EncoderDecoder Skip connections are employed to integrate the high-level features generated by the decoder and the low-level features generated by the corresponding encoder, which are the essential structure of U-Net [34]. In the recent literature [62][63][64], the plain skip connections in U-Net are substituted by more subtle and elaborate skip connections which reduce the semantic gap between the encoder and decoder. Meanwhile, the structural development based on residual connections is also a promising direction [35,[65][66][67][68].…”

Scale-Aware Neural Network for Semantic Segmentation of Multi-Resolution Remote Sensing Images

“…To test the cross-resolution generalization capability of the proposed SaNet, we selected various competitive methods for comparison, including multi-scale feature aggregation models such as the feature pyramid network (FPN) [37] and pyramid scene network (PSPNet) [39], the multi-view context aggregation method Deeplabv3+ [41] and the criss-cross attention network (CCNet) [46], as well as specially designed models for semantic labelling of remotely sensed images, such as relational context-aware fully convolutional network (S-RA-FCN) [47], the dense dilated convolutions merging network (DDCM-Net) [43], edge-aware neural network (EaNet) [44], MACUNet [48] and MARe-sUNet [49]. Besides, ablation studies were conducted with the following model design:…”

Deep Neural Architectures for Medical Image Semantic Segmentation: Review

“…It restores lossy signals with object detection without additional complexity found in a two-stage variant. An asymmetric U-Net-based convolutional block is used in [32] to define multi-scale architecture with skipconnections. It fuses low and high-end feature maps with different scales and strengthens the representational capacity of convolutional blocks.…”