ESPC_NASUnet: An End-to-End Super-Resolution Semantic Segmentation Network for Mapping Buildings From Remote Sensing Images

Abstract

Higher resolution building mapping from lower resolution remote sensing images is in great demand due to the lack of higher resolution data access, especially in the context of disaster assessment. High resolution building layout map is crucial for emergency rescue after the disaster. The emergency response time would be reduced if detailed building footprints were delineated from more easily available low-resolution data. To achieve this goal, we propose a super-resolution semantic segmentation network called ESPC NASUnet, which consists of a feature super-resolution module and a semantic segmentation module. To the best of our knowledge, this is the first work to systematically explore a deep learning-based approach to generate semantic maps with higher spatial resolution from lower spatial resolution remote sensing images in an end-toend fashion. The experimental results for two datasets suggest that the proposed network is the best among four different end-to-end architectures in terms of both pixel-level metrics and object-level metrics. In terms of pixel-level F1-score, the improvements are greater than 0.068 and 0.055. Regarding the object-level F1-score, the disparities between ESPC NASUnet and other end-to-end methods are more than 0.083 and 0.161 in the two datasets, respectively. Compared with stage-wise methods, our end-to-end network is less impacted by low-resolution input images. Finally, the proposed network produces building semantic maps comparable to those generated by semantic segmentation networks trained with high-resolution images and the ground truth utilizing the two datasets.

CBRA: The first multi-annual (2016–2021) and high-resolution (2.5 m) building rooftop area dataset in China derived with Super-resolution Segmentation from Sentinel-2 imagery

“…Utilizing the SR methods could transfer the low-resolution image to the high-resolution, thus expanding the cheaper satellite with a coarser resolution to the application demanding high-resolution data (Shermeyer and van Etten, 2019). He et al (2021) utilize lowresolution and high-resolution image pairs to learn the SR model and map the low-resolution image to the high-resolution, while Xu et al (2021) apply only the high-resolution label, achieving strong performance in the downstream high-resolution tasks. The SS, which is a pixel-wise classification task, also has a lot of applications in earth observation, such as land use mapping (Zhu et al, 2022a) and disaster detection (Munawar et al, 2022).…”

DARN: Distance Attention Residual Network for Lightweight Remote-Sensing Image Superresolution

“…The strict requirement of lightweight models for practical tasks has prompted researchers to focus on the development of more efficient SR models [1], [19], [20], [21], [22], [23], [24], [30], [31], [32]. The IDN proposed in [19] can separately extract the result of feature segmentation through two channels.…”

Multi-channel recurrent attention network for building extraction from high resolution remote sensing images

“…To evaluate the performance of proposed MCANet, we compare MCANet with the MCFCNN [27], BRR-Net [28], Building-Net [29], NAS-UNet [30] and Siamese-UNet [31] with the same datasets and experimental settings. We chose the images of Austin, Eastern Tyrol, Bellingham, and San Francisco regions as the testing samples.…”

“…Li et al [29] developed a novel GAN based building extraction framework, which jointly trained a generative network and a discriminator network for the robust extraction of building region. Xu et al [30] proposed an end-to-end building extraction framework NAS-UNet, which used feature super-resolution module to obtain multi-scale semantic features, and then used semantic segmentation module to obtain building extraction results of different regions. Ji et al [31] proposed a novel building extraction method Siamese-UNet, it used transfer learning for model training, so that the model had strong feature learning ability, and the multi-scale prediction method was used to obtain refined building extraction results.…”