A coarse-to-fine boundary refinement network for building footprint extraction from remote sensing imagery

Guo, Haonan; Du, Bo; Zhang, Liangpei; Su, Xin

doi:10.1016/j.isprsjprs.2021.11.005

Cited by 94 publications

(50 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the Massachusetts building dataset, WHU building dataset and Inria Aerial Image Labeling dataset. The selected methods include convolutional networks, such as U-Net [21], Deeplabv3+ [88], SRI-Net [16], DS-Net [49], BRRNet [20], SiU-Net [18], CU-Net [19], EU-Net [89], DE-Net [90], MA-FCN [48], MANet [53], MAP-Net [27], Bias-UNet [57], CBRNet [35], and ViT-based networks like SwinUperNet [34], Sparse Token Transformer (STT) [79], MSST-Net [80], BANet [72], DC-Swin [69].…”

Section: B Comparison Of State-of-the-art Methodsmentioning

confidence: 99%

“…Improving the accuracy of building boundaries is vital for high-precision building extraction [35,[82][83][84]. Thus, we introduce the boundary supervision technology and adopt a joint loss to train the BuildFormer.…”

Section: E Loss Functionmentioning

confidence: 99%

“…We use the intersection over union (IoU), F1 score, precision and recall to evaluate the performance of models. These metrics are widely used in the field of building extraction [27,35], which can be defined as follows:…”

Section: B Evaluation Metricsmentioning

confidence: 99%

“…Even though the Swin Transformer adopts the hierarchical structure and designs a window-based multihead self-attention mechanism to improve efficiency, its complexity still increases quadratically along with the increasing size of the window [34]. Furthermore, ViTs mainly focus on capturing the global context while ignoring preserving the spatial-detailed context, but spatial details are also essential for fine-grained building segmentation in fine-resolution remote sensing images [35].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Building Extraction With Vision Transformer

Wang

Fang

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

As an important carrier of human productive activities, the extraction of buildings is not only essential for urban dynamic monitoring but also necessary for suburban construction inspection. Nowadays, accurate building extraction from remote sensing images remains a challenge due to the complex background and diverse appearances of buildings. The convolutional neural network (CNN) based building extraction methods, although increased the accuracy significantly, are criticized for their inability for modelling global dependencies. Thus, this paper applies the Vision Transformer for building extraction. However, the actual utilization of the Vision Transformer often comes with two limitations. First, the Vision Transformer requires more GPU memory and computational costs compared to CNNs. This limitation is further magnified when encountering large-sized inputs like fine-resolution remote sensing images. Second, spatial details are not sufficiently preserved during the feature extraction of the Vision Transformer, resulting in the inability for fine-grained building segmentation. To handle these issues, we propose a novel Vision Transformer (BuildFormer), with a dual-path structure. Specifically, we design a spatial-detailed context path to encode rich spatial details and a global context path to capture global dependencies. Besides, we develop a window-based linear multi-head self-attention to make the complexity of the multi-head self-attention linear with the window size, which strengthens the global context extraction by using large windows and greatly improves the potential of the Vision Transformer in processing large-sized remote sensing images. The proposed method yields state-of-the-art performance (75.74% IoU) on the Massachusetts building dataset. Code will be available.

show abstract

Section: B Comparison Of State-of-the-art Methodsmentioning

confidence: 99%

Section: E Loss Functionmentioning

confidence: 99%

Section: B Evaluation Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Building Extraction With Vision Transformer

Wang

Fang

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Most of these newly developed models are benchmarked on publicly available datasets, and demonstrate outperformance, while some application-oritented studies also created their own datasets [14], [31]. For example, by exploiting low and high level features and designing a boundary refinement module, the IoU metric has been improved from 90.86% to 91.4% when tested on the WHU dataset [32]. These works can provide interesting methodological insights, even though there is usually only one percentage point rise in accuracy.…”

mentioning

confidence: 99%

Transferring Transformer-Based Models for Cross-Area Building Extraction From Remote Sensing Images

Qiu

Guo

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Extracting buildings from remote sensing (RS)images is an important task with a variety of applications. 2 Considerable attention has focused on achieving new state-of-3 the-art (SOTA) accuracy with more and more advanced deep 4 learning (DL) models. However, the developed models still hardly 5 generalize across geographical areas, hindering the practical use 6 of SOTA approaches. To attack this problem, we established 7 a baseline for model cross-area generalization ability using 8 available datasets for building extraction (BE). In addition to 9 two popular FCN-based models, we first adapted two novel 10 transformer-based models, Swin Transformer and SegFormer, 11 which are all able to output SOTA accuracy with no big difference 12 when tested within one area. However, experimental results show 13 that all models fail to generalize to a different area. We then 14 propose to fine-tune pre-trained models from one area on a small 15 subset of an unseen area, the effectiveness of which depends on 16 the model choice and the data size for tuning. By jointly taking 17 advantage of the transfer learning idea and the multiscale feature 18 learning ability of SegFormer, a distinct improvement has been 19 achieved compared to results from Swin Transformer and FCN-20 based models trained on the same amount of data. Commonly 21 used metric, Intersection over Union (IoU), can be increased 22 from 38.97% to 70.86%, and from 48.36% to 74.51%, when 23 using 10% and 30% subset of the targeting area, respectively.

show abstract

Efficiency analysis of ITN loss function for deep semantic building segmentation

Omati,

Mahmoudi

2024

Earth Sci Inform

View full text Add to dashboard Cite

A coarse-to-fine boundary refinement network for building footprint extraction from remote sensing imagery

Cited by 94 publications

References 42 publications

Building Extraction With Vision Transformer

Building Extraction With Vision Transformer

Transferring Transformer-Based Models for Cross-Area Building Extraction From Remote Sensing Images

Efficiency analysis of ITN loss function for deep semantic building segmentation

Contact Info

Product

Resources

About