Improved Pseudomasks Generation for Weakly Supervised Building Extraction From High-Resolution Remote Sensing Imagery

Abstract

Benefiting from free labeling pixel-level samples, weakly supervised semantic segmentation (WSSS) is making progress in automatically extracting building from high-resolution (HR) remote sensing (RS) imagery. For WSSS methods, generating high-quality pseudo-masks is crucial for accurate building extraction. To improve the performance of generating pseudo-masks by using imagelevel labels, this paper proposes a weakly supervised building extraction method by combining adversarial climbing and gated convolution (ACGC). The proposed method optimizes class activation maps (CAMs) by using adversarial climbing strategy, generates accurate class boundary maps (CBMs) by introducing a gated convolution module (GCM), and further refines building pseudo-masks by fusing pairing semantic affinities and CAMs with a random walk strategy. Experimental results on three datasets-two ISPRS datasets and a self-annotated dataset-demonstrate that the proposed approach outperformed SOTA WSSS methods, leading to improvement of building extraction from HR RS imagery.This study provides a new approach for optimizing pseudomasks generation, and a methodological reference for the applications of weakly supervised on RS images.

Deep-Learning-Based Semantic Segmentation of Remote Sensing Images: A Survey

“…Most existing methods of WSSS follow a two-step pipeline, i.e., generating pseudo labels and training segmentation models. This section introduces the related research according to the categories of weakly-supervised labels, which are image-level labels [172], [173], [174], [175], bounding boxes [176], [177], point annotation [178], [179], [180], and scribble annotation [180], [181]. Table III compares the characteristics of some representative WSSS methods.…”

“…Li et al [153] designed a confidence area selection module and a low-to-high loss function to obtain reliable supervision information from the coarse labels. Fang et al [175] used the adversarial climbing strategy to optimize CAMs. Zeng et al [182] proposed a framework directly transferring the scene classification model to perform semantic segmentation.…”

A Siamese Network Combining Multiscale Joint Supervision and Improved Consistency Regularization for Weakly Supervised Building Change Detection

“…Thanks to its progressive development in natural pictures [48], [49], [50], [51], [52], [53], [57], [58], [59], [60], WSSS has been gradually introduced into VHR remote sensing image-based classification, such as building detection, landslide extraction, infected-tree recognition, and so on [61], [62], [63], [64], [65], [66], [67]. For example, Qiao et al [65] used a simple weakly supervised deep-learning method for individual red-attack tree detection.…”

“…Li et al [64] used a general CNN to produce CAMs and pseudomask labels, then an SS network considering CRF loss and classification loss is used to improve the performance of building extraction. Besides, Fang et al [63] utilized the adversarial climbing and gated convolution strategy to generate class boundary maps and further refined building pseudomasks by fusing pairing semantic affinities and CAMs using the random walk. Furthermore, Yan et al [30] proposed a WSSS method combining multiscale generation and super-pixel refinement to improve CAM quality for building detection.…”

Extraction of building from remote sensing imagery base on multi-attention L-CAFSFM and MFFM

“…Factors such as the size, shape, texture difference, cloud occlusion, surface OPEN ACCESS EDITED BY Bahareh Kalantar, Riken, Japan material reflection, and shadow of buildings in remote sensing images can reduce the accuracy of building detection. Improving the accuracy of building detection has essential application value and practical significance for urban 3D modeling, map updating, disaster assessment, etc (Bauchet, et al, 2021;Chen and Sun, 2022;Fang, et al, 2022;Hou, et al, 2022;Yang, et al, 2022).…”