Boundary-Aware Multitask Learning for Remote Sensing Imagery

Wang, Yufeng; Ding, Wenrui; Zhang, Ruiqian; Li, Hongguang

doi:10.1109/jstars.2020.3043442

Cited by 31 publications

(4 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, rather than adapting a simple binary label scheme (i.e., buildings vs. nonbuildings), we further propose using signed-distance labels to stress the boundary and structure information that we would like to exploit for aligning distribution during the adversarial learning process. This is motivated by the fact that boundaries information plays an important role in defining better building extraction results [67], [68]. We also have observed that, as pointed out by many works [3], [4], [5] focusing on this goal, many pretrained models fail to capture building boundaries when there are no new training samples available, while only some portion of building areas might be identified.…”

Section: A Entropy-based Boundary Adversarial Learningmentioning

confidence: 88%

Boundary-Aware Adversarial Learning Domain Adaption and Active Learning for Cross-Sensor Building Extraction

Yang,

Makkar,

Laverdiere

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Use of convolutional neural networks (CNNs) for building extraction from remote sensing images has been studied widely, and many public datasets have been made available for accelerating development of these CNN models. Yet adapting pretrained models at scale in real-world scenarios remains a challenging task. The main barrier is that certain new labels are still needed to compensate for domain shifting between the labeled data and new images that potentially cover new geographic locations or that are from a different sensor. In this paper, we propose to add informative labeled samples from a new image pool under the paradigm of active learning. To select the most useful samples based on model uncertainty, we first tackle the problem of uncalibrated uncertainty estimation due to distribution shifting by adapting feature extractors with boundary-based adversarial learning. Calibrated uncertainty is used as the query criterion in the active learning process where the most uncertain samples are selected for annotation and included for model retraining. The proposed workflow was tested with three data pairs in which each workflow represents a scenario often encountered in real-world applications, including adapting pretrained models to new images collected with different sensors or to new geographic areas where appearances and types of buildings are very different. Compared to several baselines, including random sampling, temperature scaling (a well-known uncertainty calibration technique), different query strategies, and active domain adaptation methods, the proposed workflow shows that strategically querying a smaller set of samples for labeling achieves comparable or better building extraction performance. The proposed method reduces the number of labeled samples required to achieve sufficient model accuracy, thus also significantly reducing hundreds of person-hours for labeled data creation. In addition. we include few considerations when deploying this workflow in a GPU cluster can be easily adapted to achieve operational building extraction model re-training .

show abstract

Section: A Entropy-based Boundary Adversarial Learningmentioning

confidence: 88%

Boundary-Aware Adversarial Learning Domain Adaption and Active Learning for Cross-Sensor Building Extraction

Yang,

Makkar,

Laverdiere

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…For the Potsdam dataset, a comparison of our proposed method, HeightFormer, was made with IMG2DSM, Amirkolaee et al [53], BAMTL [59], DepthsFormer [52], and Bins-Former [51]. The properties of the Potsdam dataset are presented in Table 5, while the corresponding visualized results are illustrated in Figure 9.…”

Section: Quantitative and Qualitative Analysis On Potsdammentioning

confidence: 99%

HeightFormer: A Multilevel Interaction and Image-Adaptive Classification–Regression Network for Monocular Height Estimation with Aerial Images

Chen,

Zhang,

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Height estimation has long been a pivotal topic within measurement and remote sensing disciplines, with monocular height estimation offering wide-ranging data sources and convenient deployment. This paper addresses the existing challenges in monocular height estimation methods, namely the difficulty in simultaneously achieving high-quality instance-level height and edge reconstruction, along with high computational complexity. This paper presents a comprehensive solution for monocular height estimation in remote sensing, termed HeightFormer, combining multilevel interactions and image-adaptive classification–regression. It features the Multilevel Interaction Backbone (MIB) and Image-adaptive Classification–regression Height Generator (ICG). MIB supplements the fixed sample grid in the CNN of the conventional backbone network with tokens of different interaction ranges. It is complemented by a pixel-, patch-, and feature map-level hierarchical interaction mechanism, designed to relay spatial geometry information across different scales and introducing a global receptive field to enhance the quality of instance-level height estimation. The ICG dynamically generates height partition for each image and reframes the traditional regression task, using a refinement from coarse to fine classification–regression that significantly mitigates the innate ill-posedness issue and drastically improves edge sharpness. Finally, the study conducts experimental validations on the Vaihingen and Potsdam datasets, with results demonstrating that our proposed method surpasses existing techniques.

show abstract

“…2(b). MTL has been widely utilized in various computer vision applications, including semantic segmentation [15], [16], [17], [18], [19], [20], [21] and object detection [23].…”

Section: A Multitask Learningmentioning

confidence: 99%

“…Inspired by this finding, we designed a reverse attention module (RAM) to learn features of both inapparent aquaculture areas and obvious aquaculture areas by suppressing features of seawater. In addition, considerable research [15], [16], [17], [18], [19], [20], [21] has demonstrated that dual-stream networks combining edge detection and semantic segmentation effectively utilize the boundary information of objects and significantly improve the boundaries of segmentation results. To obtain an accurate boundary of the aquaculture area, we designed a boundary attention module (BAM) using global semantic information to avoid the interference of nonboundary information on boundary extraction.…”

Section: Introductionmentioning

confidence: 99%

Reverse Attention Dual-Stream Network for Extracting Laver Aquaculture Areas From GF-1 Remote Sensing Images

Cui

Zhao

Yang

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Extracting laver aquaculture areas from remote sensing images is very important for laver aquaculture monitoring and scientific management. However, due to the large differences in spectral features of laver aquaculture areas caused by factors such as different growth stages and harvesting conditions, traditional machine learning and deep learning methods face great challenges in achieving accurate and complete extraction of raft laver aquaculture areas. In this article, a reverse attention dual-stream network (RADNet) is proposed for the extraction of laver aquaculture areas with weak spectral responses by comprehensively considering both the aquaculture boundary and surrounding sea background information. RADNet consists of a boundary stream and a segmentation stream. Considering the weaker spectral responses of certain laver aquaculture areas, we introduce a reverse attention module in the segmentation stream to amplify the weaker responses of inapparent laver aquaculture areas. To suppress the response of nonboundary details in the boundary stream, we design a boundary attention module, which is guided by high-level semantics from the segmentation stream. The structural information of the laver aquaculture area learned from the boundary stream will be fed back to the segmentation stream through a specially designed boundary guidance module. The study is conducted in Haizhou Bay, China, and is verified using a self-labeled GF-1 multispectral dataset. The experimental results show that RADNet model performs better in extracting inapparent laver aquaculture areas compared to SOTA models.

show abstract

Boundary-Aware Multitask Learning for Remote Sensing Imagery

Cited by 31 publications

References 70 publications

Boundary-Aware Adversarial Learning Domain Adaption and Active Learning for Cross-Sensor Building Extraction

Boundary-Aware Adversarial Learning Domain Adaption and Active Learning for Cross-Sensor Building Extraction

HeightFormer: A Multilevel Interaction and Image-Adaptive Classification–Regression Network for Monocular Height Estimation with Aerial Images

Reverse Attention Dual-Stream Network for Extracting Laver Aquaculture Areas From GF-1 Remote Sensing Images

Contact Info

Product

Resources

About