Meta-Knowledge Guided Weakly Supervised Instance Segmentation for Optical and SAR Image Interpretation

Abstract

The interpretation of optical and synthetic aperture radar (SAR) images in remote sensing is general for many tasks, such as environmental monitoring, marine management, and resource planning. Instance segmentation of optical and SAR images, which can simultaneously provide instance-level localization and pixel-level classification of objects of interest, is a crucial and challenging task in image interpretation. Considering that most current methods for instance segmentation of optical and SAR images rely on expensive pixel-level annotation, we develop a weakly supervised instance segmentation (WSIS) method to balance the visual processing requirements with the annotation cost. First, we decompose the prior knowledge of the mask-aware task in WSIS into three meta-knowledge components: fundamental knowledge, apparent knowledge, and detailed knowledge inspired by human visual perception habits of “whole to part” and “coarse to detailed.” Then, a meta-knowledge-guided weakly supervised instance segmentation network (MGWI-Net) is proposed. In this network, the weakly supervised mask (WSM) head can instantiate both fundamental knowledge and apparent knowledge to perform mask awareness without any annotations at the pixel level. The network also includes a mask information awareness assist (MIAA) head, which can implicitly guide the network to learn detailed information about edges through the boundary-sensitive feature of the fully connected conditional random field (CRF), facilitating the instantiation of detailed knowledge. The experimental results show that the MGWI-Net can efficiently generate instance masks for optical and SAR images and achieve the approximate instance segmentation results of the fully supervised method with about one-eighth of the annotation production time. The model parameters and processing speed of our network are also competitive. This study can provide inexpensive and convenient technical support for applying and promoting instance segmentation methods for optical and SAR images.

Semantic Attention and Structured Model for Weakly Supervised Instance Segmentation in Optical and SAR Remote Sensing Imagery

“…It is important to note that we compare SASM-Net with other methods, which can be categorized into three groups: weakly supervised paradigm methods, fully supervised paradigm methods, and hybrid supervised paradigm methods. The details of these methods used for comparison are similar to [49] and are briefly described as follows:…”

“…Adaptations of fully supervised methods directly treat the object-level labels from annotations as bounding box labels to train the original fully supervised methods. Dedicated weakly supervised methods are designed explicitly for bounding box labels, including BoxInst [50], DiscoBox [28], DBIN [51], and MGWI-Net [49]. For DBIN, we exclude the domain adaptation aspect as it is beyond the scope of this paper.…”