Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation

Abstract

Existing studies in weakly supervised semantic segmentation (WSSS) have utilized class activation maps (CAMs) to localize the class objects. However, since a classification loss is insufficient for providing precise object regions, CAMs tend to be biased towards discriminative patterns (i.e., sparseness) and do not provide precise object boundary information (i.e., impreciseness). To resolve these limitations, we propose a novel framework (composed of MainNet and SupportNet.) that derives pixel-level selfsupervision from given image-level supervision. In our framework, with the help of the proposed Regional Contrastive Module (RCM) and Multi-scale Attentive Module (MAM), MainNet is trained by self-supervision from the SupportNet. The RCM extracts two forms of selfsupervision from SupportNet: (1) class region masks generated from the CAMs and (2) class-wise prototypes obtained from the features according to the class region masks. Then, every pixel-wise feature of the MainNet is trained by the prototype in a contrastive manner, sharpening the resulting CAMs. The MAM utilizes CAMs inferred at multiple scales from the SupportNet as self-supervision to guide the MainNet. Based on the dissimilarity between the multiscale CAMs from MainNet and SupportNet, CAMs from the MainNet are trained to expand to the less-discriminative regions. The proposed method shows state-of-the-art WSSS performance both on the train and validation sets on the PASCAL VOC 2012 dataset. For reproducibility, code will be available publicly soon.

Activation extending based on long-range dependencies for weakly supervised semantic segmentation

“…According to the above steps, we obtain a CAM generative model trained by source samples and image-level classification labels, after which the conventional two post-processing steps are followed: (1) CAM regions are selected as seed regions by threshold [11]. (2) Expand it as the final pseudo-label [18]. And its visualization results are shown in Experimental Results and Disscusion.…”

“…Also, class-activated mapping (CAM) [16] is an effective solution to generate pixel-level pseudo-labels through image-level classification labels. However, due to the discriminant mode of the classifiers [17,18], and these labels contain limited spatial details [19], that often leads to the local activation regions [20], and the segmented object boundaries easily involve false activation. They thus will cause different degrees of fragmentary masks [21].…”

“…They thus will cause different degrees of fragmentary masks [21]. A lot of recent work has refined the quality of CAM by mining more semantic and object location information from limited annotation information [10,11,18]. The success of these methods depends on the long-range dependencies [22] between pixels in an image.…”