Weakly Supervised Semantic Segmentation Via Progressive Patch Learning

Li, Jinlong; Jie, Zequn; Wang, Xu; Zhou, Yu; Wei, Xiaolin; Ma, Lin

doi:10.1109/tmm.2022.3152388

Cited by 12 publications

(4 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experiments are conducted on two benchmarks: PASCAL VOC 2012 (Everingham et al 2010) with 21 classes and MS COCO 2014 (Lin et al 2014) with 81 classes. For PASCAL VOC 2012, following (Wang et al 2020;Lee, Kim, and Yoon 2021;Chen et al 2022a;Li et al 2022), we use the augmented SBD set (Hariharan et al 2011) with 10,582 annotated images. Mean Intersection over Union (mIoU) (Long, Shelhamer, and Darrell 2015) is used to evaluate segmentation results.…”

Section: Methodsmentioning

confidence: 99%

SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

Zhao,

Tang,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

Image-level weakly supervised semantic segmentation has received increasing attention due to its low annotation cost. Existing methods mainly rely on Class Activation Mapping (CAM) to obtain pseudo-labels for training semantic segmentation models. In this work, we are the first to demonstrate that long-tailed distribution in training data can cause the CAM calculated through classifier weights over-activated for head classes and under-activated for tail classes due to the shared features among head- and tail- classes. This degrades pseudo-label quality and further influences final semantic segmentation performance. To address this issue, we propose a Shared Feature Calibration (SFC) method for CAM generation. Specifically, we leverage the class prototypes which carry positive shared features and propose a Multi-Scaled Distribution-Weighted (MSDW) consistency loss for narrowing the gap between the CAMs generated through classifier weights and class prototypes during training. The MSDW loss counterbalances over-activation and under-activation by calibrating the shared features in head-/tail-class classifier weights. Experimental results show that our SFC significantly improves CAM boundaries and achieves new state-of-the-art performances. The project is available at https://github.com/Barrett-python/SFC.

show abstract

Section: Methodsmentioning

confidence: 99%

SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

Zhao,

Tang,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Seed Mask PSA(CVPR18) [7] 48.0 61.0 IRNet(CVPR19) [32] 48.3 66.5 Chang et al(CVPR20) [49] 50.9 63.4 SEAM(CVPR20) [20] 55.4 63.6 AdvCAM(CVPR21) [18] 55.6 68.0 CDA(ICCV21) [50] 55.4 63.4 Zhang et al(ICCV21) [51] 57.4 67.8 SIPE(CVPR22) [52] 58.6 64.7 ReCAM(CVPR22) [53] 56.6 -MCTformer(CVPR22) [8] 61.7 69.1 AMR(AAAI22) [54] 56.8 69.7 eX-ViT(PR23) [39] 59.1 65.3 PPL(TMM23) [55] 51.0 68.5…”

Section: Methodsmentioning

confidence: 99%

Enhancing Weakly Supervised Semantic Segmentation through Multi-Class Token Attention Learning

Luo,

Zeng

2024

Preprint

View full text Add to dashboard Cite

Weakly supervised semantic segmentation (WSSS) using image-level class labels is challenging due to the limitations of Class Activation Maps (CAMs) in convolutional neural networks (CNNs), which often highlight only the most discriminative image regions. We propose the Hierarchical Multi-Class Token Attention Network (HMCTANet), a novel approach leveraging a Conformer backbone that integrates CNN and Transformer branches. HMCTANet enhances CAMs through multi-class token attention and a Class-Aware Training (CAT) strategy that aligns class tokens with ground-truth labels. Additionally, we introduce a Class Token Regularization Module (CTRM) to improve the discriminative power of class tokens. Our Refinement Module (RM) further refines segmentation by combining class-specific attention and patch-level affinity from the Transformer branch with the CAMs from the CNN branch. HMCTANet achieves state-of-the-art performance, with mIoU scores of 69.0% and 68.4% on the PASCAL VOC 2012 validation and test sets, respectively, demonstrating the effectiveness of our approach for WSSS tasks.

show abstract

“…This module finds semantic relations between regions or pixels and uses information with the help of the PCM module proposed in the ISIM work [22]. In the PPL [24] method, the image is split into patches. Each patch is fed to subsequent convolutional layers separately.…”

Section: Weakly Supervised Methodsmentioning

confidence: 99%

Enabling Multi-Part Plant Segmentation with Instance-Level Augmentation Using Weak Annotations

et al. 2023

View full text Add to dashboard Cite

Plant segmentation is a challenging computer vision task due to plant images complexity. For many practical problems, we have to solve even more difficult tasks. We need to distinguish plant parts rather than the whole plant. The major complication of multi-part segmentation is the absence of well-annotated datasets. It is very time-consuming and expensive to annotate datasets manually on the object parts level. In this article, we propose to use weakly supervised learning for pseudo-annotation. The goal is to train a plant part segmentation model using only bounding boxes instead of fine-grained masks. We review the existing weakly supervised learning approaches and propose an efficient pipeline for agricultural domains. It is designed to resolve tight object overlappings. Our pipeline beats the baseline solution by 23% for the plant part case and by 40% for the whole plant case. Furthermore, we apply instance-level augmentation to boost model performance. The idea of this approach is to obtain a weak segmentation mask and use it for cropping objects from original images and pasting them to new backgrounds during model training. This method provides us a 55% increase in mAP compared with the baseline on object part and a 72% increase on the whole plant segmentation tasks.

show abstract

Weakly Supervised Semantic Segmentation Via Progressive Patch Learning

Cited by 12 publications

References 56 publications

SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

Enhancing Weakly Supervised Semantic Segmentation through Multi-Class Token Attention Learning

Enabling Multi-Part Plant Segmentation with Instance-Level Augmentation Using Weak Annotations

Contact Info

Product

Resources

About