Group-Wise Learning for Weakly Supervised Semantic Segmentation

Zhou, Tianfei; Li, Liulei; Li, Xueyi; Feng, Chun-Mei; Li, Jianwu; Shao, Ling

doi:10.1109/tip.2021.3132834

Cited by 76 publications

(29 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As FCNs suffer from limited visual context with local receptive fields, how to effectively capture cross-pixel relations became the main focus of follow-up studies. Scholars devised many promising solutions, by enlarging receptive fields [10,13,19,93,97,105], building image pyramids [33,46], exploring encoder-decoder architectures [3,13,62], utilizing boundary clues [23,41,99], or incorporating neural attention [25,32,35,40,42,73,84,106,108,113]. Recently, a new family of semantic segmentation models [16,69,90,107], built upon the full attention (Transformer [76]) architecture, yielded impressive performance, as it overcomes the issues in long-range cross-pixel dependency modeling.…”

Section: Related Workmentioning

confidence: 99%

Deep Hierarchical Semantic Segmentation

Li¹,

Zhou²,

Yang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Humans are able to recognize structured relations in observation, allowing us to decompose complex scenes into simpler parts and abstract the visual world in multiple levels. However, such hierarchical reasoning ability of human perception remains largely unexplored in current literature of semantic segmentation. Existing work is often aware of flatten labels and predicts target classes exclusively for each pixel. In this paper, we instead address hierarchical semantic segmentation (HSS), which aims at structured, pixel-wise description of visual observation in terms of a class hierarchy. We devise HSSN, a general HSS framework that tackles two critical issues in this task: i) how to efficiently adapt existing hierarchy-agnostic segmentation networks to the HSS setting, and ii) how to leverage the hierarchy information to regularize HSS network learning. To address i), HSSN directly casts HSS as a pixel-wise multi-label classification task, only bringing minimal architecture change to current segmentation models. To solve ii), HSSN first explores inherent properties of the hierarchy as a training objective, which enforces segmentation predictions to obey the hierarchy structure. Further, with hierarchy-induced margin constraints, HSSN reshapes the pixel embedding space, so as to generate well-structured pixel representations and improve segmentation eventually. We conduct experiments on four semantic segmentation datasets (i.e., Mapillary Vistas 2.0, Cityscapes, LIP, and PASCAL-Person-Part), with different class hierarchies, segmentation network architectures and backbones, showing the generalization and superiority of HSSN.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Hierarchical Semantic Segmentation

Li¹,

Zhou²,

Yang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Cross-image pixel contrast for semantic segmentation is a segmentation method that addresses intra-class compactness and inter-class dispersion [ 36 ] and will be incorporated into this segmentation framework in future experiments. Group-wise learning for weakly supervised semantic segmentation is another segmentation method that enables the discovery of relationships among groups of images [ 37 ]. This methodology will also be used in future experiments.…”

Section: Discussionmentioning

confidence: 99%

Deep Neural Network for Cardiac Magnetic Resonance Image Segmentation

et al. 2022

View full text Add to dashboard Cite

The analysis and interpretation of cardiac magnetic resonance (CMR) images are often time-consuming. The automated segmentation of cardiac structures can reduce the time required for image analysis. Spatial similarities between different CMR image types were leveraged to jointly segment multiple sequences using a segmentation model termed a multi-image type UNet (MI-UNet). This model was developed from 72 exams (46% female, mean age 63 ± 11 years) performed on patients with hypertrophic cardiomyopathy. The MI-UNet for steady-state free precession (SSFP) images achieved a superior Dice similarity coefficient (DSC) of 0.92 ± 0.06 compared to 0.87 ± 0.08 for a single-image type UNet (p < 0.001). The MI-UNet for late gadolinium enhancement (LGE) images also had a superior DSC of 0.86 ± 0.11 compared to 0.78 ± 0.11 for a single-image type UNet (p = 0.001). The difference across image types was most evident for the left ventricular myocardium in SSFP images and for both the left ventricular cavity and the left ventricular myocardium in LGE images. For the right ventricle, there were no differences in DCS when comparing the MI-UNet with single-image type UNets. The joint segmentation of multiple image types increases segmentation accuracy for CMR images of the left ventricle compared to single-image models. In clinical practice, the MI-UNet model may expedite the analysis and interpretation of CMR images of multiple types.

show abstract

“…Specifically, FCN [80] is a milestone; it learns dense prediction efficiently. Since it was proposed, numerous efforts have been devoted to improving FCN, by, for example, enlarging the receptive field [15,16,26,126,130,137]; strengthening context cues [4,44,48,49,57,58,71,76,78,82,91,128,130,131,134,140,143]; leveraging boundary information [6,14,28,67,129,133,139]; incorporating neural attention [35,42,43,50,51,64,69,103,112,114,138]; or automating network engineering [18,70,73,86]. Lately, Transformer based solutions [20,102,…”

Section: Related Workmentioning

confidence: 99%

Rethinking Semantic Segmentation: A Prototype View

Zhou¹,

Yang²,

Konukoğlu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Prevalent semantic segmentation solutions, despite their different network designs (FCN based or attention based) and mask decoding strategies (parametric softmax based or pixel-query based), can be placed in one category, by considering the softmax weights or query vectors as learnable class prototypes. In light of this prototype view, this study uncovers several limitations of such parametric segmentation regime, and proposes a nonparametric alternative based on non-learnable prototypes. Instead of prior methods learning a single weight/query vector for each class in a fully parametric manner, our model represents each class as a set of non-learnable prototypes, relying solely on the mean features of several training pixels within that class. The dense prediction is thus achieved by nonparametric nearest prototype retrieving. This allows our model to directly shape the pixel embedding space, by optimizing the arrangement between embedded pixels and anchored prototypes. It is able to handle arbitrary number of classes with a constant amount of learnable parameters.We empirically show that, with FCN based and attention based segmentation models (i.e., HR-Net, Swin, SegFormer) and backbones (i.e., ResNet, HRNet, Swin, MiT), our nonparametric framework yields compelling results over several datasets (i.e., ADE20K, Cityscapes, COCO-Stuff), and performs well in the large-vocabulary situation. We expect this work will provoke a rethink of the current de facto semantic segmentation model design.

show abstract

Group-Wise Learning for Weakly Supervised Semantic Segmentation

Cited by 76 publications

References 79 publications

Deep Hierarchical Semantic Segmentation

Deep Hierarchical Semantic Segmentation

Deep Neural Network for Cardiac Magnetic Resonance Image Segmentation

Rethinking Semantic Segmentation: A Prototype View

Contact Info

Product

Resources

About