Semi-supervised Semantic Segmentation with Directional Context-aware Consistency

Lai, Xin; Tian, Zhuotao; Jiang, Li; Liu, Shu; Zhao, Hengshuang; Wang, Liwei; Jia, Jiaya

doi:10.1109/cvpr46437.2021.00126

Cited by 167 publications

(120 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Without iterative re-training adopted as [50], our ST and ST++ framework surpass the previous state-of-the-art methods impressively. We surprisingly observe that, in the Full setting, where 1,464 labeled images are available, we achieve 78.9% and 79.1% mIOU with our ST and ST++ respectively, even superior to the 77.7% mIOU [25] under fully-supervised setting with auxiliary annotations from the SBD.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 79%

“…Therefore, also inspired the success in SSL, subsequent methods manage to tackle this task with simpler mechanisms, such as employing CutOut [13] or CutMix [51] for consistency training [16] and enforcing similar predictions under different perturbed embeddings [34]. The more recent work [25] adopts the contrastive loss to regularize consistent predictions for a local patch under different crops. As for the hybrid framework of consistency regularization and entropy minimization, PseudoSeg [58] further applies a calibration module to refine the produced pseudo masks.…”

Section: Related Workmentioning

confidence: 99%

“…The Cityscapes [11] contains 2,975 images with fine-grained masks for training and 500 images for validation. As for the data partition, we collect the open-sourced ids of labeled images from [25,58,32] based on different settings we are comparing. Network structure.…”

Section: Setupmentioning

confidence: 99%

“…With increasingly sophisticated mechanisms introduced to the field, FixMatch [42] manages to break this trend and achieves promising results via taking both advantages of the two strategies in a hybrid framework with fewer hyper-parameters introduced. Motivated by the tremendous progress in SSL, recent works on semi-supervised semantic segmentation are evolving from GANs-based methods [17,44,21,32] to integrating the advanced SSL methods into the segmentation scenario, such as enforcing consistent predictions of the same unlabeled image but under strong-weak perturbations [58] and consistent predictions of the same local patch but under different contextual crops [25].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation

Yang¹,

Zhuo²,

Qi³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we investigate if we could make the self-training -a simple but popular framework -work better for semi-supervised segmentation. Since the core issue in semi-supervised setting lies in effective and efficient utilization of unlabeled data, we notice that increasing the diversity and hardness of unlabeled data is crucial to performance improvement. Being aware of this fact, we propose to adopt the most plain self-training scheme coupled with appropriate strong data augmentations on unlabeled data (namely ST) for this task, which surprisingly outperforms previous methods under various settings without any bells and whistles. Moreover, to alleviate the negative impact of the wrongly pseudo labeled images, we further propose an advanced self-training framework (namely ST++), that performs selective re-training via selecting and prioritizing the more reliable unlabeled images. As a result, the proposed ST++ boosts the performance of semi-supervised model significantly and surpasses existing methods by a large margin on the Pascal VOC 2012 and Cityscapes benchmark. Overall, we hope this straightforward and simple framework will serve as a strong baseline or competitor for future works.Code is available at https://github.com/LiheYoung/ST-PlusPlus.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 79%

Section: Related Workmentioning

confidence: 99%

Section: Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation

Yang¹,

Zhuo²,

Qi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, for pixel-level dense prediction tasks like image segmentation, such image-level CL approaches ignore the spatial relation information of the image. For exploring spatial relation information, [7] and [8] construct positive instances with pairs of points under different augmentations but having the same actual locations, and select the negative instances according to semantic inconsistency. [9] combines both image-level and point-level CL, and relies on the correspondence between different views of the same image to sample positive instances for the point-level CL.…”

Section: Introductionmentioning

confidence: 99%

Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image Segmentation

Zhao¹,

Fang²,

Fan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Semi-supervised learning (SSL), which aims at leveraging a few labeled images and a large number of unlabeled images for network training, is beneficial for relieving the burden of data annotation in medical image segmentation. According to the experience of medical imaging experts, local attributes such as texture, luster and smoothness are very important factors for identifying target objects like lesions and polyps in medical images. Motivated by this, we propose a crosslevel contrastive learning scheme to enhance representation capacity for local features in semi-supervised medical image segmentation. Compared to existing image-wise, patch-wise and point-wise contrastive learning algorithms, our devised method is capable of exploring more complex similarity cues, namely the relational characteristics between global and local patch-wise representations. Additionally, for fully making use of cross-level semantic relations, we devise a novel consistency constraint that compares the predictions of patches against those of the full image. With the help of the cross-level contrastive learning and consistency constraint, the unlabelled data can be effectively explored to improve segmentation performance on two medical image datasets for polyp and skin lesion segmentation respectively. Code of our approach is available here.

show abstract

Light mixed‐supervised segmentation for 3D medical image data

Yang,

Tan,

Tegzes

et al. 2023

Medical Physics

View full text Add to dashboard Cite

BackgroundAccurate 3D semantic segmentation models are essential for many clinical applications. To train a model for 3D segmentation, voxel‐level annotation is necessary, which is expensive to obtain due to laborious work and privacy protection. To accurately annotate 3D medical data, such as MRI, a common practice is to annotate the volumetric data in a slice‐by‐slice contouring way along principal axes.PurposeIn order to reduce the annotation effort in slices, weakly supervised learning with a bounding box (Bbox) was proposed to leverage the discriminating information via a tightness prior assumption. Nevertheless, this method requests accurate and tight Bboxes, which will significantly drop the performance when tightness is not held, that is when a relaxed Bbox is applied. Therefore, there is a need to train a stable model based on relaxed Bbox annotation.MethodsThis paper presents a mixed‐supervised training strategy to reduce the annotation effort for 3D segmentation tasks. In the proposed approach, a fully annotated contour is only required for a single slice of the volume. In contrast, the rest of the slices with targets are annotated with relaxed Bboxes. This mixed‐supervised method adopts fully supervised learning, relaxed Bbox prior, and contrastive learning during the training, which ensures the network exploits the discriminative information of the training volumes properly. The proposed method was evaluated on two public 3D medical imaging datasets (MRI prostate dataset and Vestibular Schwannoma [VS] dataset).ResultsThe proposed method obtained a high segmentation Dice score of 85.3% on an MRI prostate dataset and 83.3% on a VS dataset with relaxed Bbox annotation, which are close to a fully supervised model. Moreover, with the same relaxed Bbox annotations, the proposed method outperforms the state‐of‐the‐art methods. More importantly, the model performance is stable when the accuracy of Bbox annotation varies.ConclusionsThe presented study proposes a method based on a mixed‐supervised learning method in 3D medical imaging. The benefit will be stable segmentation of the target in 3D images with low accurate annotation requirement, which leads to easier model training on large‐scale datasets.

show abstract

Semi-supervised Semantic Segmentation with Directional Context-aware Consistency

Cited by 167 publications

References 37 publications

ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation

ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation

Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image Segmentation

Light mixed‐supervised segmentation for 3D medical image data

Contact Info

Product

Resources

About