ContrastMask: Contrastive Learning to Segment Every Thing

Wang, Xuehui; Zhao, Kai; Zhang, Ruixin; Ding, Shouhong; Wang, Yan; Shen, Wei

doi:10.1109/cvpr52688.2022.01131

Cited by 31 publications

(10 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, the open-set problem has been explored in various computer vision tasks (Bendale and Boult 2015;Dhamija et al 2020;Joseph et al 2021;Gupta et al 2022;Zhao et al 2022;Vaze et al 2021;Qi et al 2021;Saito et al 2021;Wang et al 2022b;Hwang et al 2021;Wang et al 2022aWang et al , 2021. Dhamija et al (Dhamija et al 2020) first formal-ize the open-set object detection problem and propose the open-set object detection protocol to better estimate the performance under real-world conditions.…”

Section: Open-set Detection and Segmentationmentioning

confidence: 99%

Revisiting Open-Set Panoptic Segmentation

Yin,

Chen,

Zhou

et al. 2024

AAAI

View full text Add to dashboard Cite

In this paper, we focus on the open-set panoptic segmentation (OPS) task to circumvent the data explosion problem. Different from the close-set setting, OPS targets to detect both known and unknown categories, where the latter is not annotated during training. Different from existing work that only selects a few common categories as unknown ones, we move forward to the real-world scenario by considering the various tail categories (~1k). To this end, we first build a new dataset with long-tail distribution for the OPS task. Based on this dataset, we additionally add a new class type for unknown classes and re-define the training annotations to make the OPS definition more complete and reasonable. Moreover, we analyze the influence of several significant factors in the OPS task and explore the upper bound of performance on unknown classes with different settings. Furthermore, based on the analyses, we design an effective two-phase framework for the OPS task, including thing-agnostic map generation and unknown segment mining. We further adopt semi-supervised learning to improve the OPS performance. Experimental results on different datasets validate the effectiveness of our method.

show abstract

Section: Open-set Detection and Segmentationmentioning

confidence: 99%

Revisiting Open-Set Panoptic Segmentation

Yin,

Chen,

Zhou

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…The MAE [6] is a type of neural network architecture that learns to reconstruct its input data while selectively ignoring or masking certain parts of the input, typically used for feature learning and dimensionality reduction. Masked autoencoders can be successful in various computer vision tasks [39,42,45,46], such as image segmentation, tracking, and generation, because they can capture and represent meaningful features in images by learning to reconstruct them while ignoring noisy or less relevant information. The proposed BTMAE can effectively enhance RIS performance by leveraging the feature representation modeling capability of MAE to learn complex contextual relationships between modalities in multimodal spaces.…”

Section: Related Workmentioning

confidence: 99%

A Case Study on the Settlement Experience of North Korean Refugee Women in South Korea

Lee¹,

Lee²

2020

Cent Social Welf Res Yonsei Univ

View full text Add to dashboard Cite

Unsupervised Video Object Segmentation (UVOS) refers to the challenging task of segmenting the prominent object in videos without manual guidance. In other words, the network detects the accurate region of the target object in a sequence of RGB frames without prior knowledge. In recent works, two approaches for UVOS have been discussed that can be divided into: appearance and appearance-motion based methods. Appearance based methods utilize the correlation information of inter-frames to capture target object that commonly appears in a sequence. However, these methods does not consider the motion of target object due to exploit the correlation information between randomly paired frames. Appearance-motion based methods, on the other hand, fuse the appearance features from RGB frames with the motion features from optical flow. Motion cue provides useful information since salient objects typically show distinctive motion in a sequence. However, these approaches have the limitation that the dependency on optical flow is dominant. In this paper, we propose a novel framework for UVOS that can address aforementioned limitations of two approaches in terms of both time and scale. Temporal Alignment Fusion aligns the saliency information of adjacent frames with the target frame to leverage the information of adjacent frames. Scale Alignment Decoder predicts the target object mask precisely by aggregating differently scaled feature maps via continuous mapping with implicit neural representation. We present experimental results on public benchmark datasets, DAVIS 2016 and FBMS, which demonstrate the effectiveness of our method. Furthermore, we outperform the state-of-the-art methods on DAVIS 2016.

show abstract

“…One promising solution to this problem is to leverage abundant unlabeled data through semi-supervised learning (SSL), which have been actively explored in medical segmentation tasks (Luo et al 2021a;You et al 2023;Peiris et al 2023;Zhu et al 2021). Existing SSL methods typically leverage knowledge leart from a few labeled images through pseudo labeling, and then enhance representation learning through consistency regularization on unlabeled data (Luo et al 2021b;Wang et al 2022;You et al 2022;Zhou et al 2019). However, as pointed out in (Wang, Li, and Gool 2019), in practical scenarios, the distribution of labeled data often more or less deviates from the true distribution of the real-world dataset, which is caused by the small sample size and the randomness in sampling.…”

Section: Introductionmentioning

confidence: 99%

Semi-supervised TEE Segmentation via Interacting with SAM Equipped with Noise-Resilient Prompting

Deng,

Feng,

Lin

et al. 2024

AAAI

View full text Add to dashboard Cite

Semi-supervised learning (SSL) is a powerful tool to address the challenge of insufficient annotated data in medical segmentation problems. However, existing semi-supervised methods mainly rely on internal knowledge for pseudo labeling, which is biased due to the distribution mismatch between the highly imbalanced labeled and unlabeled data. Segmenting left atrial appendage (LAA) from transesophageal echocardiogram (TEE) images is a typical medical image segmentation task featured by scarcity of professional annotations and diverse data distributions, for which existing SSL models cannot achieve satisfactory performance. In this paper, we propose a novel strategy to mitigate the inherent challenge of distribution mismatch in SSL by, for the first time, incorporating a large foundation model (i.e. SAM in our implementation) into an SSL model to improve the quality of pseudo labels. We further propose a new self-reconstruction mechanism to generate both noise-resilient prompts to demonically improve SAM’s generalization capability over TEE images and self-perturbations to stabilize the training process and reduce the impact of noisy labels. We conduct extensive experiments on an in-house TEE dataset; experimental results demonstrate that our method achieves better performance than state-of-the-art SSL models.

show abstract

ContrastMask: Contrastive Learning to Segment Every Thing

Cited by 31 publications

References 37 publications

Revisiting Open-Set Panoptic Segmentation

Revisiting Open-Set Panoptic Segmentation

A Case Study on the Settlement Experience of North Korean Refugee Women in South Korea

Semi-supervised TEE Segmentation via Interacting with SAM Equipped with Noise-Resilient Prompting

Contact Info

Product

Resources

About