Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need?

Boudiaf, Malik; Kervadec, Hoel; Masud, Ziko Imtiaz; Piantanida, Pablo; Ayed, Ismail Ben; Dolz, José

doi:10.1109/cvpr46437.2021.01376

Cited by 130 publications

(69 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the second training stage, we freeze the backbone and fine-tune the classifier. 1 Our loss function, L, for this second stage of training is:…”

Section: Vanilla Fine-tuning Approachmentioning

confidence: 99%

“…Generally, such methods first train on base classes for which there is an abundance of labeled data and then try to generalize to novel classes for which only few annotated examples are available. A limitation of most models is that they are only evaluated with respect to their performance on the novel classes [1,42,49]. In other words, existing work largely ignores whether the model retains knowledge of how to analyze the base categories.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuning

Myers-Dean¹,

Zhao²,

Price³

et al. 2021

Preprint

View full text Add to dashboard Cite

Generalized few-shot semantic segmentation was introduced to move beyond only evaluating few-shot segmentation models on novel classes to include testing their ability to remember base classes. While all approaches currently are based on meta-learning, they perform poorly and saturate in learning after observing only a few shots. We propose the first fine-tuning solution, and demonstrate that it addresses the saturation problem while achieving stateof-art results on two datasets, PASCAL-5 i and COCO-20 i . We also show it outperforms existing methods whether finetuning multiple final layers or only the final layer. Finally, we present a triplet loss regularization that shows how to redistribute the balance of performance between novel and base categories so that there is a smaller gap between them.

show abstract

“…In the second training stage, we freeze the backbone and fine-tune the classifier. 1 Our loss function, L, for this second stage of training is:…”

Section: Vanilla Fine-tuning Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuning

Myers-Dean¹,

Zhao²,

Price³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…the query set) aided with a few labelled images containing the novel classes (i.e. the support set) [1], [2], [3], [4], [5], [6], [7], [8]. Most approaches follow a meta-learning scheme that emulates the inference stage during training through sampling tasks of support and query sets (i.e.…”

Section: Introductionmentioning

confidence: 99%

“…Meta-learning has been widely explored in few-shot learning [2], [7], [11], [12]. Even so, recent work has pointed out issues in the applicability of meta-learning to the fewshot setting in realistic scenarios with domain shift [8], [13]. Transductive inference has emerged as a viable means to address some of these issues [8], [14], [15], [16], [17], [18].…”

Section: Introductionmentioning

confidence: 99%

“…Even so, recent work has pointed out issues in the applicability of meta-learning to the fewshot setting in realistic scenarios with domain shift [8], [13]. Transductive inference has emerged as a viable means to address some of these issues [8], [14], [15], [16], [17], [18]. Within the context of few-shot single image segmentation, recent work has shown that transductive inference can lead to surpassing the performance of non-transductive approaches [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Temporal Transductive Inference for Few-Shot Video Object Segmentation

Siam¹,

Derpanis²,

Wildes³

2022

Preprint

View full text Add to dashboard Cite

Few-shot video object segmentation (FS-VOS) aims at segmenting video frames using a few labelled examples of classes not seen during initial training. In this paper, we present a simple but effective temporal transductive inference (TTI) approach that leverages temporal consistency in the unlabelled video frames during few-shot inference. Key to our approach is the use of both global and local temporal constraints. The objective of the global constraint is to learn consistent linear classifiers for novel classes across the image sequence, whereas the local constraint enforces the proportion of foreground/background regions in each frame to be coherent across a local temporal window. These constraints act as spatiotemporal regularizers during the transductive inference to increase temporal coherence and reduce overfitting on the few-shot support set. Empirically, our model outperforms state-of-the-art meta-learning approaches in terms of mean intersection over union on YouTube-VIS by 2.8%. In addition, we introduce improved benchmarks that are exhaustively labelled (i.e. all object occurrences are labelled, unlike the currently available), and present a more realistic evaluation paradigm that targets data distribution shift between training and testing sets. Our empirical results and in-depth analysis confirm the added benefits of the proposed spatiotemporal regularizers to improve temporal coherence and overcome certain overfitting scenarios.

show abstract

Part-Aware Prototype Network for Few-Shot Semantic Segmentation

Liu

Zhang

et al. 2020

Lecture Notes in Computer Science

248

195

View full text Add to dashboard Cite

The key challenge for few-shot semantic segmentation (FSS) is how to tailor a desirable interaction among support and query features and/or their prototypes, under the episodic training scenario. Most existing FSS methods implement such support/query interactions by solely leveraging plain operations -e.g., cosine similarity and feature concatenation -for segmenting the query objects. However, these interaction approaches usually cannot well capture the intrinsic object details in the query images that are widely encountered in FSS, e.g., if the query object to be segmented has holes and slots, inaccurate segmentation almost always happens. To this end, we propose a dynamic prototype convolution network (DPCN) to fully capture the aforementioned intrinsic details for accurate FSS. Specifically, in DPCN, a dynamic convolution module (DCM) is firstly proposed to generate dynamic kernels from support foreground, then information interaction is achieved by convolution operations over query features using these kernels. Moreover, we equip DPCN with a support activation module (SAM) and a feature filtering module (FFM) to generate pseudo mask and filter out background information for the query images, respectively. SAM and FFM together can mine enriched context information from the query features. Our DPCN is also flexible and efficient under the k-shot FSS setting. Extensive experiments on PASCAL-5 i and COCO-20 i show that DPCN yields superior performances under both 1-shot and 5-shot settings.

show abstract

Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need?

Cited by 130 publications

References 16 publications

Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuning

Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuning

Temporal Transductive Inference for Few-Shot Video Object Segmentation

Part-Aware Prototype Network for Few-Shot Semantic Segmentation

Contact Info

Product

Resources

About