Mining Latent Classes for Few-shot Segmentation

Yang, Lihe; Zhuo, Wei; Qi, Lei; Shi, Yinghuan; Gao, Yang

doi:10.48550/arxiv.2103.15402

Cited by 5 publications

(8 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Few-shot approaches are also highly related with our work. Few-shot classification [8,44], detection [6,45], and segmentation [48,51] aim to complete corresponding tasks on images of novel classes given a few exemplars. For classification, MAML [8] learns parameters which can adapt to novel classes at test time by few gradient descent steps.…”

Section: Related Workmentioning

confidence: 99%

“…For segmentation, [51] proposes a two-branch dense comparison module performing multi-level feature comparison between the input image and the exemplars, and the segmentation results are iteratively refined. [48] aims to alleviate the problem of feature undermining and enhance the feature embedding of latent novel classes. However, researches for few-shot counting are still limited.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Few-shot Object Counting with Similarity-Aware Feature Enhancement

You¹,

Shen²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Few-shot counting aims to count objects of any class in an image given only a few exemplars of the same class. Existing correlation-based few-shot counting approaches suffer from the coarseness and low semantic level of the correlation. To solve these problems, we propose an iterative framework to progressively refine the exemplar-related features based on the correlation between the image and exemplars. Then the density map is predicted from the final refined feature map. The iterative framework includes a Correlation Distillation module and a Feature Refinement module. During the iterations, the exemplar-related features are gradually refined, while the exemplar-unrelated features are suppressed, benefiting few-shot counting where the exemplar-related features are more important. Our approach surpasses all baselines significantly on few-shot counting benchmark FSC-147. Surprisingly, though designed for general class-agnostic counting, our approach still achieves state-of-the-art performance on car counting benchmarks CARPK and PUCPR+, and crowd counting benchmarks UCSD and Mall. We also achieve competitive performance on crowd counting benchmark ShanghaiTech. The code will be released soon.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Few-shot Object Counting with Similarity-Aware Feature Enhancement

You¹,

Shen²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Inspired by few-shot learning paradigm [48,57], which aims to learn-to-learn a model for a novel task with only a limited number of samples, fewshot segmentation has received considerable attention. Following the success of [54], prototypical networks [57] and numerous other works [8,25,30,32,43,55,59,68,[75][76][77]82] proposed to utilize a prototype extracted from support samples, which is used to refine the query features to contain the relevant support information. In addition, inspired by [80] that observed the use of high-level features leads to a performance drop, [62] proposed to utilize high-level features by computing a prior map which takes maximum score within a correlation map.…”

Section: Related Workmentioning

confidence: 99%

“…The key to few-shot segmentation is how to effectively utilize provided support samples for a query image. While conventional methods [25,59,62,77,81] attempted to utilize global-or part-level prototypes extracted from support features, recent methods [29,38,67,74,79,81] attempted to leverage pixel-wise relationships between query and support. One of the latest work, HSNet [38], attempts to aggregate the matching scores with 4D convolutions, but it lacks an ability to consider interactions among the matching scores due to the inherent nature of convolutions.…”

Section: Motivation and Overviewmentioning

confidence: 99%

“…For a decade, numerous methods for few-shot segmentation have been proposed [2,25,30,32,37,38,67,72,[74][75][76][77][78], and most methods follow a learning-to-learn paradigm [48] to avoid the risk of overfitting due to insufficient training data. Since the prediction for query image should be conditioned on support images and corresponding masks, the key to this task is how to effectively utilize provided support samples.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cost Aggregation Is All You Need for Few-Shot Segmentation

Hong¹,

Cho²,

Nam³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce a novel cost aggregation network, dubbed Volumetric Aggregation with Transformers (VAT), to tackle the few-shot segmentation task by using both convolutions and transformers to efficiently handle high dimensional correlation maps between query and support. In specific, we propose our encoder consisting of volume embedding module to not only transform the correlation maps into more tractable size but also inject some convolutional inductive bias and volumetric transformer module for the cost aggregation. Our encoder has a pyramidal structure to let the coarser level aggregation to guide the finer level and enforce to learn complementary matching scores. We then feed the output into our affinity-aware decoder along with the projected feature maps for guiding the segmentation process. Combining these components, we conduct experiments to demonstrate the effectiveness of the proposed method, and our method sets a new state-of-the-art for all the standard benchmarks in few-shot segmentation task. Furthermore, we find that the proposed method attains state-of-the-art performance even for the standard benchmarks in semantic correspondence task although not specifically designed for this task. We also provide an extensive ablation study to validate our architectural choices. The trained weights and codes are available at: https: //seokju-cho.github.io/VAT/.

show abstract

Prototype Queue Learning for Multi-Class Few-Shot Semantic Segmentation

Wang

Jiang

Yuan

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Few-shot semantic segmentation aims to undertake the segmentation task of novel classes with only a few annotated images. However, most existing methods tend to segment the foreground and background in the image, which limits practical application. In this paper, we present a Prototype Queue Network, which performs few-shot segmentation on multiclass in the images by aggregating binary classes into multiple classes. A prototype queue learning module is proposed to achieve multi-class segmentation by mining the relationship among features of different classes with queue and pseudo labels. In addition, a background latent class distribution refinement module is proposed to prevent the latent novel class in the background from being incorrectly predicted, which refines the boundary among different classes. Furthermore, we propose a two-steps segmentation module to optimize the process of extracting feature representation by adding progressive constraints, which can further improve the accuracy of segmentation. Experiments on the UDD and Vaihingen datasets demonstrate that our method achieves state-of-the-art performance.

show abstract

Mining Latent Classes for Few-shot Segmentation

Cited by 5 publications

References 41 publications

Few-shot Object Counting with Similarity-Aware Feature Enhancement

Few-shot Object Counting with Similarity-Aware Feature Enhancement

Cost Aggregation Is All You Need for Few-Shot Segmentation

Prototype Queue Learning for Multi-Class Few-Shot Semantic Segmentation

Contact Info

Product

Resources

About