Learning Meta-class Memory for Few-Shot Semantic Segmentation

Wu, Zhonghua; Shi, Xiangxi; Lin, Guosheng; Cai, Jianfei

doi:10.1109/iccv48922.2021.00056

Cited by 87 publications

(16 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by PFENet [16,25], recent FSS models [29,30] usually leverage high-level features (e.g., conv5 of ResNet50) from the support and query set to generate the prior mask indicating the rough location of the target object in the query image. As this prior mask is usually obtained by element-to-element or square region-based matching between feature maps, a holistic context is not taken into account.…”

Section: Support Activation Modulementioning

confidence: 99%

Part-Aware Prototype Network for Few-Shot Semantic Segmentation

Liu

Zhang

et al. 2020

Computer Vision – ECCV 2020

219

195

View full text Add to dashboard Cite

The key challenge for few-shot semantic segmentation (FSS) is how to tailor a desirable interaction among support and query features and/or their prototypes, under the episodic training scenario. Most existing FSS methods implement such support/query interactions by solely leveraging plain operations -e.g., cosine similarity and feature concatenation -for segmenting the query objects. However, these interaction approaches usually cannot well capture the intrinsic object details in the query images that are widely encountered in FSS, e.g., if the query object to be segmented has holes and slots, inaccurate segmentation almost always happens. To this end, we propose a dynamic prototype convolution network (DPCN) to fully capture the aforementioned intrinsic details for accurate FSS. Specifically, in DPCN, a dynamic convolution module (DCM) is firstly proposed to generate dynamic kernels from support foreground, then information interaction is achieved by convolution operations over query features using these kernels. Moreover, we equip DPCN with a support activation module (SAM) and a feature filtering module (FFM) to generate pseudo mask and filter out background information for the query images, respectively. SAM and FFM together can mine enriched context information from the query features. Our DPCN is also flexible and efficient under the k-shot FSS setting. Extensive experiments on PASCAL-5 i and COCO-20 i show that DPCN yields superior performances under both 1-shot and 5-shot settings.

show abstract

Section: Support Activation Modulementioning

confidence: 99%

Part-Aware Prototype Network for Few-Shot Semantic Segmentation

Liu

Zhang

et al. 2020

Computer Vision – ECCV 2020

219

195

View full text Add to dashboard Cite

show abstract

“…Later on, Zhang et al [67] exploited the masked average pooling operation to obtain representative support features, which also served as the fundamental technology for subsequent works. More recently, some relevant research abandoned the training process of heavy backbone networks in favor of building powerful blocks on the fixed ones to boost performance, such as CANet [66], PFENet [51], ASGNet [23], SAGNN [60], and MM-Net [58].…”

Section: Related Workmentioning

confidence: 99%

Learning What Not to Segment: A New Perspective on Few-Shot Segmentation

Lang¹,

Chen²,

Tu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently few-shot segmentation (FSS) has been extensively developed. Most previous works strive to achieve generalization through the meta-learning framework derived from classification tasks; however, the trained models are biased towards the seen classes instead of being ideally class-agnostic, thus hindering the recognition of new concepts. This paper proposes a fresh and straightforward insight to alleviate the problem. Specifically, we apply an additional branch (base learner) to the conventional FSS model (meta learner) to explicitly identify the targets of base classes, i.e., the regions that do not need to be segmented. Then, the coarse results output by these two learners in parallel are adaptively integrated to yield precise segmentation prediction. Considering the sensitivity of meta learner, we further introduce an adjustment factor to estimate the scene differences between the input image pairs for facilitating the model ensemble forecasting. The substantial performance gains on PASCAL-5 i and COCO-20 i verify the effectiveness, and surprisingly, our versatile scheme sets a new state-of-the-art even with two plain learners. Moreover, in light of the unique nature of the proposed approach, we also extend it to a more realistic but challenging setting, i.e., generalized FSS, where the pixels of both base and novel classes are required to be determined. The source code is available at github.com/chunbolang/BAM.

show abstract

“…Few-shot semantic segmentation is pioneered by Shaban et al [69]. Later works have mainly adopted the metricbased mainstream paradigm [18] with various improvements, e.g., improving the matching procedure between support-query images with various attention mechanisms [70,54,83], better optimizations [94,53], memory modules [77,79], graph neural networks [78,74,87], learning-based classifiers [72,59], progressive matching [32,96], or other advanced techniques [86,52,61,48]. We are the first to perform self-support matching between query prototype and query features.…”

Section: Related Workmentioning

confidence: 99%

Self-Support Few-Shot Semantic Segmentation

Fan¹,

Pei²,

Tai³

et al. 2022

Preprint

View full text Add to dashboard Cite

Existing few-shot segmentation methods have achieved great progress based on the support-query matching framework. But they still heavily suffer from the limited coverage of intra-class variations from the few-shot supports provided. Motivated by the simple Gestalt principle that pixels belonging to the same object are more similar than those to different objects of same class, we propose a novel self-support matching strategy to alleviate this problem, which uses query prototypes to match query features, where the query prototypes are collected from high-confidence query predictions. This strategy can effectively capture the consistent underlying characteristics of the query objects, and thus fittingly match query features. We also propose an adaptive self-support background prototype generation module and self-support loss to further facilitate the self-support matching procedure. Our self-support network substantially improves the prototype quality, benefits more improvement from stronger backbones and more supports, and achieves SOTA on multiple datasets. Codes are at https://github.com/fanq15/SSP.

show abstract

Learning Meta-class Memory for Few-Shot Semantic Segmentation

Cited by 87 publications

References 18 publications

Part-Aware Prototype Network for Few-Shot Semantic Segmentation

Part-Aware Prototype Network for Few-Shot Semantic Segmentation

Learning What Not to Segment: A New Perspective on Few-Shot Segmentation

Self-Support Few-Shot Semantic Segmentation

Contact Info

Product

Resources

About