On the Texture Bias for Few-Shot CNN Segmentation

Azad, Reza; Fayjie, Abdur Razzaq; Kauffmann, Claude; Ayed, Ismail Ben; Pedersoli, Marco; Dolz, José

doi:10.1109/wacv48630.2021.00272

Cited by 55 publications

(30 citation statements)

References 33 publications

(59 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Figure 1, images from both support and query sets are first sent to the backbone network to extract features. Feature processing can be accomplished by generating weights for the classifier [33], [41], cosinesimilarity calculation [5], [45], [23], or convolutions [15], [54], [49], [9], [1] to generate the final prediction.…”

Section: Introductionmentioning

confidence: 99%

Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Tian

Zhao

Shu

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

346

410

View full text Add to dashboard Cite

State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results and hardly work on unseen classes without fine-tuning. Few-shot segmentation is thus proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information of training classes and spatial inconsistency between query and support targets. To alleviate these issues, we propose the Prior Guided Feature Enrichment Network (PFENet). It consists of novel designs of (1) a training-free prior mask generation method that not only retains generalization power but also improves model performance and (2) Feature Enrichment Module (FEM) that overcomes spatial inconsistency by adaptively enriching query features with support features and prior masks. Extensive experiments on PASCAL-5 i and COCO prove that the proposed prior generation method and FEM both improve the baseline method significantly. Our PFENet also outperforms state-of-the-art methods by a large margin without efficiency loss. It is surprising that our model even generalizes to cases without labeled support samples. Our code is available at https://github.com/Jia-Research-Lab/PFENet/.

show abstract

Section: Introductionmentioning

confidence: 99%

Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Tian

Zhao

Shu

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

346

410

View full text Add to dashboard Cite

show abstract

“…Feature extraction and mask preparation. First of all, both the query and support images are input to a pretrained feature extractor to obtain the collections of their multi-scale multi-layer feature maps {F q i,l } and {F s i,l }, where i is the scale of the feature maps with respect to the input images and i ∈ { 1 4 , 1 8 , 1 16 , 1 32 } for the feature extractor we use, and l ∈ {1, . .…”

Section: Dcama Framework For 1-shot Learningmentioning

confidence: 99%

Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation

Shi¹,

Wang²,

Lu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Research into Few-shot Semantic Segmentation (FSS) has attracted great attention, with the goal to segment target objects in a query image given only a few annotated support images of the target class. A key to this challenging task is to fully utilize the information in the support images by exploiting fine-grained correlations between the query and support images. However, most existing approaches either compressed the support information into a few class-wise prototypes, or used partial support information (e.g., only foreground) at the pixel level, causing non-negligible information loss. In this paper, we propose Dense pixel-wise Cross-query-and-support Attention weighted Mask Aggregation (DCAMA), where both foreground and background support information are fully exploited via multi-level pixel-wise correlations between paired query and support features. Implemented with the scaled dot-product attention in the Transformer architecture, DCAMA treats every query pixel as a token, computes its similarities with all support pixels, and predicts its segmentation label as an additive aggregation of all the support pixels' labels-weighted by the similarities. Based on the unique formulation of DCAMA, we further propose efficient and effective one-pass inference for n-shot segmentation, where pixels of all support images are collected for the mask aggregation at once. Experiments show that our DCAMA significantly advances the state of the art on standard FSS benchmarks of PASCAL-5 i , COCO-20 i , and FSS-1000, e.g., with 3.1%, 9.7%, and 3.6% absolute improvements in 1-shot mIoU over previous best records. Ablative studies also verify the design DCAMA.

show abstract

“…Few-shot Segmentation. The few-shot segmentation [39,59,51,41,47,8,26,48,3,33,22,1] has received considerable attention very recently. Inspired by the fewshot learning, Shaban et al [39] contributes the first fewshot segmentation work, whose segmentation parameters are generated by a conditioning branch on the supports.…”

Section: Related Workmentioning

confidence: 99%

Mining Latent Classes for Few-shot Segmentation

Yang¹,

Zhuo²,

Qi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Few-shot segmentation (FSS) aims to segment unseen classes given only a few annotated samples. Existing methods suffer the problem of feature undermining, i.e., potential novel classes are treated as background during training phase. Our method aims to alleviate this problem and enhance the feature embedding on latent novel classes. In our work, we propose a novel joint-training framework. Based on conventional episodic training on support-query pairs, we introduce an additional mining branch that exploits latent novel classes via transferable sub-clusters, and a new rectification technique on both background and foreground categories to enforce more stable prototypes. Over and above that, our transferable sub-cluster has the ability to leverage extra unlabeled data for further feature enhancement. Extensive experiments on two FSS benchmarks demonstrate that our method outperforms previous state-ofthe-art by a large margin of 3.7% mIOU on PASCAL-5 i and 7.0% mIOU on COCO-20 i at the cost of 74% fewer parameters and 2.5x faster inference speed.

show abstract

On the Texture Bias for Few-Shot CNN Segmentation

Cited by 55 publications

References 33 publications

Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation

Mining Latent Classes for Few-shot Segmentation

Contact Info

Product

Resources

About