Prototype Mixture Models for Few-Shot Semantic Segmentation

Yang, Boyu; Chang, Liu; Li, Bohao; Jiao, J. B.; Ye, Qixiang

doi:10.1007/978-3-030-58598-3_45

Cited by 218 publications

(134 citation statements)

References 23 publications

Supporting

Mentioning

134

Contrasting

Order By: Relevance

“…Performance of our model on COCO-20 0,2,3 are better than PRMMs [38]. On COCO-20 3 , our model yields 31.37% mIoU in 1-shot setting, which is much better than PRMMs.…”

Section: Results and Comparison With Sotamentioning

confidence: 72%

“…Moreover, since segmentation is a many-to-many problem, Liu et al [24] decompose object into semantic consistent part-aware prototypes. Correlating diverse image regions with multiple prototypes with Prototype Mixture Models (PMMs), Yang et al [25] establish the element-to-element correspondence between images based on graph attention mechanism. Liu et al [26] try to make pixel-wise dense correlation by Harmonic Feature Activation(HFA), establishing robust correspondences between support and query images.…”

Section: Related Workmentioning

confidence: 99%

“…is not entirely same as PASCAL-5 i 's. [38] In each folder i of PASCAL-5 i dataset, we use 1000 support-query pairs of test images sampled from the selected five test classes. Similarly, in each folder of COCO-20 i dataset, we also sample 1000 support-query pairs of test images from the selected 20 test classes.…”

Section: B Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

Pyramid Co-Attention Compare Network for Few-Shot Segmentation

et al. 2021

View full text Add to dashboard Cite

Few-shot segmentation(FSS), which aims to extract never learned classes of objects from query images with a few annotated support samples, is a challenging problem especially in the cases that the appearance of objects in the support and the query images is significant different. Therefore, we propose a deep network called Pyramid Co-Attention Compare Network (PCCNet) to narrow the gap between them by introducing a Pyramid Co-attention Module (PCAM). PCAM acts as a task-specific transformer to transform the features of corresponding objects in query and support images into a space in which they are much closer by taking advantage of the underlying relation between query and support images. We also introduce a Prototypical Guide Module (PGM) which uses non-parametric metric learning to guide parametric metric learning so as to combine the advantages of them. In addition, a Superpixel Refine Module(SRM) is proposed to optimize the final output segmentation masks. Experiments conducted on Pascal-5 i shows that our PCCNet achieves a mean Intersection-over-Union(mIoU) score of 63.01% for 1shot segmentation and 64.57% for 5-shot segmentation, outperforming state-of-the-art methods by margin of 2.2% and 1.6%, respectively.

show abstract

“…Performance of our model on COCO-20 0,2,3 are better than PRMMs [38]. On COCO-20 3 , our model yields 31.37% mIoU in 1-shot setting, which is much better than PRMMs.…”

Section: Results and Comparison With Sotamentioning

confidence: 72%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Pyramid Co-Attention Compare Network for Few-Shot Segmentation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…With the development of deep learning, the metric-based FSL method has been widely applied in image classification [22,34,35], object detection [36,37], and other fields [38][39][40].…”

Section: Metric-based Few-shot Learningmentioning

confidence: 99%

AMN: Attention Metric Network for One-Shot Remote Sensing Image Scene Classification

Yang

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

In recent years, deep neural network (DNN) based scene classification methods have achieved promising performance. However, the data-driven training strategy requires a large number of labeled samples, making the DNN-based methods unable to solve the scene classification problem in the case of a small number of labeled images. As the number and variety of scene images continue to grow, the cost and difficulty of manual annotation also increase. Therefore, it is significant to deal with the scene classification problem with only a few labeled samples. In this paper, we propose an attention metric network (AMN) in the framework of the few-shot learning (FSL) to improve the performance of one-shot scene classification. AMN is composed of a self-attention embedding network (SAEN) and a cross-attention metric network (CAMN). In SAEN, we adopt the spatial attention and the channel attention of feature maps to obtain abundant features of scene images. In CAMN, we propose a novel cross-attention mechanism which can highlight the features that are more concerned about different categories, and improve the similarity measurement performance. A loss function combining mean square error (MSE) loss with multi-class N-pair loss is developed, which helps to promote the intra-class similarity and inter-class variance of embedding features, and also improve the similarity measurement results. Experiments on the NWPU-RESISC45 dataset and the RSD-WHU46 dataset demonstrate that our method achieves the state-of-the-art results on one-shot remote sensing image scene classification tasks.

show abstract

“…CANet reports multi-scale test performance. The single-scale test performance is reported by[19]. Mean-IoU of 2-way 1-shot and 5-shot segmentation on PASCAL-5 i .…”

mentioning

confidence: 99%

Few-Shot Semantic Segmentation via Prototype Augmentation with Image-Level Annotations

Lei

Zhang

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Despite the great progress made by deep neural networks in the semantic segmentation task, traditional neural-networkbased methods typically suffer from a shortage of large amounts of pixel-level annotations. Recent progress in fewshot semantic segmentation tackles the issue by only a few pixel-level annotated examples. However, these few-shot approaches cannot easily be applied to multi-way or weak annotation settings. In this paper, we advance the few-shot segmentation paradigm towards a scenario where image-level annotations are available to help the training process of a few pixel-level annotations. Our key idea is to learn a better prototype representation of the class by fusing the knowledge from the image-level labeled data. Specifically, we propose a new framework, called PAIA, to learn the class prototype representation in a metric space by integrating image-level annotations. Furthermore, by considering the uncertainty of pseudo-masks, a distilled soft masked average pooling strategy is designed to handle distractions in image-level annotations. Extensive empirical results on two datasets show superior performance of PAIA.

show abstract

Prototype Mixture Models for Few-Shot Semantic Segmentation

Cited by 218 publications

References 23 publications

Pyramid Co-Attention Compare Network for Few-Shot Segmentation

Pyramid Co-Attention Compare Network for Few-Shot Segmentation

AMN: Attention Metric Network for One-Shot Remote Sensing Image Scene Classification

Few-Shot Semantic Segmentation via Prototype Augmentation with Image-Level Annotations

Contact Info

Product

Resources

About