Learning Meta-class Memory for Few-Shot Semantic Segmentation

Abstract

Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foregroundbackground segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the k-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-5 i and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5% mIoU on the COCO dataset in 1-shot setting, which is 5.1% higher than the previous state-of-theart.

Cross-Image Region Mining with Region Prototypical Network for Weakly Supervised Segmentation

“…Currently, state-of-the-art methods handle image semantic segmentation as a dense prediction task and adopt fully convolutional networks to make predictions [26], [27]. To make pixel-level dense predictions, encoder-decoder structures [28], [29], [30], [31], [17], [32], [33], [34], [35], [36], [37], [38], [39] are widely used to reconstruct high-resolution prediction maps. Typically an encoder gradually downsamples the feature maps, aiming to acquire large field-of-view and capture the semantic object information.…”