PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment

Arnet: Attention-Based Refinement Network for Few-Shot Semantic Segmentation

“…It requires to perform pixel-wise segmentation from few annotated images available. Previous approaches adopt two-brach architectures to align support images with query images [Shaban et al, 2017;Zhang et al, 2018;Wang et al, 2019;Hu et al, 2019]. OSLSM [Shaban et al, 2017] was proposed for one-shot semantic segmentation which uses support branch to learn parameters for the logistic regression layer of the query branch.…”

“…Similar idea of weight learning was adopted in AMP [Siam et al, 2019] which uses the support feature after masked average pooling to imprint the 1x1 convolutional filters of the final segmentation layer of the query image. Prototype learning based methods [Dong and Xing, 2018;Wang et al, 2019] were recently proposed for few-shot segmentation. PANet [Wang et al, 2019] viewed the averaged features as prototypes and trained the feature extractor by adding alignment constraints between support prototypes and query prototypes.…”

“…Prototype learning based methods [Dong and Xing, 2018;Wang et al, 2019] were recently proposed for few-shot segmentation. PANet [Wang et al, 2019] viewed the averaged features as prototypes and trained the feature extractor by adding alignment constraints between support prototypes and query prototypes. Our method is similar to PANet but we fur-ther propose the prototype refinement approach to make prototypes more suitable to represent the target classes in lowdata regimes.…”

“…In a N -way K-shot segmentation task, we target to segment the query images given K annotated support images from each of the N classes. Prototype based methods are widely used in handling few-shot tasks like classification [Snell et al, 2017] and segmentation [Dong and Xing, 2018;Wang et al, 2019]. For few-shot segmentation, prototype based methods treat it as a classification problem that classifies each pixel to the class of the nearest prototype.…”

“…For few-shot segmentation, prototype based methods treat it as a classification problem that classifies each pixel to the class of the nearest prototype. Existing methods [Dong and Xing, 2018;Wang et al, 2019] put emphasis on learning representative prototypes which are closer to the pixels belonging to the same semantic class in the feature space. However, in few-shot segmentation, prototypes directly extracted from the support images are usually biased to represent the semantic classes in the query images, due to the data scarcity and large intra-class variance.…”