Differentiable Meta-learning Model for Few-shot Semantic Segmentation

Abstract

To address the annotation scarcity issue in some cases of semantic segmentation, there have been a few attempts to develop the segmentation model in the few-shot learning paradigm. However, most existing methods only focus on the traditional 1-way segmentation setting (i.e., one image only contains a single object). This is far away from practical semantic segmentation tasks where the K-way setting (K > 1) is usually required by performing the accurate multi-object segmentation. To deal with this issue, we formulate the fewshot semantic segmentation task as a learning-based pixel classification problem, and propose a novel framework called MetaSegNet based on meta-learning. In MetaSegNet, an architecture of embedding module consisting of the global and local feature branches is developed to extract the appropriate meta-knowledge for the few-shot segmentation. Moreover, we incorporate a linear model into MetaSegNet as a base learner to directly predict the label of each pixel for the multiobject segmentation. Furthermore, our MetaSegNet can be trained by the episodic training mechanism in an end-to-end manner from scratch. Experiments on two popular semantic segmentation datasets, i.e., PASCAL VOC and COCO, reveal the effectiveness of the proposed MetaSegNet in the K-way few-shot semantic segmentation task.

Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study

“…Maligo [143] ADVENT [120] Zou [121] Biasetton [144] Romera [145] xMUDA [146] Abdou [147] SqueezeSeg [11] SqueezeSegV2 [54] Shaban [148] CANet [149] Hu [150] Dong [151] Snell [153] PANet [154] Tian [155] Bucher [156] Barnes [124] Zhou [125] Migishima [126] Bruls [127] Saleh [157] Kolesnikov [158] Petrovai [159] Alonso [160] Mackowiak [161] Li [162] Zhang [163] Chen [164] Piewak [165] Varga [166] Luo [167] Maligo [143] VIPER [169] Krahenbuhl [170] SYNTHIA [171] VEIS [172] Fang [176][177]…”

“…Wang et al [154] also made use of learned prototypes to distinguish various semantic classes. Different from previous methods, Tian et al [155] employed an optimization-based method that leverages the linear classifier instead of nonlinear layers for training efficiency. Bucher et al [156] moved forward from few-shot to zero-shot semantic segmentation.…”

Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation

“…Exploiting network components such as attention modules [39,44] and graph networks [41,40], recent works boost segmentation accuracy [21] and enable FSS with coarse-level supervisions [42,22,24]. Exploiting learning-based optimization, [23,25] combine meta-learning with FSS. However, almost all of these methods assume abundant annotated (including weakly annotated) training data to be available, making them difficult to translate to segmentation scenarios in medical imaging.…”

“…However, training an existing few-shot semantic segmentation (FSS) model for medical imaging has not had much success in the past, as most of FSS methods rely on a large training dataset with many annotated training classes to avoid overfitting [14,15,16,17,18,19,20,21,17,22,23,24,25]. In order to bypass this unmet need of annotation, we propose to train an FSS model on unlabeled images instead via self-supervised learning, an unsupervised technique that learns generalizable image representations by solving a carefully designed task [26,27,28,29,30,31,32,33].…”

Objectness-Aware Few-Shot Semantic Segmentation

“…Prior works have explored many ways to solve few-shot segmentation, for example network parameter imprinting [28,30], meta-learning [34], prototype learning [7,36], etc. Recently, many works [25,16,46,44,43,24] tackle the problem by measuring feature similarity between the annotated example and every Fig.…”