Learning Random-Walk Label Propagation for Weakly-Supervised Semantic Segmentation

Abstract

Large-scale training for semantic segmentation is challenging due to the expense of obtaining training data for this task relative to other vision tasks. We propose a novel training approach to address this difficulty. Given cheaplyobtained sparse image labelings, we propagate the sparse labels to produce guessed dense labelings. A standard CNN-based segmentation network is trained to mimic these labelings. The label-propagation process is defined via random-walk hitting probabilities, which leads to a differentiable parameterization with uncertainty estimates that are incorporated into our loss. We show that by learning the label-propagator jointly with the segmentation predictor, we are able to effectively learn semantic edges given no direct edge supervision. Experiments also show that training a segmentation network in this way outperforms the naive approach.

CoralSeg: Learning coral segmentation from sparse annotations

“…Table 13 compares our approach using different sparsity levels (different numbers of labeled pixels for the augmentation), with other recent label augmentation or propagation methods using the PASCAL VOC 2012 data set. One of these works (Vernaza & Chandraker, 2017) uses traces as the input of the augmentation process (Traces) as well as the learned boundaries (learned by a neural network) using the RAWKS algorithm (v1) to augment the trace sparse labeling. V2 indicates the evaluation is done on 94% of the pixels, where the model is confident enough.…”

“…The authors work with sparse convolutions to learn directly from sparse labeling and show successful results with levels of sparsity between 5% and 70%. Label propagation was also used in Vernaza and Chandraker (2017), who show how to simultaneously learn a label-propagator and the image segmentation model, both with deep learning architectures. This approach propagates the ground truth labels from a few traces to estimate the main object boundaries in the image and provides a label for each pixel.…”

End-To-End Learned Random Walker for Seeded Image Segmentation

“…[28] Figure 7. Qualitative comparison between Learned Random Walker presented here and first order approximation from [28]. We can observe that neither the sparse seeding nor our sampling strategy (sec.…”

“…Different approaches have been proposed for backpropagating gradient in Laplacian systems. Very recently, a first order approximation of the true derivative has been used in [28] for semantic segmentation. Their approach has the conspicuous advantage that it requires solving only one system of linear equations.…”

“…Their approach has the conspicuous advantage that it requires solving only one system of linear equations. We set up a simple experiment to show the differences between the Learned Random Walker backpropagation and the one presented in [28].…”