Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation

Abstract

Despite the remarkable progress, weakly supervised segmentation approaches are still inferior to their fully supervised counterparts. We obverse the performance gap mainly comes from their limitation on learning to produce highquality dense object localization maps from image-level supervision. To mitigate such a gap, we revisit the dilated convolution [1] and reveal how it can be utilized in a novel way to effectively overcome this critical limitation of weakly supervised segmentation approaches. Specifically, we find that varying dilation rates can effectively enlarge the receptive fields of convolutional kernels and more importantly transfer the surrounding discriminative information to nondiscriminative object regions, promoting the emergence of these regions in the object localization maps. Then, we design a generic classification network equipped with convolutional blocks of different dilated rates. It can produce dense and reliable object localization maps and effectively benefit both weakly-and semi-supervised semantic segmentation. Despite the apparent simplicity, our proposed approach obtains superior performance over state-of-the-arts. In particular, it achieves 60.8% and 67.6% mIoU scores on Pascal VOC 2012 test set in weakly-(only image-level labels are available) and semi-(1,464 segmentation masks are available) supervised settings, which are the new stateof-the-arts.

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

“…Since CAMs only focus on small discriminative regions which are too sparse to effectively supervise a segmentation model, various techniques such as adversarial erasing [12], [17], [21], [18] and region growing [13], [22] have been developed to expand sparse object seeds. Another research line introduces dilated convolutions of different rates [14], [16], [15], [23] to enlarge receptive fields in classification networks and aggregate multiple attention maps to achieve dense localization cues. In this work, we adopt the self-attention scheme to capture richer and more extensive contextual information to mine integral object seeds, and meanwhile leverage both class-agnostic saliency cues and class-specific attention cues to ensure the seeds accurate.…”

“…The last two pooling layers are removed in order to increase the resolution of the output feature maps. Note that, unlike previous works [14], [23], [15] that enlarge the dilation rate of convolution kernels in conv5 block, we avoid the usage of dilated convolution and instead use the self-attention module to capture more extensive contexts.…”

“…It may be hard for them to expand to non-discriminative regions if initial seeds are concentrated on extremely small discriminative parts. MDC [14], DFPN [16], and FickleNet [15] 5 12,000 iterations. The initial learning rate is 0.001 and it is decreased by a factor of 0.33 every 2,000 iterations.…”

Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning

“…Recent methods (Kolesnikov and Lampert [2016]; Wei et al [2017a]; Wang et al [2018b]; Ahn and Kwak [2018]; Huang et al [2018]; Wei et al [2018]) tackle weakly-supervised semantic segmentation by a twostage procedure, which first generates initial object labels with class activation maps (Zhou et al [2016]), and then trains segmentation networks based on the response maps. Kolesnikov and Lampert [2016] present an end-to-end framework with three modules (seed, expand and constrain) as loss functions, and the class activation maps are used as supervisory signals.…”

“…However, fully-supervised methods require a large amount of pixel-wise annotations, which is time-consuming and expensive. To make semantic segmentation more practical, a number of weakly-supervised methods have been proposed in recent years based on partial information of each image, such as bounding boxes (Dai et al [2015]; Khoreva et al [2017]), scribbles (Lin et al [2016]), points (Bearman et al [2016]), and even class labels (Pathak et al [2015]; Wang et al [2018b]; Ahn and Kwak [2018]; Huang et al [2018]; Wei et al [2018]). In this paper, we present a weakly-supervised semantic segmentation algorithm based only on class labels of an image.…”