FickleNet: Weakly and Semi-Supervised Semantic Image Segmentation Using Stochastic Inference

Abstract

The main obstacle to weakly supervised semantic image segmentation is the difficulty of obtaining pixel-level information from coarse image-level annotations. Most methods based on image-level annotations use localization maps obtained from the classifier, but these only focus on the small discriminative parts of objects and do not capture precise boundaries. FickleNet explores diverse combinations of locations on feature maps created by generic deep neural networks. It selects hidden units randomly and then uses them to obtain activation scores for image classification. Fick-leNet implicitly learns the coherence of each location in the feature maps, resulting in a localization map which identifies both discriminative and other parts of objects. The ensemble effects are obtained from a single network by selecting random hidden unit pairs, which means that a variety of localization maps are generated from a single image. Our approach does not require any additional training steps and only adds a simple layer to a standard convolutional neural network; nevertheless it outperforms recent comparable techniques on the Pascal VOC 2012 benchmark in both weakly and semi-supervised settings.

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

“…Since CAMs only focus on small discriminative regions which are too sparse to effectively supervise a segmentation model, various techniques such as adversarial erasing [12], [17], [21], [18] and region growing [13], [22] have been developed to expand sparse object seeds. Another research line introduces dilated convolutions of different rates [14], [16], [15], [23] to enlarge receptive fields in classification networks and aggregate multiple attention maps to achieve dense localization cues. In this work, we adopt the self-attention scheme to capture richer and more extensive contextual information to mine integral object seeds, and meanwhile leverage both class-agnostic saliency cues and class-specific attention cues to ensure the seeds accurate.…”

“…The last two pooling layers are removed in order to increase the resolution of the output feature maps. Note that, unlike previous works [14], [23], [15] that enlarge the dilation rate of convolution kernels in conv5 block, we avoid the usage of dilated convolution and instead use the self-attention module to capture more extensive contexts.…”

“…It may be hard for them to expand to non-discriminative regions if initial seeds are concentrated on extremely small discriminative parts. MDC [14], DFPN [16], and FickleNet [15] 5 12,000 iterations. The initial learning rate is 0.001 and it is decreased by a factor of 0.33 every 2,000 iterations.…”

Frame-to-Frame Aggregation of Active Regions in Web Videos for Weakly Supervised Semantic Segmentation

“…Weakly supervised semantic segmentation methods have evolved rapidly. Scribble supervision [39] can now achieve 97% of the performance of manually supervised semantic segmentation with the same backbone network, and even the weakest image-level supervision [24] can produce * Correspondence to: Sungroh Yoon <sryoon@snu.ac.kr>. Figure 1: (a) Our method discovers activated regions from each frame and aggregates them into a single frame using a warping technique based on optical flow.…”

“…However, the rate at which segmentation techniques relying on weak annotations are improving is declining rapidly. For instance, under imagelevel supervision, a 7.9% improvement on PASCAL VOC 2012 validation images [7] was achieved between 2016 and 2017 [23,2], 1.8% between 2017 and 2018 [46], but only 0.8% since then [24].…”