Exploiting Saliency for Object Segmentation from Image Level Labels

Oh, Seong Joon; Benenson, Rodrigo; Khoreva, Anna; Akata, Zeynep; Fritz, Mario; Schiele, Bernt

doi:10.1109/cvpr.2017.535

Cited by 175 publications

(93 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…• In contrast to existing WSSS methods [18], [19], [20] that directly combine class-agnostic saliency maps with class-specific attention maps in user-defined ways, our approach fuses these two cues adaptively via the learning of the proposed self-attention network.…”

Section: Introductionmentioning

confidence: 99%

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

Yao

Gong

2020

IEEE Access

View full text Add to dashboard Cite

Weakly supervised semantic segmentation (WSSS) using only image-level labels can greatly reduce the annotation cost and therefore has attracted considerable research interest. However, its performance is still inferior to the fully supervised counterparts. To mitigate the performance gap, we propose a saliency guided self-attention network (SGAN) to address the WSSS problem. The introduced self-attention mechanism is able to capture rich and extensive contextual information but also may mis-spread attentions to unexpected regions. To enable this mechanism work effectively under weak supervision, we integrate class-agnostic saliency priors into the self-attention mechanism to prevent the attentions on discriminative parts from misspreading to the background. And meanwhile we utilize classspecific attention cues as an additional supervision for SGAN, which reduces the mis-spread of attentions in regions belonging to different foreground categories. The proposed approach is able to produce dense and accurate localization cues, by which the segmentation performance is boosted. Experiments on PASCAL VOC 2012 dataset show that the proposed approach outperforms all other state-of-the-art methods.Abstract-Weakly supervised semantic segmentation (WSSS) using only image-level labels can greatly reduce the annotation cost and therefore has attracted considerable research interest. However, its performance is still inferior to the fully supervised counterparts. To mitigate the performance gap, we propose a saliency guided self-attention network (SGAN) to address the WSSS problem. The introduced self-attention mechanism is able to capture rich and extensive contextual information but also may mis-spread attentions to unexpected regions. To enable this mechanism work effectively under weak supervision, we integrate class-agnostic saliency priors into the self-attention mechanism to prevent the attentions on discriminative parts from misspreading to the background. And meanwhile we utilize classspecific attention cues as an additional supervision for SGAN, which reduces the mis-spread of attentions in regions belonging to different foreground categories. The proposed approach is able to produce dense and accurate localization cues, by which the segmentation performance is boosted. Experiments on PASCAL VOC 2012 dataset show that the proposed approach outperforms all other state-of-the-art methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

Yao

Gong

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…(Joon Oh et al, 2017;Chaudhry et al, 2017) fuse saliency cues into weakly-supervised segmentation using different methods. (Joon Oh et al, 2017) use a existing saliency method to guide the process of training. (Chaudhry et al, 2017) exploit a novel saliency detection method, and combine saliency information with fully attention maps to segment input images.…”

Section: Weakly-supervised Image Segmentationmentioning

confidence: 99%

“…The salient foregrounds have a clear boundary and generally contain several salient objects, therefore could be utilized to generate objects locations from precise and reliable cues as Figure 1 shown. (Joon Oh et al, 2017) propose to utilize saliency to assist semantic segmentation. However, they assign salient regions a category randomly picked from image-level labels initially, and these salient regions will be assigned to a category if the category's seed touching the salient regions afterwards.…”

Section: Introductionmentioning

confidence: 99%

Saliency guided deep network for weakly-supervised image segmentation

Sun

2019

Pattern Recognition Letters

View full text Add to dashboard Cite

Weakly-supervised image segmentation is an important task in computer vision. A key problem is how to obtain high quality objects location from image-level category. Classification activation mapping is a common method which can be used to generate high-precise object location cues. However these location cues are generally very sparse and small such that they can not provide effective information for image segmentation. In this paper, we propose a saliency guided image segmentation network to resolve this problem. We employ a self-attention saliency method to generate subtle saliency maps, and render the location cues grow as seeds by seeded region growing method to expand pixel-level labels extent. In the process of seeds growing, we use the saliency values to weight the similarity between pixels to control the growing. Therefore saliency information could help generate discriminative object regions, and the effects of wrong salient pixels can be suppressed efficiently. Experimental results on a common segmentation dataset PASCAL VOC2012 demonstrate the effectiveness of our method.

show abstract

“…In spite of the progress that has been accomplished in realistic scene rendering and transfer learning approaches, there is still a significant domain discrepancy between real and synthetic imagery, especially in texturing. Weak supervision on the other hand tackles this issue by leveraging weak annotations with lower acquisition costs such as bounding boxes [10,21,26,27], scribble [22], points [3] or even image-level labels [1, 6, 19, 20, 27, 30-33, 35, 38, 41-43]. This enables a more cost-effective scaling of training datasets.…”

Section: Introductionmentioning

confidence: 99%

“…Comparison of our approach to state-of-the-art domain adaptation and fully-supervised methods. Results are obtained on the SUN RGB-D validation set 26…”

mentioning

confidence: 99%

What Synthesis Is Missing: Depth Adaptation Integrated With Weak Supervision for Indoor Scene Parsing

Liu

Shen

Klopp

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Scene Parsing is a crucial step to enable autonomous systems to understand and interact with their surroundings. Supervised deep learning methods have made great progress in solving scene parsing problems, however, come at the cost of laborious manual pixel-level annotation. To alleviate this effort synthetic data as well as weak supervision have both been investigated. Nonetheless, synthetically generated data still suffers from severe domain shift while weak labels are often imprecise. Moreover, most existing works for weakly supervised scene parsing are limited to salient foreground objects. The aim of this work is hence twofold: Exploit synthetic data where feasible and integrate weak supervision where necessary. More concretely, we address this goal by utilizing depth as transfer domain because its synthetic-to-real discrepancy is much lower than for color. At the same time, we perform weak localization from easily obtainable image level labels and integrate both using a novel contour-based scheme. Our approach is implemented as a teacher-student learning framework to solve the transfer learning problem by generating a pseudo ground truth. Using only depth-based adaptation, this approach already outperforms previous transfer learning approaches on the popular indoor scene parsing SUN RGB-D dataset. Our proposed two-stage integration more than halves the gap towards fully supervised methods when compared to previous state-of-the-art in transfer learning. Weak Localizations …… … Pseudo Labels Source Domain Synthetic Data Depth Target Domain Real Data Depth RGB Domain Adaptation Teacher …… Student Fusion Depth-based Predictions Teacher arXiv:1903.09781v1 [cs.CV]

show abstract

Exploiting Saliency for Object Segmentation from Image Level Labels

Cited by 175 publications

References 54 publications

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

Saliency Guided Self-Attention Network for Weakly and Semi-Supervised Semantic Segmentation

Saliency guided deep network for weakly-supervised image segmentation

What Synthesis Is Missing: Depth Adaptation Integrated With Weak Supervision for Indoor Scene Parsing

Contact Info

Product

Resources

About