Bootstrapping the Performance of Webly Supervised Semantic Segmentation

Shen, Tong; Lin, Guosheng; Shen, Chunhua; Reid, Ian

doi:10.1109/cvpr.2018.00148

Cited by 71 publications

(80 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…CRF plays an important role to improve the accuracy of weakly supervised segmentation. Furthermore, various researches employed the CRF for refining the coarse segmentation masks [37,34,33,17,43,42,11,36]. However, CRF does not guarantee any improvement in the mean intersection over union (IoU) score, and it often degrades the segmentation masks and the scores.…”

Section: Related Workmentioning

confidence: 99%

“…Motion segmentation of videos as additional training information for weakly supervised segmentation has also been proposed [39,10]. There are also reports that web images were helpful for improving the weakly supervised segmentation accuracy [30,43,16,36]. Recently, fully supervised saliency methods are being widely used for detecting the background regions, and certain researchers have reported that this approach could substantially boost performance [35,42,44,12,11,41,4].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation

Shimoda

Yanai

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

133

View full text Add to dashboard Cite

To minimize the annotation costs associated with the training of semantic segmentation models, researchers have extensively investigated weakly-supervised segmentation approaches. In the current weakly-supervised segmentation methods, the most widely adopted approach is based on visualization. However, the visualization results are not generally equal to semantic segmentation. Therefore, to perform accurate semantic segmentation under the weakly supervised condition, it is necessary to consider the mapping functions that convert the visualization results into semantic segmentation. For such mapping functions, the conditional random field and iterative re-training using the outputs of a segmentation model are usually used. However, these methods do not always guarantee improvements in accuracy; therefore, if we apply these mapping functions iteratively multiple times, eventually the accuracy will not improve or will decrease.In this paper, to make the most of such mapping functions, we assume that the results of the mapping function include noise, and we improve the accuracy by removing noise. To achieve our aim, we propose the self-supervised difference detection module, which estimates noise from the results of the mapping functions by predicting the difference between the segmentation masks before and after the mapping. We verified the effectiveness of the proposed method by performing experiments on the PASCAL Visual Object Classes 2012 dataset, and we achieved 64.9% in the val set and 65.5% in the test set. Both of the results become new state-of-the-art under the same setting of weakly supervised semantic segmentation.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation

Shimoda

Yanai

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

133

View full text Add to dashboard Cite

show abstract

“…Weakly supervised learning has been used in many areas of computer vision, e.g., image recognition [30], segmentation [16,32] and scene text detection [38,15], etc. Weakly supervised scene text detection methods [38,15] trained a supervised character-based detector using character-level bounding box and refined this model with word-level bounding box to improve the accuracy.…”

Section: Weakly and Partially Supervised Learningmentioning

confidence: 99%

“…Therefore, we propose a text reading model with partially supervised learning, which enables to train full and weak annotations in a unified model to keep it simple yet effective. Existing weakly supervised learning methods for image classification [30], segmentation [16,32] and text detection tasks [38] mainly focus on using image-level or bounding box as weak annotations, while end-to-end text reading aims to translate an image into a set of sequences, which is more challenging to solve. To address this problem, given only weak annotations in text labels without locations, we develop an Online Proposal Matching module (OPM) to select the best matched text proposals from weakly labeled images.…”

Section: Partially Supervised Learningmentioning

confidence: 99%

Chinese Street View Text: Large-Scale Chinese Text Reading With Partially Supervised Learning

Sun

Liu

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Most existing text reading benchmarks make it difficult to evaluate the performance of more advanced deep learning models in large vocabularies due to the limited amount of training data. To address this issue, we introduce a new large-scale text reading benchmark dataset named Chinese Street View Text (C-SVT) with 430, 000 street view images, which is at least 14 times as large as the existing Chinese text reading benchmarks. To recognize Chinese text in the wild while keeping large-scale datasets labeling cost-effective, we propose to annotate one part of the C-SVT dataset (30,000 images) in locations and text labels as full annotations and add 400, 000 more images, where only the corresponding text-of-interest in the regions is given as weak annotations. To exploit the rich information from the weakly annotated data, we design a text reading network in a partially supervised learning framework, which enables to localize and recognize text, learn from fully and weakly annotated data simultaneously. To localize the best matched text proposals from weakly labeled images, we propose an online proposal matching module incorporated in the whole model, spotting the keyword regions by sharing parameters for end-to-end training. Compared with fully supervised training algorithms, this model can improve the end-to-end recognition performance remarkably by 4.03% in F-score at the same labeling cost. The proposed model can also achieve state-of-the-art results on the ICDAR 2017-RCTW dataset, which demonstrates the effectiveness of the proposed partially supervised learning framework.

show abstract

“…Combining weak annotations with small amounts of high-quality dense annotation is another strategy for reducing cost [9,26]. [52] proposes a two-stage approach where image-level class labels are automatically converted into pixel-level masks which are used to train a semantic segmentation network. We find a small number of sub-image block annotations is a competitive form of weak supervision (sec.…”

Section: Related Workmentioning

confidence: 99%

Block Annotation: Better Image Annotation With Sub-Image Decomposition

Lin

Upchurch

Bala

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Image datasets with high-quality pixel-level annotations are valuable for semantic segmentation: labelling every pixel in an image ensures that rare classes and small objects are annotated. However, full-image annotations are expensive, with experts spending up to 90 minutes per image. We propose block sub-image annotation as a replacement for full-image annotation. Despite the attention cost of frequent task switching, we find that block annotations can be crowdsourced at higher quality compared to full-image annotation with equal monetary cost using existing annotation tools developed for full-image annotation. Surprisingly, we find that 50% pixels annotated with blocks allows semantic segmentation to achieve equivalent performance to 100% pixels annotated. Furthermore, as little as 12% of pixels annotated allows performance as high as 98% of the performance with dense annotation. In weakly-supervised settings, block annotation outperforms existing methods by 3-4% (absolute) given equivalent annotation time. To recover the necessary global structure for applications such as characterizing spatial context and affordance relationships, we propose an effective method to inpaint block-annotated images with high-quality labels without additional human effort. As such, fewer annotations can also be used for these applications compared to full-image annotation.

show abstract

Bootstrapping the Performance of Webly Supervised Semantic Segmentation

Abstract: Bootstrapping the performance of webly supervised semantic segmentation

Cited by 71 publications

References 31 publications

Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation

Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation

Chinese Street View Text: Large-Scale Chinese Text Reading With Partially Supervised Learning

Block Annotation: Better Image Annotation With Sub-Image Decomposition

Contact Info

Product

Resources

About