Active and semi-supervised learning for object detection with imperfect data

Rhee, Phill Kyu; Erdenee, Enkhbayar; Kyun, Shin Dong; Ahmed, Minhaz Uddin; Jin, Shiwei

doi:10.1016/j.cogsys.2017.05.006

Cited by 47 publications

(31 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding object detection via AL, Vijayanarasimhan et al [13] proposed to refine part-based object detectors by actively requesting crowd-sourced annotations of images crawled from the Web. Rhee et al [14] presented a semi-supervised AL method to improve object detection performance by leveraging the concept of diversity adopted from the AL paradigm.…”

Section: Related Workmentioning

confidence: 99%

“…Specifically, the minority of unlabeled samples with low prediction confidences (i.e., high uncertainties), together with other informative criteria such as diversity and density, are generally treated as good candidates for model retraining. Recently, several AL-based approaches [13], [14] have been proposed for object detection in a semi-supervised or weakly supervised manner. However, these approaches usually ignore the fact that the remaining majority samples (e.g., those with low uncertainty and high confidence) are also valuable for improving the detection performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cost-Effective Object Detection: Active Sample Mining With Switchable Selection Criteria

Wang

Yan

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Though quite challenging, leveraging large-scale unlabeled or partially labeled data in learning systems (e.g., model/classifier training) has attracted increasing attentions due to its fundamental importance. To address this problem, many active learning (AL) methods have been proposed that employ up-to-date detectors to retrieve representative minority samples according to predefined confidence or uncertainty thresholds. However, these AL methods cause the detectors to ignore the remaining majority samples (i.e., those with low uncertainty or high prediction confidence). In this paper, by developing a principled active sample mining (ASM) framework, we demonstrate that cost-effective mining samples from these unlabeled majority data are a key to train more powerful object detectors while minimizing user effort. Specifically, our ASM framework involves a switchable sample selection mechanism for determining whether an unlabeled sample should be manually annotated via AL or automatically pseudolabeled via a novel self-learning process. The proposed process can be compatible with mini-batch-based training (i.e., using a batch of unlabeled or partially labeled data as a one-time input) for object detection. In this process, the detector, such as a deep neural network, is first applied to the unlabeled samples (i.e., object proposals) to estimate their labels and output the corresponding prediction confidences. Then, our ASM framework is used to select a number of samples and assign pseudolabels to them. These labels are specific to each learning batch based on the confidence levels and additional constraints introduced by the AL process and will be discarded afterward. Then, these temporarily labeled samples are employed for network fine-tuning. In addition, a few samples with low-confidence predictions are selected and annotated via AL. Notably, our method is suitable for object categories that are not seen in the unlabeled data during the learning process. Extensive experiments on two public benchmarks (i.e., the PASCAL VOC 2007/2012 data sets) clearly demonstrate that our ASM framework can achieve performance comparable to that of the alternative methods but with significantly fewer annotations.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Cost-Effective Object Detection: Active Sample Mining With Switchable Selection Criteria

Wang

Yan

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…Vijayanarasimhan et al [36] proposed a novel active learning approach in crowdsourcing settings for live learning of object detectors, in which the system autonomously identifies the most uncertain instances via a hashing based solution. Rhee et al [29] proposed to improve object detection performance by leveraging a collaborative sampling strategy, which integrates the uncertainty and diversity criteria from the AL and the feature similarity measurement of semi-supervised learning philosophy. However, these mentioned AL approaches usually emphasize those low-confidence samples (e.g., uncertain or diverse samples) while ignoring the rest majority of high-confidence samples.…”

Section: Related Workmentioning

confidence: 99%

Towards Human-Machine Cooperation: Self-Supervised Sample Mining for Object Detection

Wang

Yan

Zhang

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

106

View full text Add to dashboard Cite

Though quite challenging, leveraging large-scale unlabeled or partially labeled images in a cost-effective way has increasingly attracted interests for its great importance to computer vision. To tackle this problem, many Active Learning (AL) methods have been developed. However, these methods mainly define their sample selection criteria within a single image context, leading to the suboptimal robustness and impractical solution for large-scale object detection. In this paper, aiming to remedy the drawbacks of existing AL methods, we present a principled Self-supervised Sample Mining (SSM) process accounting for the real challenges in object detection. Specifically, our SSM process concentrates on automatically discovering and pseudo-labeling reliable region proposals for enhancing the object detector via the introduced cross image validation, i.e., pasting these proposals into different labeled images to comprehensively measure their values under different image contexts. By resorting to the SSM process, we propose a new AL framework for gradually incorporating unlabeled or partially labeled data into the model learning while minimizing the annotating effort of users. Extensive experiments on two public benchmarks clearly demonstrate our proposed framework can achieve the comparable performance to the state-of-the-art methods with significantly fewer annotations.

show abstract

“…The framework begins with a very small number of labeled data samples and incrementally learns by using unlabeled data samples in an open-set setting. The main components of the proposed framework, shown in Figure 1 are the dynamic HFM, the outlier-detection algorithm, the collaborative sampling (CS) algorithm [16], and incremental ASSL [14]. We used the initial CNN model, which was pretrained by using labeled data samples, i.e., PASCAL VOC 2007 and the 2012 trainval dataset [17].…”

Section: System Overviewmentioning

confidence: 99%

“…The proposed framework can adaptively learn both closed-set classes for performance improvement and open-set object classes using unlabeled data samples. Our method combines the incremental open-set aware active semi-supervised learning (ASSL) [16] and the dynamic hierarchical feature model (HFM) update algorithm for effectively grouping unseen objects together. In real-world scenarios, an object detection system should handle noisy and open-set data.…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Open-Set Object Detection in Unseen Data

Kim

Shin²,

Ahmed

et al. 2019

Symmetry

Self Cite

View full text Add to dashboard Cite

In this paper, we propose an open-set object detection framework based on a dynamic hierarchical structure with incremental learning capabilities for unseen object classes. We were motivated by the observation that deep features extracted from visual objects show a strong hierarchical clustering property. The hierarchical feature model (HFM) was used to learn a new object class by using collaborative sampling (CS), and open-set-aware active semi-supervised learning (ASSL) algorithms. We divided object proposals into superclasses by using the agglomerative clustering algorithm. Data samples in each superclass node were classified into multiple augmented class nodes instead of directly associating with regular object classes. One or more augmented class nodes are related to a regular object class, and each augmented class has only one superclass. Object proposals from inexperienced data distribution are assigned to an augmented class node. Dynamic HFM nodes in the decision path are assembled to constitute an ensemble prediction, and the new augmented object is associated with a new regular object class. Our experimental results showed that the proposed method uses standard benchmark datasets such as PASCAL VOC, MS COCO, ILSVRC DET, and local datasets to perform better than state-of-the-art techniques.

show abstract

Active and semi-supervised learning for object detection with imperfect data

Cited by 47 publications

References 44 publications

Cost-Effective Object Detection: Active Sample Mining With Switchable Selection Criteria

Cost-Effective Object Detection: Active Sample Mining With Switchable Selection Criteria

Towards Human-Machine Cooperation: Self-Supervised Sample Mining for Object Detection

Hierarchical Open-Set Object Detection in Unseen Data

Contact Info

Product

Resources

About