EHSOD: CAM-Guided End-to-End Hybrid-Supervised Object Detection with Cascade Refinement

Abstract

Object detectors trained on fully-annotated data currently yield state of the art performance but require expensive manual annotations. On the other hand, weakly-supervised detectors have much lower performance and cannot be used reliably in a realistic setting. In this paper, we study the hybrid-supervised object detection problem, aiming to train a high quality detector with only a limited amount of fully-annotated data and fully exploiting cheap data with image-level labels. State of the art methods typically propose an iterative approach, alternating between generating pseudo-labels and updating a detector. This paradigm requires careful manual hyper-parameter tuning for mining good pseudo labels at each round and is quite time-consuming. To address these issues, we present EHSOD, an end-to-end hybrid-supervised object detection system which can be trained in one shot on both fully and weakly-annotated data. Specifically, based on a two-stage detector, we proposed two modules to fully utilize the information from both kinds of labels: 1) CAM-RPN module aims at finding foreground proposals guided by a class activation heat-map; 2) hybrid-supervised cascade module further refines the bounding-box position and classification with the help of an auxiliary head compatible with image-level data. Extensive experiments demonstrate the effectiveness of the proposed method and it achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data, e.g. 37.5% mAP on COCO. We will release the code and the trained models.

Many-shot from Low-shot: Learning to Annotate using Mixed Supervision for Object Detection

“…In this work, we instead propose a one-stage approach relying on an adaptive pool of annotations, updated dynamically as training progresses. EHSOD [7] and BAOD [16] focus on larger data regimes (e.g. 10% to 90%) and aim to reduce the data required to reach fully supervised performance using a cascaded MIL model and a student-teacher setup trained on weak and strong annotations, respectively.…”

“…Joint detection module. Similarly to the strategy proposed in [7], we combine a multiple instance learning (MIL) type image-level classification task with a fully supervised joint classification and regression task. Our joint detection module hence comprises three parallel, fully connected layers focusing on three different subtasks: proposal scoring, classification and regression (Fig.…”

“…Baselines: We evaluate our model with respect to two SOTA WSOD methods, PCL [19] and WSOD 2 [26], that were evaluated on both VOC07 and COCO14. We further compare to three MSOD approaches: the two level approach of BC-Net [15], end-to-end methods BAOD [16] and EHSOD [7]. To the best of our knowledge, these are the only three methods adopting mixed supervision.…”

“…To address the aforementioned challenges, we focus on a recent training paradigm relying on Mixed Supervision for Object Detection (MSOD) [15,7]. The distinction between this protocol and the previously introduced weak and low data settings is illustrated in Fig.…”

“…Fang et al . [7] propose a cascaded architecture yielding performance competitive with fully supervised counterparts yet using a significant fraction of the full training data to achieve comparable performance. Pan et al .…”