Disentangle Your Dense Object Detector

Abstract

Deep learning-based dense object detectors have achieved great success in the past few years and have been applied to numerous multimedia applications such as video understanding. However, the current training pipeline for dense detectors is compromised to lots of conjunctions that may not hold. In this paper, we investigate three such important conjunctions: 1) only samples assigned as positive in classification head are used to train the regression head; 2) classification and regression share the same input feature and computational fields defined by the parallel head architecture; and 3) samples distributed in different feature pyramid layers are treated equally when computing the loss. We first carry out a series of pilot experiments to show disentangling such conjunctions can lead to persistent performance improvement. Then, based on these findings, we propose Disentangled Dense Object Detector (DDOD), in which simple and effective disentanglement mechanisms are designed and integrated into the current state-of-the-art dense object detectors. Extensive experiments on MS COCO benchmark show that our approach can lead to 2.0 mAP, 2.4 mAP and 2.2 mAP absolute improvements on RetinaNet, FCOS, and ATSS baselines with negligible extra overhead. Notably, our best model reaches 55.0 mAP on the COCO test-dev set and 93.5 AP on the hard subset of WIDER FACE, achieving new state-of-the-art performance on these two competitive benchmarks. Code is available at https://github.com/zehuichen123/DDOD. CCS CONCEPTS• Computing methodologies → Object detection.

Prediction-Guided Distillation for Dense Object Detection

“…Dense Object Detection. In the last few years, object detection has seen considerable gains in performance [3,4,7,12,16,19,25,26,27,31]. The demand for simple, fast models has brought one-stage detectors into the spotlight [7,31].…”

“…Advances in deep learning have led to considerable performance gains on object detection tasks [3,7,12,16,19,25,26,27,31]. However, detectors can be computationally expensive, making it challenging to deploy them on devices with limited resources.…”

“…However, which features should the student imitate? This question is of the utmost importance for dense object detectors [7,16,19,31,38,42] because, unlike two-stage detectors [3,12,27], they do not use the RoIAlign [12] operation to explicitly pool and align object features; instead they output predictions at every location of the feature map [17]. Recent work [30,35] has shown that distilling the whole feature map with equal weighting is sub-optimal because not all features carry equally meaningful information.…”

“…Our intuition is that these regions should be distilled because they hold the information that leads to the best predictions; other areas will be less informative and can contaminate the distillation process by distracting from more essential features. To achieve our goal, we adapt the quality measure from [7] to score teacher predictions. Then, we conduct an experiment to visualise how these scores are distributed and verify that high-scoring key predictive regions contribute the most to teacher performance.…”

“…Then, we conduct an experiment to visualise how these scores are distributed and verify that high-scoring key predictive regions contribute the most to teacher performance. Those findings drive us to propose a Prediction-Guided Weighting (PGW) module to weight the foreground distillation loss: inspired by recent progress in label assignment [7,21,32,38,42] for dense detectors, we sample the top-K positions with the highest quality score from the teacher model and use an adaptive Gaussian distribution to fit the key predictive regions for smoothly weighting the distillation loss. Fig.…”