OMNIA Faster R-CNN: Detection in the wild through dataset merging and soft distillation

Abstract

Object detectors tend to perform poorly in new or open domains, and require exhaustive yet costly annotations from fully labeled datasets. We aim at benefiting from several datasets with different categories but without additional labelling, not only to increase the number of categories detected, but also to take advantage from transfer learning and to enhance domain independence.Our dataset merging procedure starts with training several initial Faster R-CNN on the different datasets while considering the complementary datasets' images for domain adaptation. Similarly to self-training methods, the predictions of these initial detectors mitigate the missing annotations on the complementary datasets. The final OM-NIA Faster R-CNN is trained with all categories on the union of the datasets enriched by predictions. The joint training handles unsafe targets with a new classification loss called SoftSig in a softly supervised way.Experimental results show that in the case of fashion detection for images in the wild, merging Modanet with COCO increases the final performance from 45.5% to 57.4% in mAP. Applying our soft distillation to the task of detection with domain shift between GTA and Cityscapes enables to beat the state-of-the-art by 5.3 points. Our methodology could unlock object detection for real-world applications without immense datasets.

Model Composition: Can Multiple Neural Networks Be Combined into a Single Network Using Only Unlabeled Data?

“…Incremental Learning (IL): Gradually adding new categories while trying to limit the catastrophic forgetting [22]. Dataset Merging: [23] Dataset merging is closest work to our study. It proposes to combine datasets by filling the missing annotations of non-overlapping categories.…”

Active learning using weakly supervised signals for quality inspection

“…This has been done extensively in semi-supervised settings with the use of pseudo-labels. 20 Similarly, the authors of OMNIA 21 enable merging of datasets with different target classes using model predictions as a weakly supervised training signal. Our method of partial backpropagation takes inspiration from the literature.…”

“…23 Close to our work, the idea has been applied to histopathology to solve spatially partial segmentation annotations, 24 as well as in OMNIA. 21 Finally, we make use of domain adaptation techniques to alleviate any data distribution shift impact on performances. We refer the reader to the review on domain adaptation for segmentation by Toldo, Marco, et al 25 Domain adaptation techniques often use a regularization term preventing the network to learn different representation per input space.…”

“…Another way to use the original baseline model to reduce false positives on the new data is to use its highconfidence predicted areas as ground truth. The intuition behind the method takes inspiration from OMNIA, 21 and works as follows. If the baseline model predicts an area as healthy with a sufficiently high confidence (determined with a fixed threshold), the area can then safely be considered as truly healthy, and used for backpropagation during the training of the new model.…”

“…Thus, we experiment four techniques to improve the method and effectively reduce false positives: transfer learning (using the previous model state, i.e. re-training the model from the baseline model), OMNIA 21 -like (using the previous model confident background predictions as ground truth), taking advantage of additional prior knowledge on the healthyness of the mentored images (healthy-flagged mentored images are treated as fully annotated), and domain adaptation through domain-adversarial training (DANN). 2 We show that all four techniques can improve mAP on the mentored evaluation set by around 0.20 points compared to the baseline (Table 1), which is significantly better than our first mentored model.…”