Restoring Negative Information in Few-Shot Object Detection

Abstract

Few-shot learning has recently emerged as a new challenge in the deep learning field: unlike conventional methods that train the deep neural networks (DNNs) with a large number of labeled data, it asks for the generalization of DNNs on new classes with few annotated samples. Recent advances in few-shot learning mainly focus on image classification while in this paper we focus on object detection. The initial explorations in few-shot object detection tend to simulate a classification scenario by using the positive proposals in images with respect to certain object class while discarding the negative proposals of that class. Negatives, especially hard negatives, however, are essential to the embedding space learning in few-shot object detection. In this paper, we restore the negative information in few-shot object detection by introducing a new negative-and positive-representative based metric learning framework and a new inference scheme with negative and positive representatives. We build our work on a recent few-shot pipeline RepMet [1] with several new modules to encode negative information for both training and testing. Extensive experiments on ImageNet-LOC and PASCAL VOC show our method substantially improves the state-of-the-art few-shot object detection solutions. Our code is available at https://github.com/yang-yk/NP-RepMet.

Bi-path Combination YOLO for Real-time Few-shot Object Detection

“…Motivated from the effectiveness in image classification by meta-learning based approaches. Some meta-detectors are proposed and achieve good detection performance [11,12,13,14,15]. For example, FSRW [11] is proposed to extract a few sample features and reweight to query features in channel dimension.…”

“…To ensure the fairness of comparison, the data and class splits adopted are the same as the settings from previous works [11,12,13,14,16,17,18,19], i.e., the overall categories in PASCAL VOC are divided into 15 base and 5 novel classes with three different splits. For MS COCO, all 20 categories in PASCAL VOC can be seen as novel classes and the rest of 60 categories are base classes.…”

“…To better adapt the quantitative limitation of instances, there are currently few-shot object detectors based on two types of mainstream thinking, i.e., meta-learning and transfer learning. For the meta-learning based methods [11,12,13,14,15], the aim is to build the feature relevance between the query image and the few support samples. Although the detection performance gets improved, the computational complexity also increases severely as the feature extractor in the few-shot branch, the relation builder between input features and few support features, and the number of object categories.…”

Few-Shot Object Detection in Remote Sensing Image Interpretation: Opportunities and Challenges

“…Increasing positive instances is feasible, and so do utilize negative proposals [76]. Negative proposals are generated when regressing bounding boxes, which are incorrect proposals including partial foreground objects or total background.…”

“…The principle of classification in detection pipeline is to achieve separability between embedding vectors of samples from different categories in a high-dimensional embedding space, where the embedding vectors are usually generated by positive proposals [77][78][79]. In addition, NP-Repmet [76] figures out that the introduction of negative proposals could enhance the separability of all classes in the embedding space. Proposals belonging to a certain category should be both close to the positive representatives and away from the negative representatives of that class.…”