Experience feedback using Representation Learning for Few-Shot Object Detection on Aerial Images

Abstract

This paper proposes a few-shot method based on Faster R-CNN and representation learning for object detection in aerial images. The two classification branches of Faster R-CNN are replaced by prototypical networks for online adaptation to new classes. These networks produce embeddings vectors for each generated box, which are then compared with class prototypes. The distance between an embedding and a prototype determines the corresponding classification score. The resulting networks are trained in an episodic manner. A new detection task is randomly sampled at each epoch, consisting in detecting only a subset of the classes annotated in the dataset. This training strategy encourages the network to adapt to new classes as it would at test time. In addition, several ideas are explored to improve the proposed method such as a hard negative examples mining strategy and self-supervised clustering for background objects. The performance of our method is assessed on DOTA, a large-scale remote sensing images dataset. The experiments conducted provide a broader understanding of the capabilities of representation learning. It highlights in particular some intrinsic weaknesses for the few-shot object detection task. Finally, some suggestions and perspectives are formulated according to these insights.

A Unified Framework for Attention-Based Few-Shot Object Detection

“…These vectors are used in the classification head of the detector as class prototypes. The difference is that the vectors are learned through training and not simply computed from examples like in Prototypical Faster R-CNN [10]. The authors of this method proposed to embed a vanilla prototypical network into Faster R-CNN not only in the detection head but in the RPN as well.…”

“…Indeed, there exist some papers that use metric learning to solve FSOD (see e.g. [9][10][11]). Instead of combining query and support features, a generic embedding function is learned.…”

Few-Shot Object Detection in Remote Sensing Image Interpretation: Opportunities and Challenges

“…Negative proposals are generated when regressing bounding boxes, which are incorrect proposals including partial foreground objects or total background. The principle of classification in detection pipeline is to achieve separability between embedding vectors of samples from different categories in a high-dimensional embedding space, where the embedding vectors are usually generated by positive proposals [77][78][79]. In addition, NP-Repmet [76] figures out that the introduction of negative proposals could enhance the separability of all classes in the embedding space.…”

“…To our best knowledge, instead of designing a distinctly new detection framework from scratch, FSOD appends the process of meta knowledge extraction and sharing on classic deep learning object detection baselines, such as two-stage Faster R-CNN [11,16,17,72,73,78,[83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98], one-stage YOLO [10,74,99], CenterNet [100,101], and Vision Transformer [102]. The numerous published reports indicate that the two-stage network is more favored, and the two-stage network has more advantages because of its higher detection accuracy, more interpretive, and extensible network structure.…”

“…Limited by a few shots from novel classes, it is impossible to completely reconstruct a representative space, which will lead to a serious overfitting problem. Thus, the novel classification space is built on the basis of base representation space [16,17,78,98].…”