Few-Shot Object Detection in Aerial Imagery Guided by Text-Modal Knowledge

Tang

IEEE Geosci. Remote Sensing Lett.

et al. 2022

Object detection is an essential and fundamental task in computer vision and satellite image processing. Existing deep learning methods have achieved impressive performance thanks to the availability of large-scale annotated datasets. Yet, in real-world applications the availability of labels is limited. In this context, few-shot object detection (FSOD) has emerged as a promising direction, which aims at enabling the model to detect novel objects with only few of them annotated. However, many existing FSOD algorithms overlook a critical issue: when an input image contains multiple novel objects and only a subset of them are annotated, the unlabeled objects will be considered as background during training. This can cause confusions and severely impact the model's ability to recall novel objects. To address this issue, we propose a self-training-based FSOD (ST-FSOD) approach, which incorporates the self-training mechanism into the few-shot fine-tuning process. ST-FSOD aims to enable the discovery of novel objects that are not annotated, and take them into account during training. On the one hand, we devise a two-branch region proposal networks (RPN) to separate the proposal extraction of base and novel objects, On another hand, we incorporate the student-teacher mechanism into RPN and the region of interest (RoI) head to include those highly confident yet unlabeled targets as pseudo labels. Experimental results demonstrate that our proposed method outperforms the state-ofthe-art in various FSOD settings by a large margin. The codes will be publicly available at https://github.com/zhu-xlab/ST-FSOD.

Section: Few-shot Object Detection In Remote Sensingmentioning

confidence: 99%

Few-Shot Object Detection of Remote Sensing Images via Two-Stage Fine-Tuning

Tang

IEEE Geosci. Remote Sensing Lett.

et al. 2022

IEEE Geosci. Remote Sensing Lett.

“…To address this limitation, people have been exploring alternative techniques that require fewer RSI annotations. These techniques include semisupervised learning [9,10,19], weakly-supervised learning [20,21,22,23,24], few-shot learning [25,26,27,28,29,30] and active learning [31,32,33],. These methods aim to minimize annotation expenses and speed up the labeling process, making them highly valuable for practical applications in remote sensing.…”

Section: Related Work a Label Efficient Object Detection In Rsismentioning

confidence: 99%

Feature Alignment FPN for Oriented Object Detection in Remote Sensing Images

et al. 2023

Annotating remote sensing images (RSIs) presents a notable challenge due to its labor-intensive nature. Semisupervised object detection (SSOD) methods tackle this issue by generating pseudo-labels for the unlabeled data, assuming that all classes found in the unlabeled dataset are also represented in the labeled data. However, real-world situations introduce the possibility of out-of-distribution (OOD) samples being mixed with in-distribution (ID) samples within the unlabeled dataset. In this paper, we delve into techniques for conducting SSOD directly on uncurated unlabeled data, which is termed Open-Set Semi-Supervised Object Detection (OSSOD). Our approach commences by employing labeled in-distribution data to dynamically construct a class-wise feature bank (CFB) that captures features specific to each class. Subsequently, we compare the features of predicted object bounding boxes with the corresponding entries in the CFB to calculate OOD scores. We design an adaptive threshold based on the statistical properties of the CFB, allowing us to filter out OOD samples effectively. The effectiveness of our proposed method is substantiated through extensive experiments on two widely used remote sensing object detection datasets: DIOR and DOTA. These experiments showcase the superior performance and efficacy of our approach for OSSOD on RSIs.

“…Few-shot feature reweighting (FSRW) [5], Meta region-convolutional neural network (R-CNN) [7] aggregate image or region of interest features with support features generated by meta-learners. On the one hand, feature aggregation [9,10] was used, with Xiao et al [9] using class representative features to guide network prediction and Lu et al [10] directing the model with text modality knowledge. On the other hand, feature space enhancement [11,12] was explored, with Zhang et al [11] proposing a metricbased discriminative loss based on transfer learning algorithms and Li et al [12] suggesting a class margin equilibrium method to optimise feature space division and new class reconstruction systematically.…”

Section: Introductionmentioning

confidence: 99%

“…Few‐shot feature reweighting (FSRW) [5], Meta region‐convolutional neural network (R‐CNN) [7] aggregate image or region of interest features with support features generated by meta‐learners. On the one hand, feature aggregation [9, 10] was used, with Xiao et al. [9] using class representative features to guide network prediction and Lu et al.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Few‐shot object detection based on global context and implicit knowledge decoupled head

Li,

Yang,

Liu

et al. 2024

IET Image Processing

The acquisition cycle of remote sensing images is slow, and the labelling process encounters challenges, which have become prominent with the rapid development of remote sensing image object detection research. Therefore, this article provides a way to make the model better capture the diversity and contextual relationships in the data, and solve this problem by more than just data augmentation. Specifically, this method is a few‐shot object detection method for remote sensing based on global context combined with implicit knowledge decoupled head (GC‐IKDH). This method first uses a segmentation strategy to convert high‐resolution images into low‐resolution images and expands the sample size through a generative model. Secondly, GC attention is introduced to generate a GC vector by weighting and averaging the information of each position in the input sequence, which helps the model better understand the semantics of the input sequence. Finally, an IKDH is added to improve the model head, which is used to learn specific features in the data so that the model can better handle the diversity in the data. Experimental results show that GC attention and IKDH boosting provide a good performance boost to the baseline model. Compared with other few‐shot samples, this method achieves state‐of‐the‐art performance under different shot settings and highly competitive results on two benchmark datasets (NWPU VHR‐10 and DIOR).