UFPMP-Det: Toward Accurate and Efficient Object Detection on Drone Imagery

Abstract

This paper proposes a novel approach to object detection on drone imagery, namely Multi-Proxy Detection Network with Unified Foreground Packing (UFPMP-Det). To deal with the numerous instances of very small scales, different from the common solution that divides the high-resolution input image into quite a number of chips with low foreground ratios to perform detection on them each, the Unified Foreground Packing (UFP) module is designed, where the sub-regions given by a coarse detector are initially merged through clustering to suppress background and the resulting ones are subsequently packed into a mosaic for a single inference, thus significantly reducing overall time cost. Furthermore, to address the more serious confusion between inter-class similarities and intraclass variations of instances, which deteriorates detection performance but is rarely discussed, the Multi-Proxy Detection Network (MP-Det) is presented to model object distributions in a fine-grained manner by employing multiple proxy learning, and the proxies are enforced to be diverse by minimizing a Bag-of-Instance-Words (BoIW) guided optimal transport loss. By such means, UFPMP-Det largely promotes both the detection accuracy and efficiency. Extensive experiments are carried out on the widely used VisDrone and UAVDT datasets, and UFPMP-Det reports new state-of-the-art scores at a much higher speed, highlighting its advantages.

MStrans: Multiscale Vision Transformer for Aerial Objects Detection

“…In recent years, the deep learning method has been widely used in aerial object detection. The proposed MStrans was compared with some detection frameworks, including the detection framework based on CNN (i.e., UFPMP-Det [8], ClusDet [4], and GLSAN [5]) and the detection framework based on transformer (i.e., TPH [27], ViT-YOLO [24], TRD [23], and LeViT [56]).…”

“…Similar to the evaluation metrics used in [4,5,8], this study chose average precision (AP) and APs at the threshold of 0.5 (𝐴𝑃 ) and 0.75 (𝐴𝑃 ) to evaluate aerial object detection performance. Meanwhile, the variants of AP were also taken as evaluation metrics, including 𝐴𝑃 , 𝐴𝑃 , and 𝐴𝑃 for the object instances in a small, medium, and large size on the DOTA dataset and DIOR dataset.…”

“…Similarly, Xu et al [7] constructed an adaptive zoom detection network (AdaZoom) based on reinforcement learning and strategy gradient, which improved the multi-scale robustness of the network. To better handle the shape similarity of dense small objects in aerial images, Huang et al [8] abandoned the coarse-to-fine detection strategy and adopted a fine-grained strategy to model the object distribution. Also, they designed a multi-agent detection network based on unified foreground packaging called UFPMP-Det.…”

NRT-YOLO: Improved YOLOv5 Based on Nested Residual Transformer for Tiny Remote Sensing Object Detection

“…Clustering guided cropping [14][15][16][17]; • Density-map based cropping [18][19][20]; • Reinforcement-learning based cropping [21,22].…”

“…One can be obviously seen that this kind of cropping approach has less efficiency nor accuracy. Thus, more and more studies began to focus on cropping methods based on clustering [14][15][16][17], density mapping [18][19][20], and reinforcement-learning [21,22]. The detectors utilizing these three manners usually contain two-stage networks: the first are cropping images with coarse-grained detectors which can describe object distribution; and then there is detecting and classifying images with a fine-grained detector.…”