The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking

Abstract

With the advantage of high mobility, Unmanned Aerial Vehicles (UAVs) are used to fuel numerous important applications in computer vision, delivering more efficiency and convenience than surveillance cameras with fixed camera angle, scale and view. However, very limited UAV datasets are proposed, and they focus only on a specific task such as visual tracking or object detection in relatively constrained scenarios. Consequently, it is of great importance to develop an unconstrained UAV benchmark to boost related researches. In this paper, we construct a new UAV benchmark focusing on complex scenarios with new level challenges. Selected from 10 hours raw videos, about 80, 000 representative frames are fully annotated with bounding boxes as well as up to 14 kinds of attributes (e.g., weather condition, flying altitude, camera view, vehicle category, and occlusion) for three fundamental computer vision tasks: object detection, single object tracking, and multiple object tracking. Then, a detailed quantitative study is performed using most recent state-of-the-art algorithms for each task. Experimental results show that the current state-of-the-art methods perform relative worse on our dataset, due to the new challenges appeared in UAV based real scenes, e.g., high density, small object, and camera motion. To our knowledge, our work is the first time to explore such issues in unconstrained scenes comprehensively.

Delving Into Robust Object Detection From Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach

“…We end up with an improved AP of 45.64 (using the same IoU threshold = 0.7 as the authors) as our baseline performance. We also communicated with the authors of [12] in person and they acknowledged this improved baseline. We then implement NDFT-Faster-RCNN using the architecture depicted in Figure 2, also with a ResNet-101 backbone.…”

“…Since public UAV-based object detection datasets (in particular those with nuisance annotations) are currently of very limited availability, we design three sets of experiments to validate the effectiveness, robustness, and generality of NDFT. First, we perform the main body of experiments on the UAVDT benchmark [12], which provides all three UAV-specific nuisance annotations (altitude, weather, and view angle). We demonstrate the clear observation that the more variations are disentangled via NDFT, the larger AP improvement we will gain on UAVDT; and eventually we achieve the state-of-the-art performance on UAVDT.…”

“…It has three categories: car, truck and bus, but the class distribution is highly imbalanced (the latter two occupy less than 5% of bounding boxes). Hence following the convention by the authors in [12], we combine the three into one vehicle class and report AP based on that. All frames are also annotated with three categories of UAV-specific nuisances: flying altitude (low, medium and high), camera views (front-view, side-view and bird-view), and weather condition 3 (daylight, night).…”

“…We follow the tracking-by-detection framework adopted in [12], and compare the tracking results based on the detection inputs from vanilla Faster-RCNN and NDFT-Faster-RCNN (A + V + W ), respectively. All evaluation protocols are inherited from [12]. As in Table 5, NDFT-FRCNN largely outperforms the vanilla baseline in 10 out of the 11 metrics, showing its promise even beyond detection.…”

“…Public benchmarks were unavailable for specifically UAV-based object detection until recently. Two datasets, UAVDT [12] and VisDrone2018 [57], were released to address this gap. UAVDT consists of 100 video sequences (about 80k frames) captured from UAVs under complex scenarios.…”