MS-Faster R-CNN: Multi-Stream Backbone for Improved Faster R-CNN Object Detection and Aerial Tracking from UAV Images

Abstract

Tracking objects across multiple video frames is a challenging task due to several difficult issues such as occlusions, background clutter, lighting as well as object and camera view-point variations, which directly affect the object detection. These aspects are even more emphasized when analyzing unmanned aerial vehicles (UAV) based images, where the vehicle movement can also impact the image quality. A common strategy employed to address these issues is to analyze the input images at different scales to obtain as much information as possible to correctly detect and track the objects across video sequences. Following this rationale, in this paper, we introduce a simple yet effective novel multi-stream (MS) architecture, where different kernel sizes are applied to each stream to simulate a multi-scale image analysis. The proposed architecture is then used as backbone for the well-known Faster-R-CNN pipeline, defining a MS-Faster R-CNN object detector that consistently detects objects in video sequences. Subsequently, this detector is jointly used with the Simple Online and Real-time Tracking with a Deep Association Metric (Deep SORT) algorithm to achieve real-time tracking capabilities on UAV images. To assess the presented architecture, extensive experiments were performed on the UMCD, UAVDT, UAV20L, and UAV123 datasets. The presented pipeline achieved state-of-the-art performance, confirming that the proposed multi-stream method can correctly emulate the robust multi-scale image analysis paradigm.

MultiRPN-DIDNet: Multiple RPNs and Distance-IoU Discriminative Network for Real-Time UAV Target Tracking

“…The basic idea of this type of trackers [21][22][23][24] is to divide the video frame into the background and the target area, so the target tracking problem is transformed into a classification problem. Multi-domain Network (MDNet) tracker [21] adopts a shallow CNN consisting of three convolutional layers and three fully connected (FC) layers, by using a multi-domain learning strategy to improve the tracking accuracy.…”

“…MS-Faster R-CNN tracker [24] integrates multi-stream (MS) into Faster-R-CNN and combines it with the Simple Online and Real-time Tracking with a Deep Association Metric (Deep SORT) algorithm to achieve real-time tracking capabilities on UAV images.…”

“…We compared the proposed tracking method with 13 state-of-the-art trackers on the UAV20L dataset. The 13 trackers are MS-Faster R-CNN [24], KCC [38], SRDCF [4], Auto-Track [26], CSRDCF [39], ECO_HC [40], ARCF [25], ARCF_H [25], STRCF [41], BACF [42], MDNet [21], SiamFCpp_googlenet [43] and SiamRPN++ [12]. The comparison results by using different trackers are shown in Table 4.…”

Low-Altitude Aerial Video Surveillance via One-Class SVM Anomaly Detection from Textural Features in UAV Images

“…The authors of [64], instead, propose a visual cryptography approach to detect hidden targets, thus enabling the design of a new paradigm for the localization and communication of sensitive military objectives. In [65], the authors propose a faster Region-based CNN (R-CNN) for object detection and tracking. Finally, in [66], the authors present a feature-based Simultaneous Localization And Mapping (SLAM) algorithm for small-scale UAVs with nadir view.…”

Multi-Dimensional Automatic Detection of Scanning Radar Images of Marine Targets Based on Radar PPInet

“…The methods mainly include a two-stage detection algorithm and single-stage detection algorithm. The former is R-CNN, spatial pyramid pooling (SPP)-Net, Fast R-CNN, Faster R-CNN [27], FPN, Mask R-CNN [28], etc. A region suggestion generator is formed, where features are extracted, and a classifier is used to predict the category of the proposed area.…”