YOLBO: You Only Look Back Once–A Low Latency Object Tracker Based on YOLO and Optical Flow

Abstract

One common computer vision task is to track an object as it moves from frame to frame within a video sequence. There are a myriad of applications for such capability and the underlying technologies to achieve this tracking are very well understood. More recently, deep convolutional neural networks have been employed to not only track, but also to classify objects as they are tracked from frame to frame. These models can be used in a tracking paradigm known as tracking by detection and can achieve very high tracking accuracy. The major drawback to these deep neural networks is the large amount of mathematical operations that must be performed for each inference which negatively impacts the number of tracked frames per second. For edge applications residing on size, weight, and power limited platforms, such as unmanned aerial vehicles, high frame rate and low latency real time tracking can be an elusive target. To overcome the limited power and computational resources of an edge compute device, various optimizations have been performed to trade off tracking speed, accuracy, power, and latency. Previous works on motion based interpolation with neural networks either do not take into account the latency accrued from camera image capture to tracking result or they compensate for this latency but are bottlenecked by the motion interpolation operation instead. The algorithm presented in this work gains the performance speedup used in previous motion based neural network inference papers and also performs a novel look back operation that is less cumbersome than other competing motion interpolation methods.INDEX TERMS CNN, classifier, detector, neural network, low latency, tracker, UAV, YOLO, look back, drone, image processing.

Real-Time Object Detection Based on UAV Remote Sensing: A Systematic Literature Review

“…Within less extensive application scenarios, flight planning emerges, which is primarily employed to ensure UAV flight safety by surveying surrounding targets, such as drones [30,91,92]. Additionally, specific targets are addressed for UAV flight testing [93][94][95].…”

“…In [6], the authors highlighted that it has been challenging to improve the detection time of detectors, and they provided an overview of the speed-up techniques, including feature map shared computation, cascaded detection, network pruning and quantification, lightweight network design, and numerical acceleration. In our review study on UAV real-time object detection, the most commonly used methods to improve the speed of the algorithm are network pruning [54,56,65,112], quantification [67,82,92], and the use of a lightweight network design.…”

Spatiotemporal fusion target detection and location based on depth optical flow and YOLOv3

“…In 2020, action recognition using OF-DIS optical flow and YOLOv3 fusion network was realized 26 . In 2021, YOLO and an optical flow algorithm were applied to the field of drone tracking 27 . Although deep learning-based object detection is widely used, the current research on human foot object detection is still in its infancy.…”