IntPred

Abstract

Deep Neural-Network (DNN) based Object Detection is one of the most important and time-consuming stages of Autonomous Driving software in cars. In non-critical domains, the performance and energy requirements of object detection can be reduced at the cost of accuracy in the detected objects. This is not the case in a critical domain like automotive, for which a delicate balance between performance/energy overheads and accuracy of object detection must be found. We propose IntPred to achieve such a balance by leveraging on the fact that, with high frame rates, objects do not move significantly across frames. IntPred tailors object interpolation for the case of object detection in autonomous driving frameworks, in line with approaches devised for other domains, thus heavily reducing the performance requirements of full-fledged DNN-based object prediction. IntPred results in comparable accuracy to the original object detection, while saving more than 70% of the computations. The latter allows using lower-performance and cheaper platforms resulting in saving energy and reducing heat dissipation: for instance, in an NVIDIA Jetson TX2 platform, specific for autonomous driving systems, our technique increases the frame processing rate by 4.6x. IntPred also allows consolidating additional applications onto the same platform.

Motion Vector Extrapolation for Video Object Detection

“…Several pieces of object tracking research utilize motion vectors to accelerate their tracking performance [20] [21] [22]. The common approach among these works is interactions between an off-the-shelf object detection approach, such as Faster R-CNN [8] or YOLOv4 [4] explained above, and the motion vectors or motion predictions of each frame.…”

Motion Vector Extrapolation for Video Object Detection

“…Several pieces of object tracking research utilize motion vectors to accelerate their tracking performance [23][24][25]. The common approach among these works is interactions between an off-the-shelf object detection approach, such as Faster R-CNN [3] or YOLOv4 [1] explained above, and the motion vectors or motion predictions of each frame.…”

“…The common approach among these works is interactions between an off-the-shelf object detection approach, such as Faster R-CNN [3] or YOLOv4 [1] explained above, and the motion vectors or motion predictions of each frame. Each approach computes a CNN inference as part of the object detection step at a sparse key frame and perturbs the detections by using the motion vectors through linear interpolation [23][24][25]. This approach yields increased the frame processing rates in all three of these approaches, ranging from 4.6 times for Tabani et al [25] to 6 times for Liu et al [24] in off-the-shelf hardware and 12 times in specialized hardware, such as an FPGA, for Ujiie et al [23].…”

“…Additionally, since the AP decreased by 4.4 and 4.1 in both the MOT20 and MOT16 datasets, respectively, it is demonstrated that this global statistic is an important factor in the performance of the MOVEX technique. 4 denotes the results of inferencing every k th frame, such as in [12,[23][24][25], versus the proposed multi-processing-based approach. The values of 5 and 10 in evaluation for k were compared.…”

“…Multi-processing was removed from the optimistic SDPF and was replaced with a blocking inference every k th frame, which mimics the approach of Zhu et al to detection propagation and other state-of-the-art approaches [12,[23][24][25].…”