Filtering Empty Video Frames for Efficient Real-Time Object Detection

Abstract

Deep learning models have significantly improved object detection, which is essential for visual sensing. However, their increasing complexity results in higher latency and resource consumption, making real-time object detection challenging. In order to address the challenge, we propose a new lightweight filtering method called L-filter to predict empty video frames that include no object of interest (e.g., vehicles) with high accuracy via hybrid time series analysis. L-filter drops those frames deemed empty and conducts object detection for nonempty frames only, significantly enhancing the frame processing rate and scalability of real-time object detection. Our evaluation demonstrates that L-filter improves the frame processing rate by 31–47% for a single traffic video stream compared to three standalone state-of-the-art object detection models without L-filter. Additionally, L-filter significantly enhances scalability; it can process up to six concurrent video streams in one commodity GPU, supporting over 57 fps per stream, by working alongside the fastest object detection model among the three models.

Proposal-Free Fully Convolutional Network: Object Detection Based on a Box Map

“…Furthermore, the region proposal-free approach introduces an important innovation by treating object detection as a direct-regression problem or keypoint prediction [ 35 , 36 , 37 , 38 ]. As real-world applications increasingly require real-time processing [ 39 , 40 , 41 , 42 , 43 ], the importance of region proposal-free detectors such as the proposed PF-FCN becomes more remarkable. In summary, from region proposal-based detectors to region proposal-free detectors, the research emphasis is always on achieving a delicate balance between accuracy and efficiency.…”