Real-Time Object Detection and Localization in Compressive Sensed Video

Abstract

Typically a 1-2MP CCTV camera generates around 7-12GB of data per day. Frame-by-frame processing of such an enormous amount of data requires hefty computational resources. In recent years, compressive sensing approaches have shown impressive compression results by reducing the sampling bandwidth. Different sampling mechanisms were developed to incorporate compressive sensing in image and video acquisition. Though all-CMOS [1, 2] sensor cameras that perform compressive sensing can help save a lot of bandwidth on sampling and minimize the memory required to store videos, the traditional signal processing, and deep learning models can realize operations only on the reconstructed data. To realize the original uncompressed domain, most reconstruction techniques are computationally expensive and time-consuming. To bridge this gap, we propose a novel task of detection and localization of objects directly on the compressed frames. Thereby mitigating the need to reconstruct the frames and reducing the search rate up to 20× (compression rate). We achieved an accuracy of 46.27% mAP with the proposed model on a GeForce GTX 1080 Ti. We were also able to show real-time inference on an NVIDIA TX2 embedded board with 45.11% mAP, thereby achieving the best balance between the accuracy, inference time, and memory constraints.

From compressive sampling to compressive tasking: retrieving semantics in compressed domain with low bandwidth

“…As two sequential modules in the visual information acquisition and processing pipeline, hardware-oriented CI and task-driven CV are naturally complementary. Thus, to improve the overall performance has become a new trend to combine SCI and SCV [15,16,[62][63][64].…”

“…For large-scale SCI imaging, the reconstruction process is time-consuming, and the reconstructed video will occupy a large amount of memory, making the subsequent semantic information retrieval steps challenging. Considering that the coded measurements from SCI contain nearly all the useful information of the original scene, performing semantic information retrieval directly on coded measurements is theoretically feasible and has become a new trend [15,16,[62][63][64].…”