Incident Retrieval and Recognition in Video Stream Using Wi-Fi Signal

Abstract

Retrieving incidents from video stream plays an important role in many computer vision applications. However, most video surveillance system can neither recognize incidents nor support contentbased retrieval before the video stream is saved into files. As an emerging type of sensing modality, Wi-Fi signal have the potential to become a signal synchronized with the video stream to perform the incidents detection and recognition. In this work, we simultaneously collect the video stream and the Wi-Fi signal in two surveillance scenarios, and develop a LSTM-based classification model that is able to recognize the incidents in surveillance scenarios. Specifically, we first deploy a video surveillance system in two scenarios to capture the video stream and the synchronized Wi-Fi signal that is very sensitive to environmental changes. Second, an incident detection method based on the entropy change of Wi-Fi signal is proposed to find out the start and end time of the incident in the CSI sequence, thus greatly reducing the computational complexity compared with shot detection in the video stream. Third, the deep network LSTM is adopted to develop an incident recognition model that would be used to classify each size-variable CSI segments into known categories corresponding to the types of incidents. Fourth, using Wi-Fi signal to locate and recognize incidents in the video stream, we build a quick content-based video retrieval system. Last, the experimental evaluation was performed on a group of real Wi-Fi signal samples. The statistical results shows that the proposed incident detection method is feasible and effective to find out the incidents in video files with an average error of 1.5 s. And the evaluation experiment results demonstrates that the proposed multi-classification model acquires an average value of 0.972, 0.973, 0.985, 0.972 and 0.962 for recall, precision, accuracy, F-1 score and Kappa coefficient, respectively.

DLLBVS: Design of a High-Efficiency Deep Learning based Low-BER Video Streaming Model for High-Noise Wireless Networks

“…Models that use Deep Reinforcement Learning (DRL) [14], Linear Optimizations [15], Online Bitrate Selection [16], Deep Learning Optimizations [17], cycle vector-quantized variational autoencoder (cycle-VQ-VAE) [18], Flexible Latency Aware Streaming (FLAS) [19], and Reinforcement Learning-Based Rate Adaptation (RLRA) [20], for dynamic control over streaming operations are discussed & evaluated under different scenarios. These models are further extended via the work in [21,22,23,24,25], which propose use of Shift-Tile-Tracking (STC), LSTM based streaming, scalable-high-efficiency-video-coding (SHVC) with device-to-device communications, Sliding-Window Forward Error Correction (SW FEC), and context-aware streaming, which enables real-time processing for different video types. Extended models that use Proactive Caching [26], and Video streaming based on super resolution [27] are also discussed, and are highly useful for a wide variety of real-time use cases.…”