Detecting human activities in retail surveillance using hierarchical finite state machine

Abstract

Cashiers in retail stores usually exhibit certain repetitive and periodic activities when processing items. Detecting such activities plays a key role in most retail fraud detection systems. In this paper, we propose a highly efficient, effective and robust vision technique to detect checkout-related primitive activities, based on a hierarchical finite state machine (FSM). Our deterministic approach uses visual features and prior spatial constraints on the hand motion to capture particular motion patterns performed in primitive activities. We also apply our approach to the problem of retail fraud detection. Experimental results on a large set of video data captured from retail stores show that our approach, while much simpler and faster, achieves significantly better results than state-of-the-art machine learning-based techniques both in detecting checkout-related activities and in detecting checkoutrelated fraudulent incidents.

Multimodal ranking for non-compliance detection in retail surveillance

“…Performance of the multimodal ranking approach in terms of recall and alarms rate. For the same recall (detection rate), our approach achieves better alarms rate compared to the heuristics-based approach, as well as compared to the best performance of [15] and [5]. system described in [15].…”

“…For the same recall (detection rate), our approach achieves better alarms rate compared to the heuristics-based approach, as well as compared to the best performance of [15] and [5]. system described in [15]. We rank the list of nonscans detected by [15] based on their confidence scores, and returns nonscans with scores above a certain threshold.…”

“…There has been many works on retail video surveillance for sweethearting detection [14,4,1,5,6,15,7,11,8]. However, none of the approaches above employed a featurebased representation of nonscans.…”

“…Since we implement our approach as a post-processing stage of an existing video surveillance system, all these four primitive events are intermediate results of the video analytics being employed. Both [5] and in particular [15] can detect such activities with high accuracy and robustness. …”

“…Many video analytics systems such as [14,4,1,5,15] have been introduced to detect nonscans, showing more advantages than human surveillance in effectiveness, efficiency and scalability. The ultimate goal of these systems is to detect nonscans, create a real-time alert for each of them for human verification.…”