Modeling Two-Person Segmentation and Locomotion for Stereoscopic Action Identification: A Sustainable Video Surveillance System

Abstract

Due to the constantly increasing demand for automatic tracking and recognition systems, there is a need for more proficient, intelligent and sustainable human activity tracking. The main purpose of this study is to develop an accurate and sustainable human action tracking system that is capable of error-free identification of human movements irrespective of the environment in which those actions are performed. Therefore, in this paper we propose a stereoscopic Human Action Recognition (HAR) system based on the fusion of RGB (red, green, blue) and depth sensors. These sensors give an extra depth of information which enables the three-dimensional (3D) tracking of each and every movement performed by humans. Human actions are tracked according to four features, namely, (1) geodesic distance; (2) 3D Cartesian-plane features; (3) joints Motion Capture (MOCAP) features and (4) way-points trajectory generation. In order to represent these features in an optimized form, Particle Swarm Optimization (PSO) is applied. After optimization, a neuro-fuzzy classifier is used for classification and recognition. Extensive experimentation is performed on three challenging datasets: A Nanyang Technological University (NTU) RGB+D dataset; a UoL (University of Lincoln) 3D social activity dataset and a Collective Activity Dataset (CAD). Evaluation experiments on the proposed system proved that a fusion of vision sensors along with our unique features is an efficient approach towards developing a robust HAR system, having achieved a mean accuracy of 93.5% with the NTU RGB+D dataset, 92.2% with the UoL dataset and 89.6% with the Collective Activity dataset. The developed system can play a significant role in many computer vision-based applications, such as intelligent homes, offices and hospitals, and surveillance systems.

An LSTM-Based Approach for Understanding Human Interactions Using Hybrid Feature Descriptors Over Depth Sensors

“…Moreover, suspicious activities including touching someone's pocket, pushing someone, or fighting are also of interest for researchers in this field. HIR has become a trending topic in the field of artificial intelligence because of its wide range of applications, including security [1][2][3], content-based video retrieval [4][5][6], healthcare [7][8][9][10][11], and surveillance [12][13][14][15].…”

HF-SPHR: Hybrid Features for Sustainable Physical Healthcare Pattern Recognition Using Deep Belief Networks

“…After that, we apply a sliding window algorithm to find hybrid features of different types [34]. In the perspective of multisensory systems, these hybrid features are then fused [35] through a feature-in-feature-out technique [36,37] to improve, refine, and obtain new merged features. The dimensions of these fused data features are reduced using our novel modified multi-layer sequential forward selection algorithm.…”

A Smart Surveillance System for People Counting and Tracking Using Particle Flow and Modified SOM

“…Here, the number of clusters in frame F l is denoted as c q. Our method calculates the average number of clusters [68] C in the frames used for pre-learning, as shown in Equation (14).…”