RGBD Human Action Recognition using Multi-Features Combination and K-Nearest Neighbors Classification

Abstract

Abstract-In this paper, we present a novel system to analyze human body motions for action recognition task from two sets of features using RGBD videos. The Bag-of-Features approach is used for recognizing human action by extracting local spatialtemporal features and shape invariant features from all video frames. These feature vectors are computed in four steps: Firstly, detecting all interest keypoints from RGB video frames using Speed-Up Robust Features and filters motion points using Motion History Image and Optical Flow, then aligned these motion points to the depth frame sequences. Secondly, using a Histogram of orientation gradient descriptor for computing the features vector around these points from both RGB and depth channels, then combined these feature values in one RGBD feature vector. Thirdly, computing Hu-Moment shape features from RGBD frames; fourthly, combining the HOG features with Hu-moments features in one feature vector for each video action. Finally, the k-means clustering and the multi-class K-Nearest Neighbor is used for the classification task. This system is invariant to scale, rotation, translation, and illumination. All tested, are utilized on a dataset that is available to the public and used often in the community. By using this new feature combination method improves performance on actions with low movement and reach recognition rates superior to other publications of the dataset.

Local Feature Extraction from RGB and Depth Videos for Human Action Recognition

“…In order to generate the Bag-of-Words (BoWs), we followed the method in [24]. For each feature vector of the video frames is compared to each centroid of the cluster in the dictionary using Euclidean distance measure e as formulated in equation 3:…”

“…This dataset contains seven actions categories which are recorded in the living room such as: drinking, eating, using a laptop, picking up a phone, reading phone (sending SMS), reading a book, and using a remote, as shown in Fig. 5, [24]. In this work, the comparison results is done with the state-of-the-art methods on the same environment test setting, where half of the subjects are used as training data and the rest of the subjects are used as test data.…”