Recognizing Gestures by Learning Local Motion Signatures of HOG Descriptors

Abstract

We introduce a new gesture recognition framework based on learning local motion signatures (LMSs) of HOG descriptors introduced by [1]. Our main contribution is to propose a new probabilistic learning-classification scheme based on a reliable tracking of local features. After the generation of these LMSs computed on one individual by tracking Histograms of Oriented Gradient (HOG) [2] descriptor, we learn a code-book of video-words (i.e. clusters of LMSs) using kmeans algorithm on a learning gesture video database. Then the video-words are compacted to a code-book of code-words by the Maximization of Mutual Information (MMI) algorithm. At the final step, we compare the LMSs generated for a new gesture w.r.t. the learned code-book via the k-nearest neighbors (k-NN) algorithm and a novel voting strategy. Our main contribution is the handling of the N to N mapping between code-words and gesture labels within the proposed voting strategy. Experiments have been carried out on two public gesture databases: KTH [3] and IXMAS [4]. Results show that the proposed method outperforms recent state-of-the-art methods.

Real-Time Action Detection in Video Surveillance using a Sub-Action Descriptor with Multi-Convolutional Neural Networks

“…Figure 16 shows the confusion matrix of the proposed method on the KTH dataset. [57] 94.5 -Wang et al [9] 94.2 -Gilbert et al [58] 94.5 -Baccouche et al [59] 94.4 -Zhang et al [60] 94.1 -Kaâniche and Brémond [61] 94.7 -Ji et al [44] 90.2 -Bilinski et al [62] 94.9 -Chen et al [63] 94.4 -Selmi et al [64] 95.8 -Liu et al [65] 95.0 -Multi-CNN classifier with a sub-action descriptor 96.3 86 Figure 16: Confusion matrix of the multi-CNN classifier with a sub-action descriptor on the KTH dataset. The horizontal rows are the ground truth, and the vertical columns are the predictions.…”

Low-Complexity Hand Gesture Recognition System for Continuous Streams of Digits and Letters

“…Wu et al [39] performed one-shot learning based on the extended motion-historyimage. Kaâniche and Brémond [40] learned local motion signatures of histograms of oriented gradients [41], while Yang et al [42] discovered high-level motion primitives by hierarchical clustering of optical flow. Both works achieved high-recognition results on several activity and gesture datasets.…”

Continuous rotation invariant features for gradient-based texture classification

“…The block size is not restricted to construct an extensive set of texture features, which allow extracting high-discriminated features in order to improve classification accuracy and reduce computational time of classification algorithms [59,60]. HOG is a window based algorithm computed local to a detected interest point.…”