One-Shot-Learning Gesture Recognition Using Motion History Based Gesture Silhouettes

Abstract

A novel approach for gesture recognition based on motion history images is proposed in this paper for one-shot learning gesture recognition task. The challenge here is to perform satisfactory recognition operations with only one training example of each action, while no prior knowledge about actions, foreground/background segmentation, or any motion estimation and tracking are available. In the proposed scheme motion history imaging technique is applied to track the motion flow in consecutive frames. The information of motion flow is later utilized to calculate the percent change of motion flow for an action in different spatial regions of the frame. The space-time descriptor computed this way from the query video is a measure of the likeness of a gesture in a lexicon. Finally, gesture classification is performed based on correlation based and Euclidean distance based classifiers and the results are compared. Through extensive experimentations on a much diversified dataset the effectiveness of employing the proposed scheme is established.

Discriminating features learning in hand gesture classification

“…Both extended‐MHI [60] and manifold LSR [61] adopted more complex classifiers. In the case of motion history feature, if Euclidean distance‐based classifiers is adopted [49], our prosed EK method can also get better performance. In extended‐MHI, the similar motion history feature is used, whereas a correlation coefficient method with higher computation complexity is adopted, leading to a higher accuracy than the proposed EK.…”

“…On the basis of 3D histogram of flow (3DHOF) and global HOG (GHOG), Fanello et al [43] apply adaptive sparse coding to capture high‐level feature patterns. Mahbub et al [31] propose a space–time descriptor and apply motion history imaging (MHI) techniques to track the motion flow in consecutive frames.…”

“…Subsequently, it was Hotelling [48] who adopted PCA for the analysis of the correlation structure between many random variables. Some interesting applications may be found in some recent books, for example [49, 50]. Assuming Gaussian distributed data, PCA is well known to be optimal under both mean‐square‐error and maximum‐entropy criteria.…”

BoMW: Bag of Manifold Words for One-Shot Learning Gesture Recognition From Kinect

“…The widely used features for gesture recognition are color [27], [28], shapes [29], [30] and motion [31], [32]. Compared to color and shape features, motion features extracted from two consecutive frames are more discriminative for gesture recognition because most of gestures can be distinguished by different motion patterns.…”

“…A geometric framework for least square regression is further presented and applied to gesture recognition. Mahbub et al [32] propose a space-time descriptor and apply Motion History Imaging (MHI) techniques to track the motion flow in consecutive frames. Seo and Milanfar [58] present a novel action recognition method based on space-time locally adaptive regression kernels.…”