Synthesis of spatio-temporal descriptors for dynamic hand gesture recognition using genetic programming

Abstract

Abstract-Automatic gesture recognition has received much attention due to its potential in various applications. In this paper, we successfully apply an evolutionary method-genetic programming (GP) to synthesize machine learned spatio-temporal descriptors for automatic gesture recognition instead of using hand-crafted descriptors. In our architecture, a set of primitive low-level 3D operators are first randomly assembled as treebased combinations, which are further evolved generation-bygeneration through the GP system, and finally a well performed combination will be selected as the best descriptor for high-level gesture recognition. To the best of our knowledge, this is the first report of using GP to evolve spatio-temporal descriptors for gesture recognition.We address this as a domain-independent optimization issue and evaluate our proposed method, respectively, on two public dynamic gesture datasets: Cambridge hand gesture dataset and Northwestern University hand gesture dataset to demonstrate its generalizability. The experimental results manifest that our GP-evolved descriptors can achieve better recognition accuracies than state-of-the-art hand-crafted techniques.

Using Motion Blur to Recognize Hand Gestures in Low-light Scenes

“…In this experiment, we compared the proposed method with the following competitive methods: Liu's method (Liu and Shao, 2013) and 3D SIFT descriptor (Scocanner et al, 2007).…”

“…In recent years, the development of motion analysis techniques using camera images has highly contributed to the advances in the field of hand gesture recognition. In fact, many methods have been proposed to improve the performance of hand gesture recognition (Pfister et al, 2014;Yamato et al, 1992;Stamer and Pentland, 1995;Freeman and Roth, 1995;Shen et al, 2012;Ikizler-Cinbis and Sclaroff, 2010;Liu and Shao, 2013;Tang et al, 2014;Davis, 2001;Scocanner et al, 2007;Bregonzio et al, 2009;Niebles et al, 2008). Early studies investigated approaches to recognize spatio-temporal hand gesture motion modeling by using a hidden Markov model (HMM) (Yamato et al, 1992;Stamer and Pentland, 1995).…”

“…Since these methods have scale-and position-invariant characteristics, they have enabled to achieve high recognition performance. Liu et al pointed out that the use of local image descriptor representations cannot be used for easily modeling precise hand gesture motions because of the loss of the structural relationships between the extracted local features (Liu and Shao, 2013). To address this, they proposed a hand gesture recognition system by automatically synthesizing spatio-temporal descriptors using genetic programming.…”

Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion

“…Methods Accuracy Liu and Shao [59] Genetic Programming 96.1% Shen et al [53] Motion Divergence fields 95.8% Our method Key Frames + Feature Fusion 96.89% ± 1.08% [59] 6.45s 13.32s 15.32s 6.43s Zhao and Elgammal [29] 5.34s 11.78s 14.98s 5.21s tion3D datasets. We re-implement both methods with the same running settings for fair comparison, including hardware platform and programming language.…”

“…Methods Accuracy Wong and Cipolla [56] Sparse Bayesian Classifier 44% Niebles et al [57] Spatial-Temporal Words 67% Kim et al [52] Tensor Canonical Correlation Analysis 82% Kim and Cipolla [58] Canonical Correlation Analysis 82% Liu and Shao [59] Genetic Programming 85% Lui et al [60] High Order Singular Value Decomposition 88% Lui and Beveridge [61] Tangent Bundle 91% Wong et al [62] Probabilistic Latent Semantic Analysis 91.47% Sanin et al [63] Spatio-Temporal Covariance Descriptors 93% Baraldi et al [64] Dense Trajectories + Hand Segmentation 94% Zhao and Elgammal [29] Information Theoretic 96.22% Ours Key Frames + Feature Fusion 98.23% ± 0.84% Table 7: Comparison between the state-of-the-art methods and our method on the Northwestern University dataset.…”