Dynamic gesture recognition based on feature fusion network and variant ConvLSTM

Abstract

Gesture is a natural form of human communication, and it is of great significance in human–computer interaction. In the dynamic gesture recognition method based on deep learning, the key is to obtain comprehensive gesture feature information. Aiming at the problem of inadequate extraction of spatiotemporal features or loss of feature information in current dynamic gesture recognition, a new gesture recognition architecture is proposed, which combines feature fusion network with variant convolutional long short‐term memory (ConvLSTM). The architecture extracts spatiotemporal feature information from local, global and deep aspects, and combines feature fusion to alleviate the loss of feature information. Firstly, local spatiotemporal feature information is extracted from video sequence by 3D residual network based on channel feature fusion. Then the authors use the variant ConvLSTM to learn the global spatiotemporal information of dynamic gesture, and introduce the attention mechanism to change the gate structure of ConvLSTM. Finally, a multi‐feature fusion depthwise separable network is used to learn higher‐level features including depth feature information. The proposed approach obtains very competitive performance on the Jester dataset with the classification accuracies of 95.59%, achieving state‐of‐the‐art performance with 99.65% accuracy on the SKIG (Sheffifield Kinect Gesture) dataset.

[Retracted] Dynamic Gesture Recognition Algorithm Based on 3D Convolutional Neural Network

“…Dynamic gesture recognition should not only consider hand postures and shapes but also pay attention to the spatial displacement and spatiotemporal correlation of the whole gesture [ 12 – 14 ]. Compared with static gesture recognition, dynamic gesture is closer to people's expression habits with more abundant information expression [ 15 – 17 ], which has more practical significance. Nowadays, researchers have proposed a variety of dynamic gesture recognition algorithms, including dynamic gesture feature extraction algorithm such as MEI algorithm, HOG algorithm, and HOF [ 18 ] algorithm and classification algorithm such as hidden Markov model [ 19 ].…”

Human Motion Target Posture Detection Algorithm Using Semi-Supervised Learning in Internet of Things

“…P-N Learning is now a typical semi-supervised learning algorithm. This method introduces the three restrictions of time domain, space domain and data, restricts the process of unlabeled data processing [42][43] and constructs a semi-supervised learning framework to complete the classifier training and improve the classification performance.…”

Dynamic Gesture Contour Feature Extraction Method Using Residual Network Transfer Learning

“…The dynamic gesture recognition speed of this method is fast, but the contour and texture of dynamic gesture recognition are not considered, resulting in low accuracy of dynamic gesture recognition. The literature [9] proposed a dynamic gesture recognition method based on a feature fusion network and variant [11] proposes a dynamic gesture recognition method based on short-time sampling neural network. The short-time sampling neural network is used to integrate verified modules to learn short-term and long-term features from video input.…”

“…According to Figure 4. When there are 800 dynamic gesture contour feature points, the average dynamic gesture contour feature recognition rate of Bastos et al's [8] method is 60%, and the average dynamic gesture contour feature recognition rate of Peng et al's [9] method is 57%, the average dynamic gesture contour feature recognition rate of Tang et al's [10] method is 82%, the average dynamic gesture contour feature recognition rate of Zhang et al's [11] method is 76%, and the average dynamic gesture contour feature recognition rate of Liang and Liao's [12] method is 10%. The average recognition rate of dynamic gesture contour features is 91%.…”