An Approach to Gesture Recognition with Skeletal Data Using Dynamic Time Warping and Nearest Neighbour Classifier

Abstract

Gestures are natural means of co mmun ication between humans, and therefore their application would benefit to many fields where usage of typical input devices, such as keyboards or joysticks is cu mbersome or unpractical (e.g., in noisy environ ment). Recently, together with emergence of new cameras that allow obtaining not only colour images of observed scene, but also offer the software developer rich informat ion on the number of seen hu mans and, what is most interesting, 3D positions of their body parts, practical applications using body gestures have become more popular. Such informat ion is presented in a form o f skeletal data. In this paper, an approach to gesture recognition bas ed on skeletal data using nearest neighbour classifier with dynamic time warping is presented. Since similar approaches are widely used in the literature, a few practical improvements that led to better recognition results are proposed. The approach is extensively evaluated on three publicly availab le gesture datasets and compared with state-of-the-art classifiers. For some gesture datasets, the proposed approach outperformed its competitors in terms of recognition rate and time of recognition.

Advanced Machine Learning for Gesture Learning and Recognition Based on Intelligent Big Data of Heterogeneous Sensors

“…To enhance the gesture recognition accuracy, the appropriate learning algorithms can be selected through the learning algorithm selection module. In a previous study, it was shown that the recognition accuracies of HMMs or RNNs are higher than that of DTW [34][35][36]. Moreover, in our experiment, the highest accuracy was obtained when using HMMs.…”

Real-Time Arm Gesture Recognition Using 3D Skeleton Joint Data

“…Reyes et al [8] proposed a feature weighted variation of DTW on 3D joints, using the neck as a reference point. Ribó et al [9] presented a similar approach which relies on DTW, KNN classifiers and some heuristics to improve performance. Ibañez et al [10] used 3D joints and compared DTW with an HMM, acquiring the same precision.…”

Hand Posture Recognition Using Modified Ensemble of Shape Functions and Global Radius-Based Surface Descriptor

“…Li [12] utilized information about the angles between extended fingers and a two-stage classification that eliminated gestures with different numbers of detected fingers in the first stage. Plouffe et al [23] proposed a method of comparing data sequences (Dynamic Time Warping -DTW) to static hand posture recognition, although this technique is usually used for the classification of dynamic gestures (e.g., [28]). In [25], the authors developed a system of gesture recognition based on the convex shape decomposition method that uses Morse functions.…”