Real-Time and Embedded Detection of Hand Gestures with an IMU-Based Glove

Abstract

This article focuses on the use of data gloves for human-computer interaction concepts, where external sensors cannot always fully observe the user's hand. A good concept hereby allows to intuitively switch the interaction context on demand by using different hand gestures. The recognition of various, possibly complex hand gestures, however, introduces unintentional overhead to the system. Consequently, we present a data glove prototype comprising a glove-embedded gesture classifier utilizing data from Inertial Measurement Units (IMUs) in the fingertips. In an extensive set of experiments with 57 participants, our system was tested with 22 hand gestures, all taken from the French Sign Language (LSF) alphabet. Results show that our system is capable of detecting the LSF alphabet with a mean accuracy score of 92% and an F1 score of 91%, using complementary filter with a gyroscope-to-accelerometer ratio of 93%. Our approach has also been compared to the local fusion algorithm on an IMU motion sensor, showing faster settling times and less delays after gesture changes. Real-time performance of the recognition is shown to occur within 63 milliseconds, allowing fluent use of the gestures via Bluetooth-connected systems.

Modelling a Spatial-Motion Deep Learning Framework to Classify Dynamic Patterns of Videos

“…Therefore, recently, hand gesture recognition has motivated new technologies in the area of computer vision. Previous studies have been proposed to solve hand gesture recognition tasks such as the glove-based approach [37], device-related methods [38] and the data glove-based method to address the issue of external sensors that enable us to monitor a user's hand motions more frequently [39]. Apart from such methods, nowadays, deep learning-based models are utilized to solve the hand gesture recognition and classification problems more efficiently and accurately [40,41].…”

Dynamic Hand Gesture Recognition Using 3DCNN and LSTM with FSM Context-Aware Model

“…where, i t , f t , o t , z t represent the input gate, forget gate, output gate, and cell gate respectively. c t and h t are memory and output activation at time t. The Equations (10), (11), (13) and (14) are the formulas for forget, cell, output gates and hidden state.…”

“…Even though most of such glove-based systems focusing on sensors, these external sensors enable to observe the user's hand always. To address this drawback, a glove-based concept which utilizes the data gloves for human-computer interaction has proposed [13]. Besides the study [14] evaluate the performance of a wearable gesture recognition system that captures hand, finger and arm.…”

Mobile manipulator control through gesture recognition using IMUs and Online Lazy Neighborhood Graph search

“…For IMU-based approaches, many use glove-mounted sensors. Mummadi et al [12] used an IMU-based glove for realtime sign language recognition. They used various machine learning algorithms, such as Support Vector Machines, Naive Bayes, Multi-Layer Perceptron, and Random Forest, to classify the gestures.…”

“…They employed HMMs as the underlying algorithm for gesture recognition. However, these methods either only classified static gestures with the hand and fingers [8], [12], [14] or needed a huge database (about 1000 samples for each kind) [12].…”