Data glove-based gesture recognition using CNN-BiLSTM model with attention mechanism

Abstract

As a novel form of human machine interaction (HMI), hand gesture recognition (HGR) has garnered extensive attention and research. The majority of HGR studies are based on visual systems, inevitably encountering challenges such as depth and occlusion. On the contrary, data gloves can facilitate data collection with minimal interference in complex environments, thus becoming a research focus in fields such as medical simulation and virtual reality. To explore the application of data gloves in dynamic gesture recognition, this paper proposes a data glove-based dynamic gesture recognition model called the Attention-based CNN-BiLSTM Network (A-CBLN). In A-CBLN, the convolutional neural network (CNN) is employed to capture local features, while the bidirectional long short-term memory (BiLSTM) is used to extract contextual temporal features of gesture data. By utilizing attention mechanisms to allocate weights to gesture features, the model enhances its understanding of different gesture meanings, thereby improving recognition accuracy. We selected seven dynamic gestures as research targets and recruited 32 subjects for participation. Experimental results demonstrate that A-CBLN effectively addresses the challenge of dynamic gesture recognition, outperforming existing models and achieving optimal gesture recognition performance, with the accuracy of 95.05% and precision of 95.43% on the test dataset.

A Systematic Review on Semantic Role Labeling for Information Extraction in Low-Resource Data

“…BiLSTM was created to overcome the limitations of LSTM, which can only spread information in one direction, so it can only understand forward features and not capture backward features. Combining two sets of LSTMs, working on and backward, can make BiLSTM obtain contextual information in data sequences effectively because it allows for capturing patterns and temporal relationships from beginning to end [93]. Dewandaru et al [44] used a combination of BiLSTM and CRF because it outperforms the performance of other method combinations for SRL tasks.…”