Hand Gesture Recognition Using Temporal Convolutions and Attention Mechanism

Rahimian, Elahe; Zabihi, Soheil; Asif, Amir; Farina, Dario; Atashzar, S. Farokh; Mohammadi, Arash

doi:10.1109/icassp43922.2022.9746174

Cited by 14 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By recording the physiological signals associated to movement it may be possible to solve this paradox, and the literature is filled with examples [6]- [37] with some being more successful then others.…”

Section: Discussionmentioning

confidence: 99%

“…The first and arguably most accurate (in terms of small errors and high prediction accuracy) group interprets the problem at hand as a classification task and wants to reliably distinguish between different hand postures. This approach is called gesture recognition [6]- [10], [17], [27], [32], [33], [35]- [37], [50]. Although achieving impressive results such as ≥ 90% accuracy on the Italian sign language [8] or 93.84% accuracy on 49 distinct poses [36], we believe that the method itself is lacking due to the users inability to chose the speed of executing the desired pose.…”

Section: Discussionmentioning

confidence: 99%

“…The neural interface has been hampered by a number of factors of varying severity and prevalence, theoretical shortcomings being the most important. Much attention is paid to discrete methods such as gesture recognition [15], [16] and discrete joint angle classification [17], [18], both of which are by definition incapable of providing continuous output, leading to limitations in motion output. The solution to this problem is to continuously predict either the joints as 3D points in Euclidean space [19] or the angles between the joints [20]- [28].…”

Section: Introductionmentioning

confidence: 99%

“…If we want on-demand kinematics and kinetics recognition for humans, it becomes clear that we need to find something other than cameras for use at home or on the street, where we cannot set up a recording volume. There have been numerous relevant attempts trying to fill this specific need [6]- [37] by exploiting the EMG signals to reconstruct the movement intent, yet all have some limitations that hinder the access to the large number of degrees of freedom from the human hand. We will delve into the shortcomings of these works later in the discussion section, but for now, here is a brief overview of the limitations of current sEMG decoding systems: 1) errors that are too big for some or all of the fingers [13]- [16], [18], [20]- [23], [25], [26], [38], 2) achieving less than 14 degrees of freedom (DoF) that the hand needs for executing flexion and extension movements [14], [15], [19], [21]- [26], [28]- [30], [38], and 3) or not attempting unseen combinations of different movements [13]- [16], [18]- [26], [28], [29], [31], [38].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Sensing the Full Dynamics of the Human Hand with a Neural Interface and Deep Learning

Sîmpetru

Arkudas

Braun

et al. 2022

Preprint

View full text Add to dashboard Cite

Theories on the neural control of movement are largely based on movement-sensing devices that capture the dynamics of predefined anatomical landmarks. Neuromuscular interfaces, such as surface electromyography (sEMG), can in theory surpass the limitations imposed by motion-based technologies by sensing the motor commands transmitted by the final pathway of movement, the motor units. The recording of motor unit activity may allow the prediction of the kinetics and kinematics continuously in time and space, without being limited to several biological and physical boundaries that digital cameras or inertial sensors suffer. However, current sEMG decoding algorithms can only predict few degrees of freedom (<3). By combining markerless machine vision and high-density sEMG electrodes, we aimed to test the hypothesis that a physiologically inspired deep neural network can reconstruct the movement of the human hand as precise as digital cameras and with the additional benefit of predicting the underlying forces (i.e., grasping a cup of coffee). We demonstrate that our deep learning model can continuously predict all degrees of freedom of the hand with negligible errors during natural motion tasks from 320 sEMG sensors placed only on the extrinsic hand muscles. Our deep learning model was able to display the 3D hand kinematics and the full force range of the hand digits during isometric contractions. The current results demonstrate that deep learning applied to EMG signals gives access to an unprecedented representation of the final neural code of movement.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sensing the Full Dynamics of the Human Hand with a Neural Interface and Deep Learning

Sîmpetru

Arkudas

Braun

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Fan et al [26] proposed the CSAC-Net network model, leveraging attention mechanisms to focus on crucial information in the channel space, achieving a gesture recognition accuracy of 82.50%. Rahimian et al [27] employed the attention mechanism and temporal convolution in the TC-HGR architecture, achieving a gesture recognition accuracy of 81.65%. Hu et al [28] proposed a hybrid CNN-RNN network structure based on the attention mechanism, achieving an average gesture recognition accuracy of 84.80% based on the NinaPro DB1 dataset.…”

Section: Introductionmentioning

confidence: 99%

InRes-ACNet: Gesture Recognition Model of Multi-Scale Attention Mechanisms Based on Surface Electromyography Signals

Luo,

Huang,

Wang

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Surface electromyography (sEMG) signals are the sum of action potentials emitted by many motor units; they contain the information of muscle contraction patterns and intensity, so they can be used as a simple and reliable source for grasping mode recognition. This paper introduces the InRes-ACNet (inception–attention–ACmix-ResNet50) model, a novel deep-learning approach based on ResNet50, incorporating multi-scale modules and self-attention mechanisms. The proposed model aims to improve gesture recognition performance by enhancing its ability to extract channel feature information within sparse sEMG signals. The InRes-ACNet model is evaluated on the NinaPro DB1 and NinaPro DB5 datasets; the recognition accuracy for these datasets can reach 87.94% and 87.04%, respectively, and recognition accuracy can reach 88.37% in the grasping mode prediction of an electromyography manipulator. The results show that the fusion of multi-scale modules and self-attention mechanisms endows a strong ability for the task of gesture recognition based on sparse sEMG signals.

show abstract

Light-Weight CNN-Attention Based Architecture Trained with a Hybrid Objective Function for EMG-Based Human Machine Interfaces

Zabihi,

Rahimian,

Asif

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Hand Gesture Recognition Using Temporal Convolutions and Attention Mechanism

Cited by 14 publications

References 22 publications

Sensing the Full Dynamics of the Human Hand with a Neural Interface and Deep Learning

Sensing the Full Dynamics of the Human Hand with a Neural Interface and Deep Learning

InRes-ACNet: Gesture Recognition Model of Multi-Scale Attention Mechanisms Based on Surface Electromyography Signals

Light-Weight CNN-Attention Based Architecture Trained with a Hybrid Objective Function for EMG-Based Human Machine Interfaces

Contact Info

Product

Resources

About