Signgraph: An Efficient and Accurate Pose-Based Graph Convolution Approach Toward Sign Language Recognition

Naz, Neelma; Sajid, Hasan; Ali, Sara; Hasan, Osman; Ehsan, Muhammad Khurram

doi:10.1109/access.2023.3247761

Cited by 21 publications

(5 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because sign language is performed by multiple parts of the body, the nodes of the graph structure used in sign language recognition should respond to information from those parts. However, too many nodes do not provide additional useful information to the model, but instead introduce noise into the model, which affects the accuracy of the model [118,126]. Therefore, for skeleton-based SLR, it is important to choose the right nodes for model learning.…”

Section: C: Other Methodsmentioning

confidence: 99%

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

Tao,

Zhao,

Liu

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The Deaf are a large social group in society. Their unique way of communicating through sign language is often confined within their community due to limited understanding by individuals outside of this demographic. This is where sign language recognition (SLR) comes in to help normal people understand the meaning of sign language. In recent years, new methods of sign language recognition have been developed and achieved good results, so it is necessary to make a summary. This review mainly focuses on the introduction of sign language recognition techniques based on algorithms especially in recent years, including the recognition models based on traditional methods and deep learning approaches, sign language datasets, challenges and future directions in SLR. To make the method structure clearer, this article explains and compares the basic principles of different methods from the perspectives of feature extraction and temporal modelling. We hope that this review will provide some reference and help for future research in sign language recognition.

show abstract

Section: C: Other Methodsmentioning

confidence: 99%

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

Tao,

Zhao,

Liu

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Also, the parameter count of this model is 5.92 M, which makes it time-consuming during training and inference. The architecture proposed in [1,2] employed a graph convolutional network (GCN) that has a time complexity of O (k × e × d + k × n × d 2 ), where the variables n, e, K, and d represent the total number of nodes, edges, layers, and dimensions of the node hidden features utilizing pose data as input.…”

Section: Computational Performance Analysismentioning

confidence: 99%

“…A lot of preprocessing is required to enhance the model's efficiency. These approaches could be more efficient in dynamic sign language gestures [1][2][3][4][5].…”

Section: Introductionmentioning

confidence: 99%

Isolated Video-Based Sign Language Recognition Using a Hybrid CNN-LSTM Framework Based on Attention Mechanism

Kumari,

Anand

2024

Electronics

View full text Add to dashboard Cite

Sign language is a complex language that uses hand gestures, body movements, and facial expressions and is majorly used by the deaf community. Sign language recognition (SLR) is a popular research domain as it provides an efficient and reliable solution to bridge the communication gap between people who are hard of hearing and those with good hearing. Recognizing isolated sign language words from video is a challenging research area in computer vision. This paper proposes a hybrid SLR framework that combines a convolutional neural network (CNN) and an attention-based long-short-term memory (LSTM) neural network. We used MobileNetV2 as a backbone model due to its lightweight structure, which reduces the complexity of the model architecture for deriving meaningful features from the video frame sequence. The spatial features are fed to LSTM optimized with an attention mechanism to select the significant gesture cues from the video frames and focus on salient features from the sequential data. The proposed method is evaluated on a benchmark WLASL dataset with 100 classes based on precision, recall, F1-score, and 5-fold cross-validation metrics. Our methodology acquired an average accuracy of 84.65%. The experiment results illustrate that our model performed effectively and computationally efficiently compared to other state-of-the-art methods.

show abstract

“…The WLASL dataset has the largest number of videos of American Sign Language hand gestures [73]. It has a total of 2000 hand gesture classes.…”

Section: Wlasl Datasetmentioning

confidence: 99%

Smart Home Automation-Based Hand Gesture Recognition Using Feature Fusion and Recurrent Neural Network

Alabdullah,

Ansar,

Mudawi

et al. 2023

Sensors

View full text Add to dashboard Cite

Gestures have been used for nonverbal communication for a long time, but human–computer interaction (HCI) via gestures is becoming more common in the modern era. To obtain a greater recognition rate, the traditional interface comprises various devices, such as gloves, physical controllers, and markers. This study provides a new markerless technique for obtaining gestures without the need for any barriers or pricey hardware. In this paper, dynamic gestures are first converted into frames. The noise is removed, and intensity is adjusted for feature extraction. The hand gesture is first detected through the images, and the skeleton is computed through mathematical computations. From the skeleton, the features are extracted; these features include joint color cloud, neural gas, and directional active model. After that, the features are optimized, and a selective feature set is passed through the classifier recurrent neural network (RNN) to obtain the classification results with higher accuracy. The proposed model is experimentally assessed and trained over three datasets: HaGRI, Egogesture, and Jester. The experimental results for the three datasets provided improved results based on classification, and the proposed system achieved an accuracy of 92.57% over HaGRI, 91.86% over Egogesture, and 91.57% over the Jester dataset, respectively. Also, to check the model liability, the proposed method was tested on the WLASL dataset, attaining 90.43% accuracy. This paper also includes a comparison with other-state-of-the art methods to compare our model with the standard methods of recognition. Our model presented a higher accuracy rate with a markerless approach to save money and time for classifying the gestures for better interaction.

show abstract

Signgraph: An Efficient and Accurate Pose-Based Graph Convolution Approach Toward Sign Language Recognition

Cited by 21 publications

References 51 publications

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

Isolated Video-Based Sign Language Recognition Using a Hybrid CNN-LSTM Framework Based on Attention Mechanism

Smart Home Automation-Based Hand Gesture Recognition Using Feature Fusion and Recurrent Neural Network

Contact Info

Product

Resources

About