An Efficient Feature Extraction of Isolated Word for Dynamic Sign Language Classification

Aldelfy, Hussein Ali; Al-Mufraji, Mahmood Hamza; Saeed, Thamir R.

doi:10.1109/scee.2018.8684044

Cited by 1 publication

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The vision-based group consists of 2D and 3D approaches, as shown in Table 1. The 2D method [21][22][23][24][25][26][27][28][29][30][31][32][33] uses video and image data for sign language interpretation. For example, a frequency-based model [21,22] has been proposed which uses a histogram, discrete wavelet transform, and a Gabor filter for extracting features from 2D images.…”

Section: Related Workmentioning

confidence: 99%

“…Another proposed approach is the deformable gabarit model [23,24], which uses an active contour and the hand shape as features. Subsequently, the motion-based model [25] was proposed, which uses the motion of the hand as a feature via hand trajectory tracking. Finally, the deep-based model [26][27][28][29][30][31][32][33][34][35] has been proposed, which uses a novel deep-learning-based architecture to learn spatial hierarchies of features automatically and adaptively for 2D images, such as the convolutional neural network model (CNN), the convolutional neural network-long short-term memory model (CNN-LSTM), and recurrent neural networks (RNN).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Backhand-Approach-Based American Sign Language Words Recognition Using Spatial-Temporal Body Parts and Hand Relationship Patterns

Chophuk

Chamnongthai

Chinnasarn

2022

Sensors

View full text Add to dashboard Cite

Most of the existing methods focus mainly on the extraction of shape-based, rotation-based, and motion-based features, usually neglecting the relationship between hands and body parts, which can provide significant information to address the problem of similar sign words based on the backhand approach. Therefore, this paper proposes four feature-based models. The spatial–temporal body parts and hand relationship patterns are the main feature. The second model consists of the spatial–temporal finger joint angle patterns. The third model consists of the spatial–temporal 3D hand motion trajectory patterns. The fourth model consists of the spatial–temporal double-hand relationship patterns. Then, a two-layer bidirectional long short-term memory method is used to deal with time-independent data as a classifier. The performance of the method was evaluated and compared with the existing works using 26 ASL letters, with an accuracy and F1-score of 97.34% and 97.36%, respectively. The method was further evaluated using 40 double-hand ASL words and achieved an accuracy and F1-score of 98.52% and 98.54%, respectively. The results demonstrated that the proposed method outperformed the existing works under consideration. However, in the analysis of 72 new ASL words, including single- and double-hand words from 10 participants, the accuracy and F1-score were approximately 96.99% and 97.00%, respectively.

show abstract