American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation

Abstract

Sign language is designed to assist the deaf and hard of hearing community to convey messages and connect with society. Sign language recognition has been an important domain of research for a long time. Previously, sensor-based approaches have obtained higher accuracy than vision-based approaches. Due to the cost-effectiveness of vision-based approaches, researchers have been conducted here also despite the accuracy drop. The purpose of this research is to recognize American sign characters using hand images obtained from a web camera. In this work, the media-pipe hands algorithm was used for estimating hand joints from RGB images of hands obtained from a web camera and two types of features were generated from the estimated coordinates of the joints obtained for classification: one is the distances between the joint points and the other one is the angles between vectors and 3D axes. The classifiers utilized to classify the characters were support vector machine (SVM) and light gradient boosting machine (GBM). Three character datasets were used for recognition: the ASL Alphabet dataset, the Massey dataset, and the finger spelling A dataset. The results obtained were 99.39% for the Massey dataset, 87.60% for the ASL Alphabet dataset, and 98.45% for Finger Spelling A dataset. The proposed design for automatic American sign language recognition is cost-effective, computationally inexpensive, does not require any special sensors or devices, and has outperformed previous studies.

TFS Recognition: Investigating MPH]{Thai Finger Spelling Recognition: Investigating MediaPipe Hands Potentials

“…MPH has been employed to address finger spelling recognition in many other sign languages with high accuracy [18,19]. Shin et al [18] address American Finger Spelling (AFS) using MPH to extract 21 hand keypoints that are converted to distance and angle features.…”

“…MPH has been employed to address finger spelling recognition in many other sign languages with high accuracy [18,19]. Shin et al [18] address American Finger Spelling (AFS) using MPH to extract 21 hand keypoints that are converted to distance and angle features. The features then are classified into AFS signs using Support Vector Machine (SVM) [20] and light Gradient Boosting Machine (GBM) [21].…”

American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven Multi-Stacked Deep LSTM

“…Research works in dynamic sign word recognition are grouped into single- and double-hand dynamic sign words. Works in single-hand [ 5 , 14 , 15 , 16 , 17 ] dynamic sign words are limited due to the fact most of the sign letters or alphabets are less complex. Double-hand sign words are commonly used in daily communications, but complex hand motion interactions are major challenge.…”

AnyGesture: Arbitrary One-Handed Gestures for Augmented, Virtual, and Mixed Reality Applications

“…Recognizing air-writing is introduced by Chen et al [14] and a character recognition system-based on finger-joint tracking is introduced by Alam et al [15]. Hand gestures also have an important role in the the recognition of sign language, as used by Shin et al [16]. New applications based on hand gestures are constantly being developed.…”