Chinese Sign Language Recognition Based on DTW-Distance-Mapping Features

Cheng, Juan; Wei, Fulin; Liu, Yu; Li, Chang; Chen, Qiang; Chen, Xun

doi:10.1155/2020/8953670

Cited by 5 publications

(5 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Tables 7-11 present the test results for precision, recall, and F1 scores for all the three classifiers on all five respective datasets. C 1 0.99 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 CL 2 0.00 0.96 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.02 0.00 0.00 0.00 0.00 0.00 CO 3 0.01 0.00 0.87 0.00 0.00 0.01 0.00 0.00 0.00 0.02 0.00 0.00 0.00 0.01 0.00 0.03 0.05 0.00 0.00 0.00 CR 4 0.00 0.00 0.00 0.97 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.00 0.00 F 5 0.00 0.00 0.00 0.00 0.99 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 H 6 0.03 0.00 0.01 0.00 0.00 0.85 0.01 0.00 0.02 0.00 0.02 0.00 0.00 0.01 0.00 0.02 0.03 0.00 0.00 0.00 HM 7 0.00 0.00 0.00 0.00 0.00 0.00 0.95 0.00 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.00 0.00 L 8 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.96 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.02 LV 9 0.00 0.00 0.01 0.00 0.00 0.03 0.00 0.00 0.94 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.97 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 S 13 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.00 0.96 0.00 0.00 0.00 0.00 0.00 0.00 0.01 ST 14 0.00 0.00 0.03 0.00 0.00 0.02 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.85 0.00 0.07 0.02 0.00 0.00 0.00 T 15 0.00 0.04 0.01 0.00 0.00 0.02 0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.89 0.00 0.01 0.00 0.00 0.00 WA 16 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.02 0.00 0.00 0.00 0.00 0.02 0.00 0.87 0.08 0.00 0.00 0.00 WO 17 0.00 0.00 0.04 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.05 0.89 0.02 0.00 0.00 Y 18 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.88 0.07 0.00 U 19 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.08 0.89 0.00 IL 20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.97…”

Section: Precision Recall and F1 Scoreunclassified

“…Gestures are extensively characterized as static and dynamic in a natural way of communication [9]. A static gesture is seen at the spurt of time, whereas a dynamic gesture changes with a time frame.…”

Section: Introductionmentioning

confidence: 99%

“…call,2 close,3 cold,4 correct,5 fine,6 help,7 home,8 like,9 love, 10 no,11 ok,12 please,13 single,14 sit,15 tall,16 wash,17 work,18 yes,19 you,20 iLoveYou.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Hand Gesture Recognition Based on Auto-Landmark Localization and Reweighted Genetic Algorithm for Healthcare Muscle Activities

et al. 2021

View full text Add to dashboard Cite

Due to the constantly increasing demand for the automatic localization of landmarks in hand gesture recognition, there is a need for a more sustainable, intelligent, and reliable system for hand gesture recognition. The main purpose of this study was to develop an accurate hand gesture recognition system that is capable of error-free auto-landmark localization of any gesture dateable in an RGB image. In this paper, we propose a system based on landmark extraction from RGB images regardless of the environment. The extraction of gestures is performed via two methods, namely, fused and directional image methods. The fused method produced greater extracted gesture recognition accuracy. In the proposed system, hand gesture recognition (HGR) is done via several different methods, namely, (1) HGR via point-based features, which consist of (i) distance features, (ii) angular features, and (iii) geometric features; (2) HGR via full hand features, which are composed of (i) SONG mesh geometry and (ii) active model. To optimize these features, we applied gray wolf optimization. After optimization, a reweighted genetic algorithm was used for classification and gesture recognition. Experimentation was performed on five challenging datasets: Sign Word, Dexter1, Dexter + Object, STB, and NYU. Experimental results proved that auto landmark localization with the proposed feature extraction technique is an efficient approach towards developing a robust HGR system. The classification results of the reweighted genetic algorithm were compared with Artificial Neural Network (ANN) and decision tree. The developed system plays a significant role in healthcare muscle exercise.

show abstract

Section: Precision Recall and F1 Scoreunclassified

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hand Gesture Recognition Based on Auto-Landmark Localization and Reweighted Genetic Algorithm for Healthcare Muscle Activities

et al. 2021

View full text Add to dashboard Cite

show abstract

“…For individuals in these random populations of disabled people, communication hurdles arise from factors such as sign language variance [6], sensor setup, classifying signs, study design, and the inability to comprehend or employ sign language. A continuous or dynamic sentence in sign language is broken into subwords via data segmentation [7], a crucial preprocessing operation. This operation is performed by detecting the hand's structure, orientation, and motion.…”

Section: Introductionmentioning

confidence: 99%

Progression Learning Convolution Neural Model-Based Sign Language Recognition Using Wearable Glove Devices

Liang,

Jettanasen,

Chiradeja

2024

Computation

View full text Add to dashboard Cite

Communication among hard-of-hearing individuals presents challenges, and to facilitate communication, sign language is preferred. Many people in the deaf and hard-of-hearing communities struggle to understand sign language due to their lack of sign-mode knowledge. Contemporary researchers utilize glove and vision-based approaches to capture hand movement and analyze communication; most researchers use vision-based techniques to identify disabled people’s communication because the glove-based approach causes individuals to feel uncomfortable. However, the glove solution successfully identifies motion and hand dexterity, even though it only recognizes the numbers, words, and letters being communicated, failing to identify sentences. Therefore, artificial intelligence (AI) is integrated with the sign language prediction system to identify disabled people’s sentence-based communication. Here, wearable glove-related sign language information is utilized to analyze the recognition system’s efficiency. The collected inputs are processed using progression learning deep convolutional neural networks (PLD-CNNs). The technique known as progression learning processes sentences by dividing them into words, creating a training dataset. The model assists in efforts to understand sign language sentences. A memetic optimization algorithm is used to calibrate network performance, minimizing recognition optimization problems. This process maximizes convergence speed and reduces translation difficulties, enhancing the overall learning process. The created system is developed using the MATLAB (R2021b) tool, and its proficiency is evaluated using performance metrics. The experimental findings illustrate that the proposed system works by recognizing sign language movements with excellent precision, recall, accuracy, and F1 scores, rendering it a powerful tool in the detection of gestures in general and sign-based sentences in particular.

show abstract

“…The commonly used algorithms based on patterns include SVM (Support Vector Machine), Gaussian mixture model (GMM), neural network, etc. Among the distance-based algorithms, the most widely used is DTW (dynamic time warping) [3,4]. In this paper, we combine the design ideas of model-based and time-based pattern recognition algorithms, adopt LSTM neural network as the basic training model, and use DTW template matching verification.…”

Section: Introductionmentioning

confidence: 99%

Design of material handling robot based on LSTM+DTW algorithm

Zhao,

Zhang

et al. 2023

International Conference on Automation Control, Algorithm, and Intelligent Bionics (ACAIB 2023)

View full text Add to dashboard Cite

For the sake of satisfying the needs of material handling in intelligent logistics warehouses and industrial buildings better and improving the deficiencies of existing material handling forms, in this paper, we design an intelligent material handling robot servo system that integrates DTW (Dynamic time warping) and LSTM (Long Short Term Memory) algorithms. The operator can command the robot to move in multiple directions according to the motion intention of the human body through the human manipulative mode based on the hand movement or the rod traction mode with weak traction. This project collects various motion pattern data through experiments. After preprocessing the data, features are extracted and repeated motion pattern recognition training and template matching are carried out to deeply learn and perceive human intention. We input the data collected in real time into the generated LSTM network model, identify the motion pattern in real time, and introduce the DTW algorithm for template matching verification. Through testing, the accuracy of motion pattern recognition can exceed 95% under the effective combination of the two algorithms, which is higher than that of the DTW and LSTM algorithms.

show abstract

Chinese Sign Language Recognition Based on DTW-Distance-Mapping Features

Cited by 5 publications

References 44 publications

Hand Gesture Recognition Based on Auto-Landmark Localization and Reweighted Genetic Algorithm for Healthcare Muscle Activities

Hand Gesture Recognition Based on Auto-Landmark Localization and Reweighted Genetic Algorithm for Healthcare Muscle Activities

Progression Learning Convolution Neural Model-Based Sign Language Recognition Using Wearable Glove Devices

Design of material handling robot based on LSTM+DTW algorithm

Contact Info

Product

Resources

About