A Signer Independent Sign Language Recognition with Co-articulation Elimination from Live Videos: An Indian Scenario

Athira, P.K.; Sruthi, C. J.; Lijiya, A

doi:10.1016/j.jksuci.2019.05.002

Cited by 77 publications

(27 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recorded video clips of continuing signed letters were processed according to our proposed method, and experimental results in average accuracy rate are shown in Table III. In that table, the experimental results of our proposed method are compared with conventional methods based on isolated signed letters using forehand in 2D [10] and 3D [19], and backhand in 2D [22], as shown in the to 4 ℎ rows, and continuous signed letters using forehand in 2D [42,43] and 3D [46], as shown in the 5 ℎ to 7 ℎ rows. The experimental results of the proposed method based on backhand view in 3D is allocated in the bottom row.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…To show the improvement in accuracy of each signed letter, recognition results of the proposed method are compared with conventional methods, as shown in Table IV. The average accuracy rates of each signed letter performed by conventional methods using isolated signed letter in 2D for forehand [10], and backhand [22], and forehand in 3D [19] are shown in columns 2-4, and the ones with the method using continuous signed letters for forehand in 2D [43] are revealed in the 5 ℎ column. Finally, the accuracy of each signed letter and its standard deviation performed by the proposed method are introduced in the 6 ℎ and the last columns.…”

Section: Experiments and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Backhand-View-Based Continuous-Signed-Letter Recognition Using a Rewound Video Sequence and the Previous Signed-Letter Information

Chophuk

Chamnongthai

2021

IEEE Access

View full text Add to dashboard Cite

In sign language, when signed letters are continuously spelled based on backhand view, a previous signed letter influences the trajectory of fingers and a hand to approach the pause duration for signing the current signed letter. Since those varied trajectories are regarded as parts of the current signed letter, hand gesture during pause duration of the current signed letter is regarded as insufficient for recognition of the current signed letter. The previous signed letters, trajectories of fingers, and hand between the previous and the current signed letters should be included as data for classification. This paper proposes a method of backhand-view-based continuous-signed-letter recognition using a rewound video sequence with previous signed letter. In the method, a hand shape of previous signed letter and trajectories of finger joints moving from the previous signed letter to the current one are detected, features are then extracted, and finally, the features are classified for signed letter recognition. To evaluate performance of the proposed method, experiments with 10 participants were performed 20 times each, and the results revealed 96.07% accuracy approximately which were improved significantly from the conventional methods using forehand and backhand.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Experiments and Resultsmentioning

confidence: 99%

Backhand-View-Based Continuous-Signed-Letter Recognition Using a Rewound Video Sequence and the Previous Signed-Letter Information

Chophuk

Chamnongthai

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The traditional approach to HGR involves extraction of spatial or spatio-temporal features from image sequences followed by classification. Some of the prominent works in this direction include, but are not limited to, Chinese finger sign language recognition with gray-level co-occurrence matrix features and k-nearest neighbor (KNN) classifier by Jiang et al [ 13 ], HGR model with infrared information captured with the leap motion controller and machine learning techniques like KNN, support vector machine (SVM) and decision trees by Nogales and Benalcazar [ 14 ], Grassmann manifold-based discriminant analysis model with the finger tip-based hand trajectory features extracted through either the depth or skeleton information [ 15 ], neural network (NN) model with the feature vectors extracted through video summarization technique for Peruvian sign language recognition [ 16 ], support vector machine (SVM) classification of the shape and trajectory features extracted via 3-D hand skeleton data [ 17 ], SVM model with combined shape and trajectory information by Bai et al [ 18 ], NN model with the textural feature descriptors presented by Agab and Chelali [ 19 ], local binary pattern features with hidden markov model (HMM) for Arabic sign language recognition (ArSLR) by Ahmed et al [ 20 ], artificial neural network (ANN) model with the discrete cosine transform (DCT) features extracted from the selfie video sequences of ISL gestures by Rao et al [ 21 ], and the multiclass SVM model trained with hand shape and trajectory features for ISL words by Athira et al [ 22 ].…”

Section: Related Workmentioning

confidence: 99%

Applying Hybrid Deep Neural Network for the Recognition of Sign Language Words Used by the Deaf COVID-19 Patients

Venugopalan

Reghunadhan

2022

Arab J Sci Eng

View full text Add to dashboard Cite

The rapid spread of the novel corona virus disease (COVID-19) has disrupted the traditional clinical services all over the world. Hospitals and healthcare centers have taken extreme care to minimize the risk of exposure to the virus by restricting the visitors and relatives of the patients. The dramatic changes happened in the healthcare norms have made it hard for the deaf patients to communicate and receive appropriate care. This paper reports a work on automatic sign language recognition that can mitigate the communication barrier between the deaf patients and the healthcare workers in India. Since hand gestures are the most expressive components of a sign language vocabulary, a novel dataset of dynamic hand gestures for the Indian sign language (ISL) words commonly used for emergency communication by deaf COVID-19 positive patients is proposed. A hybrid model of deep convolutional long short-term memory network has been utilized for the recognition of the proposed hand gestures and achieved an average accuracy of 83.36%. The model performance has been further validated on an alternative ISL dataset as well as a benchmarking hand gesture dataset and obtained average accuracies of and , respectively.

show abstract

“…Most of these studies extract specific features and then use machine learning algorithms to classify the SL images. Many different SL have been used in these studies, namely American SL [1][2][3][4][5][6][7][8] , Arabic SL [9][10][11][12] , British SL 13 , Chinese SL 14 , German SL 15,16 , Indian SL 17 , Irish SL 18 , Pakistani SL [19][20][21][22][23][24] , Persian SL 25 , and more in combination such as American & German SL 26 , American & Thai SL 27 and American & Indian SL 28 .…”

Section: Literature Reviewmentioning

confidence: 99%

Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

Mirza

Munaf

Ali

et al. 2022

Preprint

View full text Add to dashboard Cite

In order to perform their daily activities, a person is required to communicating with others. This can be a major obstacle for the deaf population of the world, who communicate using sign languages (SL). Pakistani Sign Language (PSL) is used by more than 250,000 deaf Pakistanis. Developing a SL recognition system would greatly facilitate these people. This study aimed to collect data of static and dynamic PSL alphabets and to develop a vision-based system for their recognition using Bag-of-Words (BoW) and Support Vector Machine (SVM) techniques. A total of 5,120 images for 36 static PSL alphabet signs and 353 videos with 45,224 frames for 3 dynamic PSL alphabet signs were collected from 10 native signers of PSL. The developed system used the collected data as input, resized the data to various scales and converted the RGB images into grayscale. The resized grayscale images were segmented using Thresholding technique and features were extracted using Speeded Up Robust Feature (SURF). The obtained SURF descriptors were clustered using K-means clustering. A BoW was obtained by computing the Euclidean distance between the SURF descriptors and the clustered data. The codebooks were divided into training and testing using 5-fold cross validation. The highest overall classification accuracy for static PSL signs was 97.80% at 750×750 image dimensions and 500 Bags. For dynamic PSL signs a 96.53% accuracy was obtained at 480×270 video resolution and 200 Bags.

show abstract

A Signer Independent Sign Language Recognition with Co-articulation Elimination from Live Videos: An Indian Scenario

Cited by 77 publications

References 14 publications

Backhand-View-Based Continuous-Signed-Letter Recognition Using a Rewound Video Sequence and the Previous Signed-Letter Information

Backhand-View-Based Continuous-Signed-Letter Recognition Using a Rewound Video Sequence and the Previous Signed-Letter Information

Applying Hybrid Deep Neural Network for the Recognition of Sign Language Words Used by the Deaf COVID-19 Patients

Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

Contact Info

Product

Resources

About