RETRACTED ARTICLE: Machine learning based sign language recognition: a review and its research frontier

Elakkiya, R.

doi:10.1007/s12652-020-02396-y

Cited by 52 publications

(29 citation statements)

References 189 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each data points placed in a latent space contribute some portion of the video to establish the sequential relationship that exists between sign gestures, helps to automate the high-quality video generation process. The earlier approaches [1][2][3][4][5][6][7][8][9][10] discusses the generation of images or videos from noise vector by randomly selecting some data points. Due to a lack of efficient training process and various factors, these models mostly produce blurry and inconsistent results.…”

Section: Related Workmentioning

confidence: 99%

Dynamic GAN for High-Quality Sign Language Video Generation from Skeletal poses using Generative Adversarial Networks

Natarajan

Elakkiya

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The emergence of unsupervised generative models has resulted in greater performance in image and video generation tasks. However, existing generative models pose huge challenges in high-quality video generation process due to blurry and inconsistent results. In this paper, we introduce a novel generative framework named Dynamic Generative Adversarial Networks (Dynamic GAN) model for regulating the adversarial training and generating photorealistic high-quality sign language videos from skeletal poses. The proposed model comprises three stages of development such as generator network, classification and image quality enhancement and discriminator network. In the generator fold, the model generates samples similar to real images using random noise vectors, the classification of generated samples are carried out using the VGG-19 model and novel techniques are employed for improving the quality of generated samples in the second fold of the model and finally the discriminator networks fold identifies the real or fake samples. Unlike, existing approaches the proposed novel framework produces photo-realistic video quality results without using any animation or avatar approaches. To evaluate the model performance qualitatively and quantitatively, the proposed model has been evaluated using three benchmark datasets that yield plausible results. The datasets are RWTH-PHOENIX-Weather 2014T dataset, and our self-created dataset for Indian Sign Language (ISL-CSLTR), and the UCF-101 Action Recognition dataset. The output samples and performance metrics show the outstanding performance of our model.

show abstract

Section: Related Workmentioning

confidence: 99%

Dynamic GAN for High-Quality Sign Language Video Generation from Skeletal poses using Generative Adversarial Networks

Natarajan

Elakkiya

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sensor-based approaches extracts the hand measurements, i.e., joints orientation, hands position, and hand velocity [ 11 ], and can be conducted using microcontrollers and specific sensors, such as data gloves [ 12 , 13 ], power gloves [ 14 ], digital camera [ 15 ], accelerometer [ 16 , 17 ], depth camera [ 18 ], Kinect [ 19 ], leap motion controller [ 20 ], dexterous master gloves [ 21 ], etc. The advantage of the sensor-based approach is the higher recognition rate because of the skeletal data [ 22 ]. However, sensor-based approaches are expensive, allow limited movement, require specialized devices, environment, and training to utilize the systems fully [ 22 ].…”

Section: Introductionmentioning

confidence: 99%

“…The advantage of the sensor-based approach is the higher recognition rate because of the skeletal data [ 22 ]. However, sensor-based approaches are expensive, allow limited movement, require specialized devices, environment, and training to utilize the systems fully [ 22 ]. There is also a risk that noise will reduce the recognition rate of sensor-based systems as sensors, such as accelerometers are sensitive to noise, and even a slight movement can be identified as a waveform [ 22 ].…”

Section: Introductionmentioning

confidence: 99%

“…However, sensor-based approaches are expensive, allow limited movement, require specialized devices, environment, and training to utilize the systems fully [ 22 ]. There is also a risk that noise will reduce the recognition rate of sensor-based systems as sensors, such as accelerometers are sensitive to noise, and even a slight movement can be identified as a waveform [ 22 ]. Hence, researchers have proposed vision-based approaches in recent years by utilizing inputs of the camera, such as web camera, stereo camera, or 3D camera [ 23 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation

Shin

Matsuoka

Hasan

et al. 2021

Sensors

View full text Add to dashboard Cite

Sign language is designed to assist the deaf and hard of hearing community to convey messages and connect with society. Sign language recognition has been an important domain of research for a long time. Previously, sensor-based approaches have obtained higher accuracy than vision-based approaches. Due to the cost-effectiveness of vision-based approaches, researchers have been conducted here also despite the accuracy drop. The purpose of this research is to recognize American sign characters using hand images obtained from a web camera. In this work, the media-pipe hands algorithm was used for estimating hand joints from RGB images of hands obtained from a web camera and two types of features were generated from the estimated coordinates of the joints obtained for classification: one is the distances between the joint points and the other one is the angles between vectors and 3D axes. The classifiers utilized to classify the characters were support vector machine (SVM) and light gradient boosting machine (GBM). Three character datasets were used for recognition: the ASL Alphabet dataset, the Massey dataset, and the finger spelling A dataset. The results obtained were 99.39% for the Massey dataset, 87.60% for the ASL Alphabet dataset, and 98.45% for Finger Spelling A dataset. The proposed design for automatic American sign language recognition is cost-effective, computationally inexpensive, does not require any special sensors or devices, and has outperformed previous studies.

show abstract

“…In this work, we propose an attention-enhanced multi-scale and dual Sign Language Recognition Network based on Graph Convolution Network (GCN), which is capable if matching the performance of the state-of-the-art on two large Chinese sign language datasets. A large body of work has been proposed for sign language recognition (SLR) [ 2 , 3 ]. Before 2016, the traditional sign language recognition technology based on vision has been studied extensively, see [ 4 ] for details.…”

Section: Introductionmentioning

confidence: 99%

An Attention-Enhanced Multi-Scale and Dual Sign Language Recognition Network Based on a Graph Convolution Network

Lü

2021

Sensors

View full text Add to dashboard Cite

Sign language is the most important way of communication for hearing-impaired people. Research on sign language recognition can help normal people understand sign language. We reviewed the classic methods of sign language recognition, and the recognition accuracy is not high enough because of redundant information, human finger occlusion, motion blurring, the diversified signing styles of different people, and so on. To overcome these shortcomings, we propose a multi-scale and dual sign language recognition Network (SLR-Net) based on a graph convolutional network (GCN). The original input data was RGB videos. We first extracted the skeleton data from them and then used the skeleton data for sign language recognition. SLR-Net is mainly composed of three sub-modules: multi-scale attention network (MSA), multi-scale spatiotemporal attention network (MSSTA) and attention enhanced temporal convolution network (ATCN). MSA allows the GCN to learn the dependencies between long-distance vertices; MSSTA can directly learn the spatiotemporal features; ATCN allows the GCN network to better learn the long temporal dependencies. The three different attention mechanisms, multi-scale attention mechanism, spatiotemporal attention mechanism, and temporal attention mechanism, are proposed to further improve the robustness and accuracy. Besides, a keyframe extraction algorithm is proposed, which can greatly improve efficiency by sacrificing a little accuracy. Experimental results showed that our method can reach 98.08% accuracy rate in the CSL-500 dataset with a 500-word vocabulary. Even on the challenging dataset DEVISIGN-L with a 2000-word vocabulary, it also reached a 64.57% accuracy rate, outperforming other state-of-the-art sign language recognition methods.

show abstract

RETRACTED ARTICLE: Machine learning based sign language recognition: a review and its research frontier

Cited by 52 publications

References 189 publications

Dynamic GAN for High-Quality Sign Language Video Generation from Skeletal poses using Generative Adversarial Networks

Dynamic GAN for High-Quality Sign Language Video Generation from Skeletal poses using Generative Adversarial Networks

American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation

An Attention-Enhanced Multi-Scale and Dual Sign Language Recognition Network Based on a Graph Convolution Network

Contact Info

Product

Resources

About