Retraction Note to: Machine learning based sign language recognition: a review and its research frontier

Elakkiya, R.

doi:10.1007/s12652-022-04314-w

Cited by 7 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sign word for predicted class represents each row, and each column represents the correct class in the confusion matrix. Our RTS invariant model produced good accuracy for all classes and correctly classified almost more than 95% accuracy except one class, and the maximum misclassification rate is 5% (mentioned in [4,8] location of the confusion matrix). The performance of the proposed model for each class, and there is a minimal amount of true negative and false positive while producing high accuracy.…”

Section: Performance Evaluation With Rts Version Dataset Of [4]mentioning

confidence: 86%

“…The importance of sign language recognition has increased because of the high growth rate of the deaf and hard-of-hearing population globally and the extended use of vision-based application devices [6,7]. In recent years, many researchers have proposed vision-based sign language recognition by utilizing inputs of the camera, such as 3d camera, web camera and stereo camera [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Rotation, Translation and Scale Invariant Sign Word Recognition Using Deep Learning

Miah¹,

Shin²,

Hasan³

et al. 2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Communication between people with disabilities and people who do not understand sign language is a growing social need and can be a tedious task. One of the main functions of sign language is to communicate with each other through hand gestures. Recognition of hand gestures has become an important challenge for the recognition of sign language. There are many existing models that can produce a good accuracy, but if the model test with rotated or translated images, they may face some difficulties to make good performance accuracy. To resolve these challenges of hand gesture recognition, we proposed a Rotation, Translation and Scale-invariant sign word recognition system using a convolutional neural network (CNN). We have followed three steps in our work: rotated, translated and scaled (RTS) version dataset generation, gesture segmentation, and sign word classification. Firstly, we have enlarged a benchmark dataset of 20 sign words by making different amounts of Rotation, Translation and Scale of the original images to create the RTS version dataset. Then we have applied the gesture segmentation technique. The segmentation consists of three levels, i) Otsu Thresholding with YCbCr, ii) Morphological analysis: dilation through opening morphology and iii) Watershed algorithm. Finally, our designed CNN model has been trained to classify the hand gesture as well as the sign word. Our model has been evaluated using the twenty sign word dataset, five sign word dataset and the RTS version of these datasets. We achieved 99.30% accuracy from the twenty sign word dataset evaluation, 99.10% accuracy from the RTS version of the twenty sign word evolution, 100% accuracy from the five sign word dataset evaluation, and 98.00% accuracy from the RTS version five sign word dataset evolution. Furthermore, the influence of our model exists in competitive results with state-of-the-art methods in sign word recognition.

show abstract

Section: Performance Evaluation With Rts Version Dataset Of [4]mentioning

confidence: 86%

Section: Introductionmentioning

confidence: 99%

Rotation, Translation and Scale Invariant Sign Word Recognition Using Deep Learning

Miah¹,

Shin²,

Hasan³

et al. 2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

show abstract

“…The conventional sensorbased approaches demand extra equipment to be worn by the signer. The use of data gloves, color gloves, depth cameras, and leap motion controllers creates additional overhead for the signer to communicate normally and poses huge limitations [41]. Although it gives good prediction results, drastically loses the scope in real time applications.…”

Section: Table 1: Comparison Of Existing Sl Recognition Frameworkmentioning

confidence: 99%

Development of an End-to-End Deep Learning Framework for Sign Language Recognition, Translation, and Video Generation

et al. 2022

Self Cite

View full text Add to dashboard Cite

The recent developments in deep learning techniques evolved to new heights in various domains and applications. The recognition, translation, and video generation of Sign Language (SL) still face huge challenges from the development perspective. Although numerous advancements have been made in earlier approaches, the model performance still lacks recognition accuracy and visual quality. In this paper, we introduce novel approaches for developing the complete framework for handling SL recognition, translation, and production tasks in real-time cases. To achieve higher recognition accuracy, we use the MediaPipe library and a hybrid Convolutional Neural Network + Bi-directional Long Short Term Memory (CNN + Bi-LSTM) model for pose details extraction and text generation. On the other hand, the production of sign gesture videos for given spoken sentences is implemented using a hybrid Neural Machine Translation (NMT) + MediaPipe + Dynamic Generative Adversarial Network (GAN) model. The proposed model addresses the various complexities present in the existing approaches and achieves above 95% classification accuracy. In addition to that, the model performance is tested in various phases of development, and the evaluation metrics show noticeable improvements in our model. The model has been experimented with using different multilingual benchmark sign corpus and produces greater results in terms of recognition accuracy and visual quality. The proposed model has secured a 38.06 average Bilingual Evaluation Understudy (BLEU) score, remarkable human evaluation scores, 3.46 average Fréchet Inception Distance to videos (FID2vid) score, 0.921 average Structural Similarity Index Measure (SSIM) values, 8.4 average Inception Score, 29.73 average Peak Signal-to-Noise Ratio (PSNR) score, 14.06 average Fréchet Inception Distance (FID) score, and an average 0.715 Temporal Consistency Metric (TCM) Score which is evidence of the proposed work.

show abstract

“…Two convincing ways to conceive such systems are: using special devices like Leap Motion Controller (LMC), Kinect sensors, or other motion sensors [4,5]. Another way is vision-based automatic inspection and analysis [6], where researchers perform machine learning and vision-based deep learning to understand Sign Language (SL) [7]. Techniques based on some devices did not arouse well, requiring signers to wear expensive additional devices.…”

Section: Introductionmentioning

confidence: 99%

A Novel Action Transformer Network for Hybrid Multimodal Sign Language Recognition

Javaid¹,

Rizvi²

2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Sign language fills the communication gap for people with hearing and speaking ailments. It includes both visual modalities, manual gestures consisting of movements of hands, and non-manual gestures incorporating body movements including head, facial expressions, eyes, shoulder shrugging, etc. Previously both gestures have been detected; identifying separately may have better accuracy, but much communicational information is lost. A proper sign language mechanism is needed to detect manual and non-manual gestures to convey the appropriate detailed message to others. Our novel proposed system contributes as Sign Language Action Transformer Network (SLATN), localizing hand, body, and facial gestures in video sequences. Here we are expending a Transformer-style structural design as a "base network" to extract features from a spatiotemporal domain. The model impulsively learns to track individual persons and their action context in multiple frames. Furthermore, a "head network" emphasizes hand movement and facial expression simultaneously, which is often crucial to understanding sign language, using its attention mechanism for creating tight bounding boxes around classified gestures. The model's work is later compared with the traditional identification methods of activity recognition. It not only works faster but achieves better accuracy as well. The model achieves overall 82.66% testing accuracy with a very considerable performance of computation with 94.13 Giga-Floating Point Operations per Second (G-FLOPS). Another contribution is a newly created dataset of Pakistan Sign Language for Manual and Non-Manual (PkSLMNM) gestures.

show abstract

Retraction Note to: Machine learning based sign language recognition: a review and its research frontier

Cited by 7 publications

References 0 publications

Rotation, Translation and Scale Invariant Sign Word Recognition Using Deep Learning

Rotation, Translation and Scale Invariant Sign Word Recognition Using Deep Learning

Development of an End-to-End Deep Learning Framework for Sign Language Recognition, Translation, and Video Generation

A Novel Action Transformer Network for Hybrid Multimodal Sign Language Recognition

Contact Info

Product

Resources

About