Towards Indian Sign Language Sentence Recognition using INSIGNVID: Indian Sign Language Video Dataset

Mistree, Kinjal; Thakor, Devendra; Bhatt, Brijesh

doi:10.14569/ijacsa.2021.0120881

Cited by 7 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to the empirical analysis with all pretrained models, freezing less than 10 layers produced satisfactory results for sign recognition. Mistree et al (2020) describes the performance of static sign recognition using the MobileNetV2 model by keeping different layers trainable. The results in Table 7 show that the MobileNetV2 model outperforms other Using the proposed Indian sign language sentence recognition approach, an experiment was carried out on the INCLUDE dataset created by Sridhar et al (2020).…”

Section: Resultsmentioning

confidence: 99%

An approach based on deep learning for Indian sign language translation

Mistree

Thakor²,

Bhatt³

2022

IJICC

View full text Add to dashboard Cite

PurposeAccording to the Indian Sign Language Research and Training Centre (ISLRTC), India has approximately 300 certified human interpreters to help people with hearing loss. This paper aims to address the issue of Indian Sign Language (ISL) sentence recognition and translation into semantically equivalent English text in a signer-independent mode.Design/methodology/approachThis study presents an approach that translates ISL sentences into English text using the MobileNetV2 model and Neural Machine Translation (NMT). The authors have created an ISL corpus from the Brown corpus using ISL grammar rules to perform machine translation. The authors’ approach converts ISL videos of the newly created dataset into ISL gloss sequences using the MobileNetV2 model and the recognized ISL gloss sequence is then fed to a machine translation module that generates an English sentence for each ISL sentence.FindingsAs per the experimental results, pretrained MobileNetV2 model was proven the best-suited model for the recognition of ISL sentences and NMT provided better results than Statistical Machine Translation (SMT) to convert ISL text into English text. The automatic and human evaluation of the proposed approach yielded accuracies of 83.3 and 86.1%, respectively.Research limitations/implicationsIt can be seen that the neural machine translation systems produced translations with repetitions of other translated words, strange translations when the total number of words per sentence is increased and one or more unexpected terms that had no relation to the source text on occasion. The most common type of error is the mistranslation of places, numbers and dates. Although this has little effect on the overall structure of the translated sentence, it indicates that the embedding learned for these few words could be improved.Originality/valueSign language recognition and translation is a crucial step toward improving communication between the deaf and the rest of society. Because of the shortage of human interpreters, an alternative approach is desired to help people achieve smooth communication with the Deaf. To motivate research in this field, the authors generated an ISL corpus of 13,720 sentences and a video dataset of 47,880 ISL videos. As there is no public dataset available for ISl videos incorporating signs released by ISLRTC, the authors created a new video dataset and ISL corpus.

show abstract

Section: Resultsmentioning

confidence: 99%

An approach based on deep learning for Indian sign language translation

Mistree

Thakor²,

Bhatt³

2022

IJICC

View full text Add to dashboard Cite

show abstract

“…RNN (Recurrent Neural Network) provides more memory with the limitation of computational complexity. Kinjal et al [101] attempted to recognize ISL sentencelevel recognition. They converted the continuous gesture language into the text sentence of the English language.…”

Section: A Hybrid Models Using Deep Learning Methodsmentioning

confidence: 99%

A Detail Study of Sign Language Communication for Deaf-Mute to Normal Person

Pareshbhai Mansangbhai,

B. Vaghela

2024

IJCDS

View full text Add to dashboard Cite

Even though sign language recognition systems are crucial, there isn't a well-structured literature study to help Deaf-Mute. Approximately 160 research articles were found and reviewed in this survey of the sign recognition framework. From this pool, around 100 research articles were chosen, analyzed, and categorized. Each of these articles was grouped according to several criteria, such as datasets, kind of sign language, data collection methods, preprocessing, segmentation, and learning strategy. Notably, the largest study on sign language recognition utilized cameras and focused on steady, separated, and one-handed signs. Researchers are receiving increasing attention these days for creating cost-effective sign language recognition. According to the review, the main reason why commercialized sign language hasn't yet been developed as the current methods for identifying sign language either require expensive equipment or take too much computational time. The objective of this article is to provide an overview of recent developments in the field of computer vision, enabling other researchers to gain insights and potentially develop more efficient sign language systems.

show abstract

“…The studies focused least on the sentence of a sign language. Mistree et al (2021) focus on the sentence of Indian Sign Language recognition. ; Xu et al (2021) focus on the sentence of Chinese Sign Language.…”

Section: Referencementioning

confidence: 99%

A Systematic Literature Review on Vision-Based Hand Gesture for Sign Language Translation

Johari¹,

Ramli²,

Zulkoffli³

et al. 2023

jkukm

View full text Add to dashboard Cite

Deaf and hard of hearing people use sign language to communicate. People around mute and deaf people have difficulty communicating with each other if they do not understand sign language. This problem has prompted many researchers to conduct studies on sign language translation. However, there is a lack of compilation of SLR on this topic. Therefore, this paper aims to provide a thorough literature review of previous studies on sign language to text translation based on the vision method. PRISMA (Preferred Reporting Items to writing a standard Systematic Review and Meta-Analyses) is used in this systematic review. Two primary databases, Web of Science and Scopus, have been used to search for relevant articles and resources included in this systematic literature review. Based on the outcome of the systematic review of the topic, the primary studies on sign language translation systems were conducted using self-generated datasets more than public datasets. More static action sign language was studied compared to dynamic action sign language. For the type of recognition, more alphabet sign language was studied compared to digit, word, or sentence sign language. Other than that, most studies used digital cameras rather than Microsoft Kinect or a webcam. The most used classification method was Convolution Neural Network (CNN). The study is intended to guide readers and researchers for future research and knowledge enhancement in the field of sign language recognition.

show abstract

Towards Indian Sign Language Sentence Recognition using INSIGNVID: Indian Sign Language Video Dataset

Cited by 7 publications

References 36 publications

An approach based on deep learning for Indian sign language translation

An approach based on deep learning for Indian sign language translation

A Detail Study of Sign Language Communication for Deaf-Mute to Normal Person

A Systematic Literature Review on Vision-Based Hand Gesture for Sign Language Translation

Contact Info

Product

Resources

About