Real-time Bhutanese Sign Language digits recognition system using Convolutional Neural Network

Wangchuk, Karma; Riyamongkol, Panomkhawn; Waranusast, Rattapoom

doi:10.1016/j.icte.2020.08.002

Cited by 22 publications

(9 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fine-tuning will provide the ability to use a smaller size dataset in networks. In the study of Wangchuk et al [6], the authors proposed to use CNN to extract features from images and classify digits with the trained model in real-time using the webcam. Also, Kadhim and Khamees [7] built A Real-Time American Sign Language Recognition System by a multi-classification system.…”

Section: Recognition Methodsmentioning

confidence: 99%

“…Letters (32) 25,600 image Arabic [5] Numbers (0-9) 20000 image Bhutanese [6] Letters (26) 61614 image American [7] Letters (23), numbers (0-10), words (67) 35,000 image English [8] Word (15) 13,500 image American [9] Word(20) 6600 video Italian [2] Letters (30) and numbers (1-5) 1400 video German [10] Letters (H -J) and words (8) 300 video Brazilian [11] Words (25) 200 video Arabic [12] Word(40) 8000 video Arabic [13] Words (30) 1500 video Argentinian [14] Words (10) 1080 video Indian [15] Letters (26) 34627 image American [16] Words (5) 500 video Thai [17] Paper-Dynamic Sign Language Recognition Based on Real-Time Videos…”

Section: Size Language Refmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Sign Language Recognition Based on Real-Time Videos

et al. 2022

View full text Add to dashboard Cite

Sign language is the main communication tool for the deaf and hard of hearing. Deaf people cannot interact with others without a sign language interpreter. Accordingly, sign language recognition automation has become an important application in artificial intelligence and deep learning. Specifically, the recognition of Arabic sign language has been studied using many smart and traditional methods. This research provides a system to recognize dynamic Saudi sign language based on real time videos to solve this problem. We constructed a dataset for Saudi sign language in terms of videos in the proposed system. The dataset was then used to train a deep learning model using convolutional long short-term memory (convLSTM) to recognize the dynamic signs. Implementing such a system provides a platform for deaf people to interact with the rest of the world without an interpreter to reduce deaf isolation in society.

show abstract

Section: Recognition Methodsmentioning

confidence: 99%

Section: Size Language Refmentioning

confidence: 99%

Dynamic Sign Language Recognition Based on Real-Time Videos

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The networks were fed by images of various Arabic Sign Language data and were able to achieve an accuracy of approximately 99%. The convolutional neural network (CNN) and a dataset of 20,000 sign images of 10 static digits were used in research [15] to build the BSL digits recognition system. The proposed CNN model was compared to a number of other sign language models.…”

Section: Related Workmentioning

confidence: 99%

Static hand gesture recognition of Arabic sign language by using deep CNNs

Ismail

Dawwd

Ali

2021

IJEECS

View full text Add to dashboard Cite

An Arabic sign language recognition using two concatenated deep convolution neural network models DenseNet121 & VGG16 is presented. The pre-trained models are fed with images, and then the system can automatically recognize the Arabic sign language. To evaluate the performance of concatenated two models in the Arabic sign language recognition, the red-green-blue (RGB) images for various static signs are collected in a dataset. The dataset comprises 220,000 images for 44 categories: 32 letters, 11 numbers (0:10), and 1 for none. For each of the static signs, there are 5000 images collected from different volunteers. The pre-trained models were used and trained on prepared Arabic sign language data. These models were used after some modification. Also, an attempt has been made to adopt two models from the previously trained models, where they are trained in parallel deep feature extractions. Then they are combined and prepared for the classification stage. The results demonstrate the comparison between the performance of the single model and multi-model. It appears that most of the multi-model is better in feature extraction and classification than the single models. And also show that when depending on the total number of incorrect recognize sign image in training, validation and testing dataset, the best convolutional neural networks (CNN) model in feature extraction and classification Arabic sign language is the DenseNet121 for a single model using and DenseNet121 & VGG16 for multi-model using.

show abstract

“…However, it should be noted that the use of specific gestures or sign languages is not universal, and varies from one region to another and among different ethnic communities worldwide [ 3 ]. In addition, learning multiple sign languages is complex and may not be possible for a majority of the public [ 4 ]. It is impossible for people with speech impairment to learn spoken language; this means that it is a problem for hearing-impaired people both to communicate with other people who are not conversant with sign language and to communicate among themselves.…”

Section: Introductionmentioning

confidence: 99%

“…A shared sign language (as opposed to communicating through gestures) could be the best way to solve this problem, as it is one of the most common tools used to this end. However, over 120 sign languages are used worldwide, and it is not straightforward for society at large to learn these languages and communicate effectively using them [ 4 ]. Researchers and engineers have tried to develop sign language recognition systems to narrow this gap, for example, a conduct-based system known as a glove sensor [ 16 ]; however, these systems require hardware setups that are complex and relatively expensive, and therefore they are not preferred [ 15 ].…”

Section: Introductionmentioning

confidence: 99%

Sign and Human Action Detection Using Deep Learning

2022

View full text Add to dashboard Cite

Human beings usually rely on communication to express their feeling and ideas and to solve disputes among themselves. A major component required for effective communication is language. Language can occur in different forms, including written symbols, gestures, and vocalizations. It is usually essential for all of the communicating parties to be fully conversant with a common language. However, to date this has not been the case between speech-impaired people who use sign language and people who use spoken languages. A number of different studies have pointed out a significant gaps between these two groups which can limit the ease of communication. Therefore, this study aims to develop an efficient deep learning model that can be used to predict British sign language in an attempt to narrow this communication gap between speech-impaired and non-speech-impaired people in the community. Two models were developed in this research, CNN and LSTM, and their performance was evaluated using a multi-class confusion matrix. The CNN model emerged with the highest performance, attaining training and testing accuracies of 98.8% and 97.4%, respectively. In addition, the model achieved average weighted precession and recall of 97% and 96%, respectively. On the other hand, the LSTM model’s performance was quite poor, with the maximum training and testing performance accuracies achieved being 49.4% and 48.7%, respectively. Our research concluded that the CNN model was the best for recognizing and determining British sign language.

show abstract

Real-time Bhutanese Sign Language digits recognition system using Convolutional Neural Network

Cited by 22 publications

References 4 publications

Dynamic Sign Language Recognition Based on Real-Time Videos

Dynamic Sign Language Recognition Based on Real-Time Videos

Static hand gesture recognition of Arabic sign language by using deep CNNs

Sign and Human Action Detection Using Deep Learning

Contact Info

Product

Resources

About