Recognition Bangla Sign Language using Convolutional Neural Network

Islalm, Shafiqul; Rahman, M. M. Hafizur; Rahman, Md. Hafizur; Arifuzzaman, Md; Sassi, Roberto; Aktaruzzaman, Md.

doi:10.1109/3ict.2019.8910301

Cited by 36 publications

(18 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many researchers applied CNN to recognize BdSL. Islam et al [19], Hoque et al [20], and Islam et al [21] utilized CNN to recognize two-handed BdSL alphabets, whereas, Rony et al [22], Hossen et al [23], and Rafi et al [24] implemented CNN to recognize one-handed…”

Section: Literature Reviewmentioning

confidence: 99%

Design and Development of a Humanoid Robot for Sign Language Interpretation

et al. 2021

View full text Add to dashboard Cite

Purpose With an indisputable complexity of communication for hearing and speaking impaired people, most sign language recognition systems utilize virtual reality or onscreen robots. This paper presents the design and development of a special and low-cost humanoid robot that can perform as a sign language interpreter. To the best of our knowledge, this is the first endeavor to fabricate a humanoid robot for Bangla sign language (BdSL) and Medical signs interpretation. Methods Considering the plethora of design criteria and balancing between rigidity and flexibility 3D models of the robotics parts are designed and 3D printed ensuring cost efficiency. With the help of modern fabrication technology, the robot is developed and assembled with proper actuators and circuitry. An image dataset is built comprising 950 images for BdSL recognition and made publicly available. We utilized the Recurrent neural network (RNN) and Convolutional neural network (CNN) for deep learning model establishment and feature extraction from video and image data. Results The developed humanoid robot has 43 Degrees of freedom (DoF) which includes two 15 DoF hands. It can imitate 16 BdSL alphabets in sign language, can capture a video or image input in real-time from the user, and recognize 10 medical signs and 38 alphabets of BdSL. The learning model for video-based medical sign recognition achieved 87.5% test accuracy. Image-based Bangla sign language recognition achieved an overall test accuracy of 98.19% in our dataset and 93.8% in another available dataset. Conclusion Compared to the state-of-the-art robotic systems for sign language interpretation, our approach has achieved higher kinematic characteristics, remarkable results in sign recognition, and impressive competency in sign imitation; all at almost 10 times lower cost than the state-of-the-art systems. The results are evidence that our approach is efficient and suitable in helping hearing and speaking impaired people. Moreover, this work initiates a research scope that can be further extended for creating equal opportunities for the hearing and speaking impaired community.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Design and Development of a Humanoid Robot for Sign Language Interpretation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Most of the works for BSL used CNN-based architecture along with softmax loss for the classification of images. Shafiqul et al [18], made the extensive dataset for BSL, consisting of 30916 samples which were collected from 25 different students. They developed a CNN-based architecture and reported an accuracy of 99.83%, 100%, and 99.80%, respectively, for basic characters and numerals and combined usage.…”

Section: B Deep Learning Techniquesmentioning

confidence: 99%

“…For augmented Ishara-Lipi data, the training set comprises 18968 samples, and the test set contains 978 images. Initially, we have investigated the works of [17] and [18] by classifying the 35 hand gestures in intra-datasets on the VGG16 network using cross-entropy loss. Afterward, we explore the phenomenon of generalization by assessing interdataset performance while maintaining all hyper-parameters constant.…”

Section: B Implementation Detailsmentioning

confidence: 99%

“…The dataset of [17] and [18] are mainly created for the classification of the BSL. In this subsection, we have analyzed the performance of several architectures for the classification of 35 distinct classes.…”

Section: B Performance Of Classificationmentioning

confidence: 99%

“…This amount of data is inadequate for robust and effective deep learning models. Shafiqul et al [18] constructed a dataset for BSL containing 30916 samples, of which 23864 are alphabets and 7052 are digits. This is the extensive dataset on BSL that we could find when we initiated our study.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generalization of Bangla Sign Language Recognition Using Angular Loss Functions

Youme¹,

Chowdhury

Ahamed

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Sign Language provides the means of conveying messages for deaf and mute people. Effective communication with the masses is a great challenge for the deaf and mute community, as Sign Language is not commonly understood. Many researchers have done numerous works in foreign language datasets like English, French, Japanese, etc. However, for Bangla, one of the most widely spoken languages, much significant work has not been done yet. Most of the works on Bangla Sign Language are executed on small datasets and report satisfactory performance. However, when small datasets are evaluated from the perspective of generalizability, particularly when using deep learning based solutions, these models fail to reproduce to the same performance. Therefore, this paper poses inter-dataset evaluation as the main evaluation criteria and evaluates several deep learning based models. This evaluation is done for Bangla by leveraging two popular datasets of Bangla Sign Language. Unsurprisingly, the inter-dataset performance is inferior, and several approaches to improve are explored and documented, including the use of angular margin based loss functions. The results demonstrate the importance of such an evaluation and also show that one of the proposed approaches shows promising performance, albeit with significant room for improvement. This raises the need for a standardized dataset to overcome this issue of generalization for real-life applications, as well as the need to encourage future works to concentrate on challenging evaluations instead of pursuing deceptively good intra-dataset performance.

show abstract

Deep Convolutional Neural Network-Based Bangla Sign Language Detection on a Novel Dataset

Hasan

Alam

2022

Machine Intelligence and Data Science Applications

View full text Add to dashboard Cite

Recognition Bangla Sign Language using Convolutional Neural Network

Cited by 36 publications

References 16 publications

Design and Development of a Humanoid Robot for Sign Language Interpretation

Design and Development of a Humanoid Robot for Sign Language Interpretation

Generalization of Bangla Sign Language Recognition Using Angular Loss Functions

Deep Convolutional Neural Network-Based Bangla Sign Language Detection on a Novel Dataset

Contact Info

Product

Resources

About