Arabic Sign Language Recognition Using Deep Learning Models

Al-Barham, Muhammad; Sa'Aleek, Ahmad Abu; Alodat, Mohammad; Hamad, Ghada Ismail; Yaman, Musa Al; Elnagar, Ashraf

doi:10.1109/icics55353.2022.9811162

Cited by 3 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Datasets. To evaluate our system, we make a combination of the ArSL dataset [41] and 7,057 Arabic alphabet sign language images. In partnership with Resala Association in Egypt, we combined them with a 5600-image collection from our previous work [50,51].…”

Section: Resultsmentioning

confidence: 99%

“…AASL is gathered from over 200 individuals in various lighting, backdrop, image orientation, size, and resolution conditions. For a high-quality dataset, feld experts supervised, verifed, and fltered the images [41] British sign language (BSL) BSL Corpus contains 249 video talks. Te conversational dataset contains 6,330 signer gesture annotations [42] Brazilian sign language or Língua Brasileira (Libras) LIBRAS-HC-RGBDS has sixty Libra one-hand setups.…”

Section: Referencementioning

confidence: 99%

See 1 more Smart Citation

Integrated Mediapipe with a CNN Model for Arabic Sign Language Recognition

Moustafa,

Mohd Rahim,

Bouallegue

et al. 2023

Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

Deaf and dumb people struggle with communicating on a day-to-day basis. Current advancements in artificial intelligence (AI) have allowed this communication barrier to be removed. A letter recognition system for Arabic sign language (ArSL) has been developed as a result of this effort. The deep convolutional neural network (CNN) structure is used by the ArSL recognition system in order to process depth data and to improve the ability for hearing-impaired to communicate with others. In the proposed model, letters of the hand-sign alphabet and the Arabic alphabet would be recognized and identified automatically based on user input. The proposed model should be able to identify ArSL with a rate of accuracy of 97.1%. In order to test our approach, we carried out a comparative study and discovered that it is able to differentiate between static indications with a higher level of accuracy than prior studies had achieved using the same dataset.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Referencementioning

confidence: 99%

Integrated Mediapipe with a CNN Model for Arabic Sign Language Recognition

Moustafa,

Mohd Rahim,

Bouallegue

et al. 2023

Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

show abstract

“…Moreover, the study in ref. [39] presented a modified ResNet18 model and reported the best accuracy score compared to the previous models, which was 1.87% higher than the existing works using ResNet. A recent study [49] explored continuous ArSL classification using video data and CNN-LSTM-SelfMLP models with MobileNetV2 and ResNet18 backbones for feature extraction.…”

Section: Overview Of Sl Classification Research Based On Arabic Languagementioning

confidence: 87%

“…Many types of SLs have been used in previous studies based on countries and spoken languages. Languages used in SL research are numerous including American [16,22,[41][42][43][44][45][46], Mexican [47], Arabic [3,7,10,11,39,40,48,49], Algerian [50], Pakistani [51], Indian [52][53][54], Bangla [55], Chinese [56,57], Indonesian [58], Italian [23], Peruvian [59], Turkish [34], and Urdu [24]. Based on [60], which analyzed SL classification research studies published from 2014 to 2021, another breakdown of literature works was presented according to the local variation in the SL they referred to.…”

Section: Overview Of Sl Classification Research Based On Languagesmentioning

confidence: 99%

“…Their models attained accuracies of 98.39% for dynamic, 88.89% for static signs in the dependent mode, and 96.69% for dynamic, 86.34% for static in the independent mode: When mixing both dependent and independent modes, the results achieved were 89.62% for signer dependent and 88.09% for signer independent. Additional recent research works presented a model utilizing fine-tuning with deep learning for the specific task of recognizing ArSL [39,40], which initiated the research direction of using transfer learning for ArSL and also created the datasets used in this study. CNN models reducing the size of the dataset required for training and at the same time, reaching a higher accuracy, were investigated [40].…”

Section: Overview Of Sl Classification Research Based On Arabic Languagementioning

confidence: 99%

See 1 more Smart Citation

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

Alharthi,

Alzahrani

2023

Applied Sciences

View full text Add to dashboard Cite

Sign languages are complex, but there are ongoing research efforts in engineering and data science to recognize, understand, and utilize them in real-time applications. Arabic sign language recognition (ArSL) has been examined and applied using various traditional and intelligent methods. However, there have been limited attempts to enhance this process by utilizing pretrained models and large-sized vision transformers designed for image classification tasks. This study aimed to create robust transfer learning models trained on a dataset of 54,049 images depicting 32 alphabets from an ArSL dataset. The goal was to accurately classify these images into their corresponding Arabic alphabets. This study included two methodological parts. The first one was the transfer learning approach, wherein we utilized various pretrained models namely MobileNet, Xception, Inception, InceptionResNet, DenseNet, and BiT, and two vision transformers namely ViT, and Swin. We evaluated different variants from base-sized to large-sized pretrained models and vision transformers with weights initialized from the ImageNet dataset or otherwise randomly. The second part was the deep learning approach using convolutional neural networks (CNNs), wherein several CNN architectures were trained from scratch to be compared with the transfer learning approach. The proposed methods were evaluated using the accuracy, AUC, precision, recall, F1 and loss metrics. The transfer learning approach consistently performed well on the ArSL dataset and outperformed other CNN models. ResNet and InceptionResNet obtained a comparably high performance of 98%. By combining the concepts of transformer-based architecture and pretraining, ViT and Swin leveraged the strengths of both architectures and reduced the number of parameters required for training, making them more efficient and stable than other models and existing studies for ArSL classification. This demonstrates the effectiveness and robustness of using transfer learning with vision transformers for sign language recognition for other low-resourced languages.

show abstract

IWBC and LFD for Static and Dynamic Hand Gesture Recognition

Sebbah,

Chelali

2024

2024 8th International Conference on Image and Signal Processing and Their Applications (ISPA)

View full text Add to dashboard Cite

Arabic Sign Language Recognition Using Deep Learning Models

Cited by 3 publications

References 15 publications

Integrated Mediapipe with a CNN Model for Arabic Sign Language Recognition

Integrated Mediapipe with a CNN Model for Arabic Sign Language Recognition

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

IWBC and LFD for Static and Dynamic Hand Gesture Recognition

Contact Info

Product

Resources

About