Efficient YOLO Based Deep Learning Model for Arabic Sign Language Recognition

Abstract

Verbal communication is the dominant form of self-expression and interpersonal communication. Speech is a considerable obstacle for individuals with disabilities, including those who are deaf, hard of hearing, mute, or nonverbal. Consequently, these individuals depend on sign language to communicate with others. Sign Language is a complex system of gestures and visual cues that facilitate the inclusion of individuals into vocal communication groups. In this manuscript a novel technique proposed using deep learning to recognize the Arabic Sign language (ArSL) accurately. Through this advanced system, the objective is to help in communication between the hearing and deaf community. The proposed mechanism relies on advanced attention mechanisms, and state-of-art Convolutional Neural Network (CNN) architectures with the robust YOLO object detection model that highly improves the implementation and accuracy of ArSL recognition. In our proposed method, we integrate the self-attention block, channel attention module, spatial attention module, and cross-convolution module into the features processing, and the ArSL recognition accuracy reaches 98.9%. The recognition accuracy of our method is significantly improved with higher detection rate. The presented approach showed significant improvement as compared with the conventional techniques with a precision rate of 0.9. For the mAP@0.5, the mAP score is 0.9909 while for the mAP@0.5:0.95 and the results tops all the state-of-the-art techniques. This shows that the model has the great capability to accurately detect and classify complex multiple ArSL signs. The model provides a unique way of linking people and improving the communication strategy while also promoting the social inclusion of deaf people in the Arabic region.