Integrated Mediapipe with a CNN Model for Arabic Sign Language Recognition

Abstract

Deaf and dumb people struggle with communicating on a day-to-day basis. Current advancements in artificial intelligence (AI) have allowed this communication barrier to be removed. A letter recognition system for Arabic sign language (ArSL) has been developed as a result of this effort. The deep convolutional neural network (CNN) structure is used by the ArSL recognition system in order to process depth data and to improve the ability for hearing-impaired to communicate with others. In the proposed model, letters of the hand-sign alphabet and the Arabic alphabet would be recognized and identified automatically based on user input. The proposed model should be able to identify ArSL with a rate of accuracy of 97.1%. In order to test our approach, we carried out a comparative study and discovered that it is able to differentiate between static indications with a higher level of accuracy than prior studies had achieved using the same dataset.

Revolutionizing Accessibility: Smart Wheelchair Robot and Mobile Application for Mobility, Assistance, and Home Management

“…Mannan et al (2022) [16] achieve an exceptional 97% accuracy in Chinese sign language recognition through hyperparameter optimization, yet a deeper dive into the computational demands and training resources required is necessary for a comprehensive assessment of practicality. AL Moustafa et al (2023) [17] present an integrated approach combining Mediapipe with a CNN model for Arabic Sign Language (ArSL) recognition, offering promise in its combination of hand landmarks and CNN-based classification. However, a more thorough examination of system limitations and scalability for diverse sign languages would enhance the paper's broader relevance.…”