Marathi Sign Language Recognition using MediaPipe and Deep Learning Algorithm

Abstract

Sign language is the hand gesture-based manual way of communication for mute and deaf people. The majorityof other people do not know this sign language, so it creates isolation among physically disabled people. The majority of research is already done for sign language recognition using machine and deep learning techniques for the English language. The attention needs to focus on regional sign language recognition as some of the signs vary according to region. We have considered the regional language, i.e., Marathi, for our recognition work. In this paper, we proposed the real-time method for hand gesture detection for sign language recognition using the Media Pipe along with a long short-term memory (LSTM) neural network model for the recognition of Marathi sign language. The system is built for the automatic recognition of Marathi sign language. The implemented model is trained and tested over our own dataset designed for 15 different Marathi words that are used in day-to-day communication by physically disabled people. The dataset contains a total of 37500 frames. With 97.50% accuracy, we can successfully recognize Marathi sign language.