Gesture To Speech Conversion using Flex sensors, MPU6050 and Python

Abstract

Communicating through hand gestures is one of the most common forms of non-verbal and visual communication adopted by speech impaired population all around the world. The problem existing at the moment is that most of the people are not able to comprehend hand gestures or convert them to the spoken language quick enough for the listener to understand. A large fraction of India’s population is speech impaired. In addition to this communication to sign language is not a very easy task. This problem demands a better solution which can assist speech impaired population conversation without any difficulties. As a result, reducing the communication gap for the speech impaired. This paper proposes an idea which will assist in removing or at least reducing this gap between the speech impaired and normal people. The research going on this area mostly focuses on image processing approaches. However, a cheaper and user-friendly approach has been used in this paper. The idea is to make a glove that can be worn by the speech impaired people which will further be used to convert the sign language into speech and text. Our prototype involves Arduino Uno as a microcontroller which is interfaced with flex sensors and accelerometer, gyroscopic sensor for reading the hand gestures. Furthermore, to perform better execution, we have incorporated an algorithm for better interpretation of data and therefore producing more accurate result. Thereafter, we use python to interface Arduino Uno with a microprocessor and finally converting into speech. The prototype has been calibrated in accordance with the ASL (American Sign Language).

Development of a Wearable Sensor Glove for Real-Time Sign Language Translation

“…The work obtained an exceptional accuracy of 93.4% for 75 static gestures. Similarly, Mehra et al had utilised flexible sensors and IMUs for an American Sign Language interpreter device, where the translation output was displayed on a computer monitor [15]. Another prototype of a wearable sign language interpreter, which was developed by Chong and Kim by using only six (6) IMUs mounted on the back of the hand and fingertips to detect and translate the hand motions [16].…”