Sign language translator and gesture recognition

Abstract

The mute/deaf individuals have a communication problem dealing with other people. It is hard for such individuals to express what they want to say since sign language is not understandable by everyone. This paper is to develop a Data Acquisition and Control (DAC) system that translates the sign language into text that can be read by anyone. This system is called Sign Language Translator and Gesture Recognition. We developed a smart glove that captures the gesture of the hand and interprets these gestures into readable text. This text can be sent wirelessly to a smart phone or shown in an embedded LCD display. It is evident from the experimental results that gestures can be captured by set of inexpensive sensors, which measure the positions and the orientation of the fingers. The current version of the system is able to interpret 20 out of 26 letters with a recognition accuracy of 96%.

A Review on Systems-Based Sensory Gloves for Sign Language Recognition State of the Art between 2007 and 2017

“…Twenty out of the 23 letters achieved 100% recognition, and the average system accuracy reached 92%. An intelligent glove was developed to capture the postures of the hand and translate these postures into simple text depending on the data acquisition and control system [ 50 ]. Three types of sensors were used in this data glove prototype: five flex sensors, five force sensors, one mounted on a finger, six DoF MPU6050 chip, tri axis ACC, and tri axis gyroscope sensors.…”

“…Therefore, data from various channels needs to be fused into the system [ 77 ]. A combination of different types of abduction sensors such as flex sensors, contact sensors, a gyroscope, and an ACC can capture more complicated gestures, as well as differentiate between similar letters to obtain a reliable, robust, and accurate recognition system [ 3 , 9 , 17 , 39 , 49 , 50 ]. However, using lightweight and fewer sensors helps to simplify the hardware complexity [ 8 ].…”

“…The microcontroller is the system’s mind that is accountable for gathering the data from the sensors provided by the glove and performing the required processing of these data to recognize and transfer the sign to the output port to be presented in the final stage. A high-performance microcontroller with a microchip with 8-bit AVR microcontrollers based on reduced instruction set computer (RISC), which combines 32KB in-system programming (ISP) flash memory with read–write capabilities called ATmega ( Figure 13 a), was used in [ 8 , 9 , 35 , 50 ]. A micro-controller MSP430G2553 (TEXAS INSTRUMENTS, Dallas, TX, USA), ( Figure 13 b) with an 8-channel, 10-bit (analogue to digital converter) ADC was used [ 37 ].…”

“…Despite the fact that SL has plenty of postures, many studies have concentrated on a selective set of postures that may be as small as some letters of the alphabet [ 21 , 44 , 47 , 51 , 78 ]; or the postures of the most frequently used words [ 37 , 41 , 45 , 55 , 56 , 72 , 77 , 79 ]; or a combination of the alphabet, numbers, and words [ 6 ] to perform their experiments and develop an SLR system. Others have contributed to the expansion of the database of gestures allocated for the purpose of developing a system to distinguish them and make them include either whole alphabets [ 4 , 11 , 35 , 46 , 50 , 57 , 64 , 65 , 70 ], numbers within the range 0 to 9 [ 10 ], or both, in the same system [ 39 , 42 , 69 ]. Furthermore, others contributed to their endeavor to distinguish certain words and phrases, even sentences, chosen to cover a wide range of real-life circumstances, such as family, shopping, education, sports, etc.…”

Application for Iraqi sign language translation on Android system

“…The system is called gesture recognition and sign language translator and it utilizes a smart glove that captures the gesture of the hand and interprets these gestures into readable text. This text can be sent wirelessly to a smart phone or shown in an embedded LCD display [8].…”