Low-cost intelligent gesture recognition engine for audio-vocally impaired individuals

Abstract

This paper presents a low-cost open source prototype of a gesture recognition and interpretation glove that aids audio-vocally impaired individuals in communication. A sensor based glove for recognition of the gestures in the American Sign Language (ASL) is the input module to the system. It incorporates resistive flex sensors and contact sensors appropriately positioned to obtain a theoretical efficiency of 96% in character recognition. When the sensors are subjected to change in orientation by the user, the variation in electrical resistance of each of the individual sensors is obtained as the input to an Arduino ATMega328 microcontroller that accurately maps it to the letters of the English alphabet through an efficient algorithm. An elegant methodology that is easy to implement using a microcontroller is employed for the mapping process to minimize the complexity of the system. The paper also presents a detailed study of the efficiency characteristics involved in the construction of the custom-designed flex sensors utilized in the system. The paper outlines the calibration and testing results of the sensors .The highlights of the system are its ergonomic and economical design, portability and intelligent gesture recognition and interpretation ability. The implemented system achieves a practical efficiency of up to 80% of theoretical efficiency. The cost of the system is approximately less than USD 5 at laboratory condition using off the shelf components.

A Review on Systems-Based Sensory Gloves for Sign Language Recognition State of the Art between 2007 and 2017

“…The system accuracy was from 83.1% to 94.5% with a cost of about $30. The ASL glove-based gesture recognition system is presented in [ 60 ]. The variation in electrical resistance of five custom-designed flex sensors was processed as the input to an Arduino ATMega328 microcontroller; then, it was compared with stored values corresponding to gestures.…”

“…Customarily, a user interacts with the device through output devices, so the output devices play an important role in achieving the best performance of devices implemented in the field of SLR. The main device adopted by researchers as the output was the screen of computer [ 3 , 4 , 35 , 40 , 48 , 60 , 61 , 62 ]. Other devices attracting the attention of researchers are the liquid-crystal display (LCD) [ 12 , 35 , 57 , 63 ], speaker [ 38 , 46 , 54 , 64 ], or both [ 6 , 51 , 55 , 65 , 66 , 67 , 68 ].…”

Wearable sensors for gesture analysis in smart healthcare applications

“…Other examples of glove-based systems used for sign language include clothsupported bend sensors based on the Hall Effect mounted together with an accelerometer [24] as well as flex and contact sensors [64]. Carbonaro et al [21] added textile electrodes and an inertial motion unit to the deformation sensors of the glove-based device to add emotion recognition made through electrodermal activity.…”

Development of a Wearable Sensor Glove for Real-Time Sign Language Translation

“…A wearable glove attached with flex sensors and a contact sensor is also a favoured combination adopted by researchers. Rishikanth et al proposed a gesture recognition glove that employs flexible sensors and a contact sensor [17]. The flexible sensors were fitted not only on the fingers but also on the wrist to track wrist motions.…”