Sign language translation system based on micro-inertial measurement units and ZigBee network

Abstract

Chinese sign language has been proved as an effective communication tool for deaf people. In this paper, we present a novel translation system, which can capture human gestures through micro-inertial measurement units (IMUs) and translate the gestures into specific meanings accordingly. Each micro-IMU consists of a 3D accelerometer and gyroscope. A micro-controller and ZigBee network were used to acquire data simultaneously and wirelessly. Ten types of basic Chinese sign language movements including what, how, work, today, happy, please, book, body, clothes and and were collected and stored to form a motion-sensing database. A discrete cosine transform (DCT) was performed to extract the effective features from the original data, while a hidden Markov model (HMM) was used to train the database in order to form an HMM classifier. Testing samples were used to test the HMM classifier. Different sign languages were recognized through the HMM classifier and subsequent translation processes were performed. Experimental results showed that the correct recognition rate ranges from 95% to 100% for the 10 sign language movements, and the overall correction rate is 98%. With more micro-electro-mechanical system (MEMS) sensing motes adding to the interpretation system, the performance will be enhanced.