Indonesian sign language recognition using kinect and dynamic time warping

Abstract

Sign Language Recognition System (SLRS) is a system to recognise sign language and then translate them into text. This system can be developed by using a sensor-based technique. Some studies have implemented various feature extraction and classification methods to recognise sign language in the different country. However, their systems were user dependent (the accuracy was high when the trained and the tested user were the same people, but it was getting worse when the tested user was different to the trained user). Therefore in this study, we proposed a feature extraction method which is invariant to a user. We used the distance between two users’ skeleton instead of using the users’ skeleton positions because the skeleton distance is independent to the user posture. Finally, forty-five features were extracted in this proposed method. Further, we classified the features by using a classification method that is suitable with sign language gestures characteristic (time-dependent sequence data). The classification method is Dynamic Time Wrapping. For the experiment, we used twenty Indonesian sign languages from different semantic groups (greetings, questions, pronouns, places, family and others) and different gesture characteristic (static gesture and dynamic gesture). Then the system was tested by a different user with the user who did the training. The result was promising, this proposed method produced high accuracy, reach 91% which shows that this proposed method is user independent.

Development of arabic sign language dictionary using 3D avatar technologies

“…Figure 8 shows a sample of images of the unified ArSL dictionary that used for signs generation where Table 2 shows an example of generated signs. Table 3 shows the comparing of the results of this research with others for ArSL [10,3].…”

“…In SL, the sign is specific hand shapes and movements that are associated with facial expressions to express the meaning. Also, SL is not universal, where every country has its sign language, which differs in syntax and grammar from one country to another [3,4]. SL, which used in America, is identified as American Sign Language (ASL), but the used SL in India is identified as Indian Sign Language (ISL) [5] while in Arab countries identified as Arabic Sing Language (ArSL) [6].…”

Bengali sign language translator with location tracking system

“…A computer vision-based approach is proposed to identify the Bengali numeric sign language [5]. Shaheen and Mehmood [6] explains the importance of a vision-based SLR system dynamic time wrapping based system is proposed to recognize Indonesian sign language which is user independent [7]. Sign language translator devices are usually built using three approaches, which are glove-based, vision-based, and hybrid-based [8].…”

A new hand gestures recognition system

“…Three approaches were used to obtain important information for any hand gesture recognition system, Data gloves approaches, Vision based approaches, and Color-Marker approaches as shown strictly in In Vision based approaches [5,6], the human motion obtained from one camera or more [7,8], and the devices based on vision can handle many properties for interpreting the gesture, for example, color and texture whereas the sensor has not this property. Although these approaches are simple, many challenges can appear for example lighting diversity, complex background, and presence objects with skin-color similar to the hand (clutter), as well as the system, requires some criteria like time of recognition, speed, durability, and efficiency of computation [9][10][11].…”