Korean sign language recognition using EMG and IMU sensors based on group-dependent NN models

Shin, Seongjoo; Baek, Youngmi; Lee, Jin‐Hee; Eun, Yongsoon; Son, Sang H.

doi:10.1109/ssci.2017.8280908

Cited by 20 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, five channels were placed on the forearm of three subjects and data from the accelerometer were used for classification. Moreover, 30 gestures in Korean Sign Language (KSL) were classified for six subjects using the commercial Myo TM armband [26]. The raw signals from the inertial sensors were inserted into a convolutional neural network (CNN), achieving a 98% accuracy level.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, which classifier (or classifiers) could be more suitable for an sEMG armband device for Libras recognition? These questions arise in the process of SLR recognition, and in some related works these questions are not fully answered [11,23,[25][26][27]. Thus, a methodology is presented in this paper to evaluate how the sEMG processing parameters influence classification systems for Brazilian Sign Language.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

Mendes

Freitas

Campos

et al. 2020

Sensors

View full text Add to dashboard Cite

Sign Language recognition systems aid communication among deaf people, hearing impaired people, and speakers. One of the types of signals that has seen increased studies and that can be used as input for these systems is surface electromyography (sEMG). This work presents the recognition of a set of alphabet gestures from Brazilian Sign Language (Libras) using sEMG acquired from an armband. Only sEMG signals were used as input. Signals from 12 subjects were acquired using a MyoTM armband for the 26 signs of the Libras alphabet. Additionally, as the sEMG has several signal processing parameters, the influence of segmentation, feature extraction, and classification was considered at each step of the pattern recognition. In segmentation, window length and the presence of four levels of overlap rates were analyzed, as well as the contribution of each feature, the literature feature sets, and new feature sets proposed for different classifiers. We found that the overlap rate had a high influence on this task. Accuracies in the order of 99% were achieved for the following factors: segments of 1.75 s with a 12.5% overlap rate; the proposed set of four features; and random forest (RF) classifiers.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

Mendes

Freitas

Campos

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The Myo armband is a commercial product that collects sEMG and IMU data from the user. Shin et al [24] proposed an automatic Korean sign language recognition system based on sEMG and IMU sensors and group-dependent neural network models. Du et al [13] proposed a novel semi-supervised CNN architecture that used dataglove data in the training phase to capture more discriminative features of sEMG signals.…”

Section: Related Workmentioning

confidence: 99%

sEMG-Based Gesture Recognition With Embedded Virtual Hand Poses and Adversarial Learning

Wong

Dai

et al. 2019

IEEE Access

View full text Add to dashboard Cite

To improve the accuracy of surface electromyography (sEMG)-based gesture recognition, we present a novel hybrid approach that combines real sEMG signals with corresponding virtual hand poses. The virtual hand poses are generated by means of a proposed cross-modal association model constructed based on the adversarial learning to capture the intrinsic relationship between the sEMG signals and the hand poses. We report comprehensive evaluations of the proposed approach for both frame-and window-based sEMG gesture recognitions on seven-sparse-multichannel and four-high-density-benchmark databases. The experimental results show that the proposed approach achieves significant improvements in sEMG-based gesture recognition compared to existing works. For frame-based sEMG gesture recognition, the recognition accuracy of the proposed framework is increased by an average of +5.2% on the sparse multichannel sEMG databases and by an average of +6.7% on the high-density sEMG databases compared to the existing methods. For window-based sEMG gesture recognition, the state-of-the-art recognition accuracies on three of the high-density sEMG databases are already higher than 99%, i.e., almost saturated; nevertheless, we achieve a +0.2% improvement. For the remaining eight sEMG databases, the average improvement with the proposed framework for the window-based approach is +2.5%. INDEX TERMSHand gesture recognition, surface electromyography (sEMG), myoelectric control, generative adversarial learning, virtual hand pose.

show abstract

“…Lionel and other researchers used convolutional neural networks (CNNs) on the EMG data of the forearm to recognize 20 Italian gestures [ 23 ]. Seongjoo and others used sensor fusion technology and group-dependent neural network models to recognize Korean sign language [ 24 ]. After the collection of EMG data, measures are applied to process raw EMG data.…”

Section: Introductionmentioning

confidence: 99%

Development of Sign Language Motion Recognition System for Hearing-Impaired People Using Electromyography Signal

Tateno

Liu

2020

Sensors

View full text Add to dashboard Cite

Sign languages are developed around the world for hearing-impaired people to communicate with others who understand them. Different grammar and alphabets limit the usage of sign languages between different sign language users. Furthermore, training is required for hearing-intact people to communicate with them. Therefore, in this paper, a real-time motion recognition system based on an electromyography signal is proposed for recognizing actual American Sign Language (ASL) hand motions for helping hearing-impaired people communicate with others and training normal people to understand the sign languages. A bilinear model is applied to deal with the on electromyography (EMG) data for decreasing the individual difference among different people. A long short-term memory neural network is used in this paper as the classifier. Twenty sign language motions in the ASL library are selected for recognition in order to increase the practicability of the system. The results indicate that this system can recognize these twenty motions with high accuracy among twenty participants. Therefore, this system has the potential to be widely applied to help hearing-impaired people for daily communication and normal people to understand the sign languages.

show abstract

Korean sign language recognition using EMG and IMU sensors based on group-dependent NN models

Cited by 20 publications

References 21 publications

Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

sEMG-Based Gesture Recognition With Embedded Virtual Hand Poses and Adversarial Learning

Development of Sign Language Motion Recognition System for Hearing-Impaired People Using Electromyography Signal

Contact Info

Product

Resources

About