Chronological pattern indexing: An efficient feature extraction method for hand gesture recognition with Leap Motion

Ameur, Safa; Khalifa, Anouar Ben; Bouhlel, Med Salim

doi:10.1016/j.jvcir.2020.102842

Cited by 23 publications

(6 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [18], the authors propose a new approach to extract features based on chronological pattern indexing. As a classifier, they use KNN based on Hamming distance.…”

Section: Related Workmentioning

confidence: 99%

Hand Gesture Recognition Using Automatic Feature Extraction and Deep Learning Algorithms with Memory

Nogales

Benalcázar

2023

BDCC

View full text Add to dashboard Cite

Gesture recognition is widely used to express emotions or to communicate with other people or machines. Hand gesture recognition is a problem of great interest to researchers because it is a high-dimensional pattern recognition problem. The high dimensionality of the problem is directly related to the performance of machine learning models. The dimensionality problem can be addressed through feature selection and feature extraction. In this sense, the evaluation of a model with manual feature extraction and automatic feature extraction was proposed. The manual feature extraction was performed using the statistical functions of central tendency, while the automatic extraction was performed by means of a CNN and BiLSTM. These features were also evaluated in classifiers such as Softmax, ANN, and SVM. The best-performing model was the combination of BiLSTM and ANN (BiLSTM-ANN), with an accuracy of 99.9912%.

show abstract

“…In [18], the authors propose a new approach to extract features based on chronological pattern indexing. As a classifier, they use KNN based on Hamming distance.…”

Section: Related Workmentioning

confidence: 99%

Hand Gesture Recognition Using Automatic Feature Extraction and Deep Learning Algorithms with Memory

Nogales

Benalcázar

2023

BDCC

View full text Add to dashboard Cite

show abstract

“…The authors used an LMD for hand gesture recognition for the D-M in [6]. They defined a criterion for hand gesture recognition and used different methods such as Gaussian Mixture Model (GMM), Hidden Markov Model (HMM), and Dynamic Time Warping (DTW) for image data processing.…”

Section: System Level Reviewmentioning

confidence: 99%

“…Hardware/Software Summary [2] LMD, Android app D-M to ND-M to D-M, Single SLD [3] PC, Harris algorithm D-M to ND-M to D-M, Single SLD [4] Raspberry Pi D-M to ND-M, Single SLD [5] LMD, LSTM Neural networks D-M to ND-M, Single SLD [6] LMD, (GMM, HMM, DTW) D-M to ND-M, Single SLD [7] PC, NN algorithms D-M to ND-M, Single SLD [8] PC, Arduino, Gloves, Flex sensors Calibration required for the custom-made product, D-M to ND-M, Single SLD [10] Arduino, Gloves, Sensors Calibration required for the custom-made product, D-M to ND-M, Single SLD [12] PC, TensorFlow D-M to ND-M, Single SLD [13] PC, Various ML algorithms D-M to ND-M to D-M, Single SLD [14] PC, TensorFlow, CNN D-M to ND-M, Single SLD [15] Android D-M to ND-M, Multi SLD [16] Android D-M to ND-M, Single SLD [17] PC, CNN D-M to ND-M, Single SLD [18] Arduino, Gloves, Flex sensors Calibration required for the custom-made product, D-M to ND-M, Single SLD [19] PC NN D-M to ND-M, Single SLD [20] Gloves, Flex sensors, Voice recorder, Matlab D-M to ND-M, Single SLD [21] PC, CNN D-M to ND-M, Single SLD [22] CNN D-M to ND-M to D-M, Single SLD [23] Gloves, Sensors, Arduino, KNN D-M to ND-M, Single SLD [24] PC-based, PCA D-M to ND-M, Single SLD [25] Arduino, Gloves, Sensors, KNN Calibration required for the custom-made product, D-M to ND-M, Single SLD [26] PC-based, CNN D-M to ND-M, Single SLD [27] Android application D-M to ND-M, Multi SLD [28] PC, CNN D-M to ND-M, Single SLD [29] Android D-M to ND-M to D-M, Single SLD…”

Section: Referencesmentioning

confidence: 99%

A Machine Learning Based Full Duplex System Supporting Multiple Sign Languages for the Deaf and Mute

et al. 2023

View full text Add to dashboard Cite

This manuscript presents a full duplex communication system for the Deaf and Mute (D-M) based on Machine Learning (ML). These individuals, who generally communicate through sign language, are an integral part of our society, and their contribution is vital. They face communication difficulties mainly because others, who generally do not know sign language, are unable to communicate with them. The work presents a solution to this problem through a system enabling the non-deaf and mute (ND-M) to communicate with the D-M individuals without the need to learn sign language. The system is low-cost, reliable, easy to use, and based on a commercial-off-the-shelf (COTS) Leap Motion Device (LMD). The hand gesture data of D-M individuals is acquired using an LMD device and processed using a Convolutional Neural Network (CNN) algorithm. A supervised ML algorithm completes the processing and converts the hand gesture data into speech. A new dataset for the ML-based algorithm is created and presented in this manuscript. This dataset includes three sign language datasets, i.e., American Sign Language (ASL), Pakistani Sign Language (PSL), and Spanish Sign Language (SSL). The proposed system automatically detects the sign language and converts it into an audio message for the ND-M. Similarities between the three sign languages are also explored, and further research can be carried out in order to help create more datasets, which can be a combination of multiple sign languages. The ND-M can communicate by recording their speech, which is then converted into text and hand gesture images. The system can be upgraded in the future to support more sign language datasets. The system also provides a training mode that can help D-M individuals improve their hand gestures and also understand how accurately the system is detecting these gestures. The proposed system has been validated through a series of experiments resulting in hand gesture detection accuracy exceeding 95%.

show abstract

“…A series of investigations to determine the position for the stop point were further designed, with three candidates for comfortability shown in Figure 7(a) [19] , (b) [39] and (c). The coordinates of these hand positions were captured by the leap motion and set as the reference.…”

Section: Hand Gesturementioning

confidence: 99%

Electric Wheelchair Control Using Wrist Rotation Based on Analysis of Muscle Fatigue

et al. 2022

View full text Add to dashboard Cite

The wheelchair is one of the most common assistive technologies for people with motor impairment, due to its environmentally friendly features in terms of mobility and comfort. However, the operational method for conventional wheelchair is still inconvenience for people with finger problems. Therefore, in the project , a novel control method to operate an electric wheelchair, using hand gestures ( wrist rotation) is developed. Sixty-five (65) participants were involved in this study. Five hand gestures were considered and studied for forward, backward, right, left, and stop maneuvers while considering human ergonomics factor. In this study, the stop maneuver was determined based on the most comfortable hand position to mitigate a fatigue experience, with two gesture classifying methods further investigated. The first method based on threshold has a promising accuracy of 96%, and 91% precision. This method however requires a calibration every time a new user is introduced. The second method, a Naïve Bayes approach, was observed to solve the problem, as it has about 99% of both the accuracy and the precision. The evaluation of this method was then conducted with six participants that operated the wheelchair to follow the trajectories from start to end. The results showed that the participants comfortably controlled the developed wheelchair system to the goal without any collision. Results from experiments indicate that the proposed approach has high accuracy and the potential to solve the problem related to finger dependencies and hand fatigue.

show abstract

Chronological pattern indexing: An efficient feature extraction method for hand gesture recognition with Leap Motion

Cited by 23 publications

References 54 publications

Hand Gesture Recognition Using Automatic Feature Extraction and Deep Learning Algorithms with Memory

Hand Gesture Recognition Using Automatic Feature Extraction and Deep Learning Algorithms with Memory

A Machine Learning Based Full Duplex System Supporting Multiple Sign Languages for the Deaf and Mute

Electric Wheelchair Control Using Wrist Rotation Based on Analysis of Muscle Fatigue

Contact Info

Product

Resources

About