Smart Home Automation-Based Hand Gesture Recognition Using Feature Fusion and Recurrent Neural Network

Abstract

Gestures have been used for nonverbal communication for a long time, but human–computer interaction (HCI) via gestures is becoming more common in the modern era. To obtain a greater recognition rate, the traditional interface comprises various devices, such as gloves, physical controllers, and markers. This study provides a new markerless technique for obtaining gestures without the need for any barriers or pricey hardware. In this paper, dynamic gestures are first converted into frames. The noise is removed, and intensity is adjusted for feature extraction. The hand gesture is first detected through the images, and the skeleton is computed through mathematical computations. From the skeleton, the features are extracted; these features include joint color cloud, neural gas, and directional active model. After that, the features are optimized, and a selective feature set is passed through the classifier recurrent neural network (RNN) to obtain the classification results with higher accuracy. The proposed model is experimentally assessed and trained over three datasets: HaGRI, Egogesture, and Jester. The experimental results for the three datasets provided improved results based on classification, and the proposed system achieved an accuracy of 92.57% over HaGRI, 91.86% over Egogesture, and 91.57% over the Jester dataset, respectively. Also, to check the model liability, the proposed method was tested on the WLASL dataset, attaining 90.43% accuracy. This paper also includes a comparison with other-state-of-the art methods to compare our model with the standard methods of recognition. Our model presented a higher accuracy rate with a markerless approach to save money and time for classifying the gestures for better interaction.

Cognitive Classifier of Hand Gesture Images for Automated Sign Language Recognition: Soft Robot Assistance Based on Neutrosophic Markov Chain Paradigm

“…A lightweight deep neural network with enhanced processing was proposed for realtime dynamic sign language recognition (DSLR) in ref. [33]. There are three primary parts of the solution approach.…”

End-to-End Ultrasonic Hand Gesture Recognition

“…Additionally, these technologies should be lean regarding cost and power consumption. There are HGR systems based on other sensors such as vision [5][6][7][8][9][10][11], radar [12][13][14], and WiFi [15][16][17][18][19][20][21][22]. Ultrasound-based HGR systems such as those surveyed in [12] or presented in [23] require the user to wear a device to record the hand gestures.…”

TQU-HG dataset and comparative study for hand gesture recognition of RGB-based images using deep learning

“…Hand gesture recognition has great applications in human-computer interaction (HCI) [1], home device control [2], human-robot interaction (HRI) [3] building sign language understanding systems for the hearing impaired [4]. Previously, to solve this problem, traditional machine learning (TML) models are often used (support vector machine (SVM) [5], AdaBoost [6], and random forest (RF) [7]).…”