A novel hand gesture detection and recognition system based on ensemble-based convolutional neural network

Sen, Abir; Mishra, Tapas Kumar; Dash, Ratnakar

doi:10.1007/s11042-022-11909-0

Cited by 10 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sen, Abir, Tapas Kumar Mishra [16] in this research study covered the ensemble-based CNN-based gesture detection and recognition system.Before feeding the gesture image into CNN classifiers for parallel training, it processes via a unique set of several steps in order to segment the hand region.The suggested method has been validated using two publicly accessible datasets (Datasets-1 and 2) as well as one self-constructed dataset. Accuracy results of 99.80%, 96.50%, and 99.76% using different datasets with methods are discussed in this work.…”

Section: Gesturementioning

confidence: 99%

OBT-Trace: An approach to trace and recognition object motion through prosthetic hand gestures

Surya.S

2024

jes

View full text Add to dashboard Cite

The development of prosthetic hands has advanced significantly in recent years, aiming to provide more intuitive and functional solutions for individuals with upper limb amputations. This research presents a novel approach to prosthetic hand control by integrating 3D hand gesture recognition with object manipulation and recognition capabilities. Our proposed system utilizes a pre-trained object recognition model, based on transfer learning, to enable the prosthetic hand to perceive and identify objects in its vicinity. The model leverages a vast dataset of objects, enabling the prosthetic hand to recognize a wide array of everyday items, thus enhancing its versatility.In addition, the prosthetic hand incorporates a sophisticated 3D hand gesture recognition system, allowing users to control the hand's movements and actions seamlessly. By recognizing specific gestures, such as grasping, lifting, and releasing, users can intuitively interact with their environment and perform various tasks with ease. This research leverages the synergy between gesture recognition and object recognition, creating a powerful framework for prosthetic hand control. The system's adaptability and versatility make it suitable for a broad range of applications, from assisting with daily tasks to enhancing the quality of life for individuals with upper limb amputations.The results of this study demonstrate the feasibility and effectiveness of combining 3D hand gesture recognition with pre-trained object recognition through transfer learning. This approach opens up new possibilities for enhancing prosthetic hand functionality and usability, ultimately improving the lives of those who rely on these devices for daily living. The proposed model combines the features of YOLO V7 object detection with pre-trained models. The proposed model achieves 99.8% of accuracy compared to the existing models.

show abstract

Section: Gesturementioning

confidence: 99%

OBT-Trace: An approach to trace and recognition object motion through prosthetic hand gestures

Surya.S

2024

jes

View full text Add to dashboard Cite

show abstract

“…Currently, this DL algorithm called CNN, has become a very popular choice for classification tasks. However, they also suffer from some problems, such as overfitting problem [23]- [25], high variance during prediction and prediction. To overcome these problems, in [23] the part of the gesture is detected using the binary threshold-based background separation method, the contour part is extracted, the hand region is segmented, and finally the images are resized for CNN training.…”

Section: State Of Artmentioning

confidence: 99%

“…However, they also suffer from some problems, such as overfitting problem [23]- [25], high variance during prediction and prediction. To overcome these problems, in [23] the part of the gesture is detected using the binary threshold-based background separation method, the contour part is extracted, the hand region is segmented, and finally the images are resized for CNN training. On the other hand, in [25], VGG-16 was used over VGG-19 to improve feature extraction and decrease overfitting.…”

Section: State Of Artmentioning

confidence: 99%

Peruvian Sign Language translator for people with hearing and/or communication disabilities using a convolutional neural network

Briones-Cerquín,

Tumay Guevara,

Paiva Peredo

et al. 2023

Proceedings of the 21th LACCEI International Multi-Conference for Engineering, Education and Technology (LACCEI 2023): “Leaders

View full text Add to dashboard Cite

Promoting the social inclusion of people with deafness and/or communication disabilities is a priority for many countries. For this reason, considerable emphasis is currently being given to machine learning (ML) and deep learning (DL) techniques for sign language recognition and translation, as they can be a significant contribution to the social inclusion of the deaf and/or deafened community. Thus, the present research proposes the development of a translator of Peruvian Sign Language (PSL) for the recognition and translation of static signs belonging to PSL through a convolutional neural network (CNN). These signs are the numbers from 0 to 9 and the letters of the alphabet except J, Ñ and Z because they are represented with movable signs and the letters O and W because they are very similar to the numbers "0" and "6". To achieve the development of the translator, a balanced database for PSL was built from scratch, consisting of 700 images for each static sign, for a total of 22400 images. These images have a dimension of 80x80 pixels that go through a preprocessing stage, 3 convolutional layers, filters, kernels, ReLu and MaxPooling activation functions. Experimental results show that the translator recognizes static PSL signs with an accuracy of 90%, 86% and 81% for training, validation and testing respectively.

show abstract

“…Here CNN was employed for classifying the static gesture images. Recently, Sen et al [13] developed a hand gesture recognition system, which had three stages; (1) gesture detection by binary thresholding, (2) segmentation of gesture portion, (3) training of three custom CNN classifiers in parallel, followed by calculating the output scores of these models to build the ensemble model for final prediction.…”

Section: Literature Surveymentioning

confidence: 99%

“…They have achieved the classification accuracy of 94.84% and 99.96%. Sen et al [11] have devised a novel system for recognizing hand gestures. The system consisted of three stages: (1) detecting gestures using binary thresholding, (2) segmenting the gesture portion, and (3) training three custom CNN classifiers simultaneously.…”

Section: Literature Surveymentioning

confidence: 99%

Design of Human Machine Interface through vision-based low-cost Hand Gesture Recognition system based on deep CNN

Sen

Mishra

Dash

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this work, a real-time hand gesture recognition system-based human-computer interface (HCI) is presented. The system consists of six stages: (1) hand detection, (2) gesture segmentation, (3) use of six pre-trained CNN models by using the transfer-learning method, (4) building an interactive human-machine interface, (5) development of a gesture-controlled virtual mouse, (6) use of Kalman filter to estimate the hand position, based on that the smoothness of the motion of pointer is improved. Six pre-trained convolutional neural network (CNN) models such as VGG16, VGG19, ResNet50, ResNet101, Inception-V1, and MobileNet-V1 have been used to classify hand gesture images. Three multi-class datasets (two publicly and one custom) have been used to validate the models. Considering the models' performances, it is observed that Inception-V1 has significantly shown a better classification performance compared to the other five pre-trained models in terms of accuracy, precision, recall, and F-score values. The gesture recognition system is expanded and used to control some desktop applications (such as VLC player, audio player, file management, playing 2D Super-Mario-Bros game, etc.) with different customized gesture commands in real-time scenarios. The average speed of this system has reached 35 fps (frame per second), which meets the requirements for the real-time scenario. We have also shown the performance analysis of gesture-controlled applications and obtained the average response time (ms) for each control of every application in the real-time scenario. This model will be beneficial for physically disabled people interacting with the desktops.

show abstract

A novel hand gesture detection and recognition system based on ensemble-based convolutional neural network

Cited by 10 publications

References 17 publications

OBT-Trace: An approach to trace and recognition object motion through prosthetic hand gestures

OBT-Trace: An approach to trace and recognition object motion through prosthetic hand gestures

Peruvian Sign Language translator for people with hearing and/or communication disabilities using a convolutional neural network

Design of Human Machine Interface through vision-based low-cost Hand Gesture Recognition system based on deep CNN

Contact Info

Product

Resources

About