Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network

Sahoo, Jaya Prakash; Prakash, Allam Jaya; Pławiak, Paweł; Samantray, Saunak

doi:10.3390/s22030706

Cited by 76 publications

(47 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study focuses on a literature review of hand gesture strategies and discusses their pros and limits in various situations. In addition, the performance of these methods is tabulated, with an emphasis on computer vision techniques that deal with similarity and difference points; hand segmentation techniques; classification algorithms and limitations; number and types of gestures; dataset used; detection range (distance); and camera type [ 51 ]. Convolutional neural networks (CNN) are used to categorize images of hand gestures.…”

Section: Significant Research Work On Hand Gesture Recognitionmentioning

confidence: 99%

A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System

et al. 2022

View full text Add to dashboard Cite

Researchers have recently focused their attention on vision-based hand gesture recognition. However, due to several constraints, achieving an effective vision-driven hand gesture recognition system in real time has remained a challenge. This paper aims to uncover the limitations faced in image acquisition through the use of cameras, image segmentation and tracking, feature extraction, and gesture classification stages of vision-driven hand gesture recognition in various camera orientations. This paper looked at research on vision-based hand gesture recognition systems from 2012 to 2022. Its goal is to find areas that are getting better and those that need more work. We used specific keywords to find 108 articles in well-known online databases. In this article, we put together a collection of the most notable research works related to gesture recognition. We suggest different categories for gesture recognition-related research with subcategories to create a valuable resource in this domain. We summarize and analyze the methodologies in tabular form. After comparing similar types of methodologies in the gesture recognition field, we have drawn conclusions based on our findings. Our research also looked at how well the vision-based system recognized hand gestures in terms of recognition accuracy. There is a wide variation in identification accuracy, from 68% to 97%, with the average being 86.6 percent. The limitations considered comprise multiple text and interpretations of gestures and complex non-rigid hand characteristics. In comparison to current research, this paper is unique in that it discusses all types of gesture recognition techniques.

show abstract

Section: Significant Research Work On Hand Gesture Recognitionmentioning

confidence: 99%

A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The vision-based group consists of 2D and 3D approaches, as shown in Table 1. The 2D method [21][22][23][24][25][26][27][28][29][30][31][32][33] uses video and image data for sign language interpretation. For example, a frequency-based model [21,22] has been proposed which uses a histogram, discrete wavelet transform, and a Gabor filter for extracting features from 2D images.…”

Section: Related Workmentioning

confidence: 99%

“…Subsequently, the motion-based model [25] was proposed, which uses the motion of the hand as a feature via hand trajectory tracking. Finally, the deep-based model [26][27][28][29][30][31][32][33][34][35] has been proposed, which uses a novel deep-learning-based architecture to learn spatial hierarchies of features automatically and adaptively for 2D images, such as the convolutional neural network model (CNN), the convolutional neural network-long short-term memory model (CNN-LSTM), and recurrent neural networks (RNN). However, the 2D approach has less information than the 3D approach, which makes it hard to solve the problem of similar gestures based on a backhand view.…”

Section: Related Workmentioning

confidence: 99%

Backhand-Approach-Based American Sign Language Words Recognition Using Spatial-Temporal Body Parts and Hand Relationship Patterns

Chophuk

Chamnongthai

Chinnasarn

2022

Sensors

View full text Add to dashboard Cite

Most of the existing methods focus mainly on the extraction of shape-based, rotation-based, and motion-based features, usually neglecting the relationship between hands and body parts, which can provide significant information to address the problem of similar sign words based on the backhand approach. Therefore, this paper proposes four feature-based models. The spatial–temporal body parts and hand relationship patterns are the main feature. The second model consists of the spatial–temporal finger joint angle patterns. The third model consists of the spatial–temporal 3D hand motion trajectory patterns. The fourth model consists of the spatial–temporal double-hand relationship patterns. Then, a two-layer bidirectional long short-term memory method is used to deal with time-independent data as a classifier. The performance of the method was evaluated and compared with the existing works using 26 ASL letters, with an accuracy and F1-score of 97.34% and 97.36%, respectively. The method was further evaluated using 40 double-hand ASL words and achieved an accuracy and F1-score of 98.52% and 98.54%, respectively. The results demonstrated that the proposed method outperformed the existing works under consideration. However, in the analysis of 72 new ASL words, including single- and double-hand words from 10 participants, the accuracy and F1-score were approximately 96.99% and 97.00%, respectively.

show abstract

“…There are various ways to calculate positional encodings, which can be obtained by learning or being fixed directly [39]. In this work, we use fixed positional encoding based on sine and cosine functions of different frequencies as shown in Equation ( 5) and ( 6): PE(pos, 2i) = sin pos 10, 000 2i/d ( 5) PE(pos, 2i + 1) = cos pos 10, 000 2i/d (6) where PE() denotes the function of positional encodings, pos denotes the position, i represents the dimension in the positional encoding vector, and d is a base parameter of transformer that denotes the size of the hidden layer at each position. We do this via positional encoding.…”

Section: Attention-based Encodermentioning

confidence: 99%

“…Compared to other physical input devices, hand gestures provide a more natural and convenient way for humans to interact with devices. Computer vision-based gesture recognition is a form of technology that uses a computing module to read and interpret hand movements as commands [1][2][3][4][5][6]. Computer visionbased gesture recognition technology can be used in several industries, such as interactive entertainment, smart home, VR/AR, and sign language machine translation [7][8][9][10][11][12][13].…”

Section: Introductionmentioning

confidence: 99%

Content-Adaptive and Attention-Based Network for Hand Gesture Recognition

Cao

Shin

2022

Applied Sciences

View full text Add to dashboard Cite

For hand gesture recognition, recurrent neural networks and 3D convolutional neural networks are the most commonly used methods for learning the spatial–temporal features of gestures. The calculation of the hidden state of the recurrent neural network at a specific time is determined by both input at the current time and the output of the hidden state at the previous time, therefore limiting its parallel computation. The large number of weight parameters that need to be optimized leads to high computational costs associated with 3D convolution-based methods. We introduced a transformer-based network for hand gesture recognition, which is a completely self-attentional architecture without any convolution or recurrent layers. The framework classifies hand gestures by focusing on the sequence information of the whole gesture video. In addition, we introduced an adaptive sampling strategy based on the video content to reduce the input of gesture-free frames to the model, thus reducing computational consumption. The proposed network achieved 83.2% and 93.8% recognition accuracy on two publicly available benchmark datasets, NVGesture and EgoGesture datasets, respectively. The results of extensive comparison experiments show that our proposed approach outperforms the existing state-of-the-art gesture recognition systems.

show abstract

Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network

Cited by 76 publications

References 34 publications

A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System

A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System

Backhand-Approach-Based American Sign Language Words Recognition Using Spatial-Temporal Body Parts and Hand Relationship Patterns

Content-Adaptive and Attention-Based Network for Hand Gesture Recognition

Contact Info

Product

Resources

About