3D Hand Gestures Segmentation and Optimized Classification Using Deep Learning

Khan, Fawad Salam; Mohd, Mohd Norzali Haji; Soomro, Dur Muhammad; Bagchi, Susama; Khan, M. Danial

doi:10.1109/access.2021.3114871

Cited by 10 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vision-based marker-less hand tracking holds great promise for HCI applications, including sign language recognition, augmented reality, telesurgery, home automation, and gaming. However, its implementation faces several challenges, including tracking inaccuracy caused by complex articulated hand motion, high appearance variability, and demanding computational and real-time requirements [10][11][12][13][14]. Consequently, research in this field continues to be a challenging problem that has gained significant attention from the computer vision community.…”

Section: Related Work and Motivationmentioning

confidence: 99%

Vision-Based Hand Detection and Tracking Using Fusion of Kernelized Correlation Filter and Single-Shot Detection

Haji Mohd,

Mohd Asaari,

Lay Ping

et al. 2023

Applied Sciences

Self Cite

View full text Add to dashboard Cite

Hand detection and tracking are key components in many computer vision applications, including hand pose estimation and gesture recognition for human–computer interaction systems, virtual reality, and augmented reality. Despite their importance, reliable hand detection in cluttered scenes remains a challenge. This study explores the use of deep learning techniques for fast and robust hand detection and tracking. A novel algorithm is proposed by combining the Kernelized Correlation Filter (KCF) tracker with the Single-Shot Detection (SSD) method. This integration enables the detection and tracking of hands in challenging environments, such as cluttered backgrounds and occlusions. The SSD algorithm helps reinitialize the KCF tracker when it fails or encounters drift issues due to sudden changes in hand gestures or fast movements. Testing in challenging scenes showed that the proposed tracker achieved a tracking rate of over 90% and a speed of 17 frames per second (FPS). Comparison with the KCF tracker on 17 video sequences revealed an average improvement of 13.31% in tracking detection rate (TRDR) and 27.04% in object detection error (OTE). Additional comparison with MediaPipe hand tracker on 10 hand gesture videos taken from the Intelligent Biometric Group Hand Tracking (IBGHT) dataset showed that the proposed method outperformed the MediaPipe hand tracker in terms of overall TRDR and tracking speed. The results demonstrate the promising potential of the proposed method for long-sequence tracking stability, reducing drift issues, and improving tracking performance during occlusions.

show abstract

Section: Related Work and Motivationmentioning

confidence: 99%

Vision-Based Hand Detection and Tracking Using Fusion of Kernelized Correlation Filter and Single-Shot Detection

Haji Mohd,

Mohd Asaari,

Lay Ping

et al. 2023

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…These signals are then classified using machine learning algorithms, which are trained with a large number of samples to achieve accurate recognition. Khan Fawad Salam et al [52] employed the MASK-RCNN combined with the Grass Hopper algorithm for classifying the obtained RGB images and hand keypoints. Jaya Prakash Sahoo et al [53] developed an end-to-end fine-tuning method for pre-trained CNN models using fractional-level fusion technology.…”

Section: Comparison and Analysismentioning

confidence: 99%

Research Progress of Human–Computer Interaction Technology Based on Gesture Recognition

Zhou

Wang

et al. 2023

Electronics

View full text Add to dashboard Cite

Gesture recognition, as a core technology of human–computer interaction, has broad application prospects and brings new technical possibilities for smart homes, medical care, sports training, and other fields. Compared with the traditional human–computer interaction models based on PC use with keyboards and mice, gesture recognition-based human–computer interaction modes can transmit information more naturally, flexibly, and intuitively, which has become a research hotspot in the field of human–computer interaction in recent years. This paper described the current status of gesture recognition technology, summarized the principles and development history of electromagnetic wave sensor recognition, stress sensor recognition, electromyographic sensor recognition, and visual sensor recognition, and summarized the improvement of this technology by researchers in recent years through the direction of sensor structure, selection of characteristic signals, the algorithm of signal processing, etc. By sorting out and comparing the typical cases of the four implementations, the advantages and disadvantages of each implementation and the application scenarios were discussed from the two aspects of dataset size and accuracy. Based on the abovementioned discussion, the problems and challenges of current gesture recognition technology were discussed in terms of the biocompatibility of sensor structures, wearability and adaptability, stability, robustness, and crossover of signal acquisition and analysis algorithms, and the future development directions in this field were proposed.

show abstract

“…For example, researchers leverage PoseNet networks to delineate hand poses through feature extraction and reverse inference of hand key points [18][19][20], thereby enhancing accuracy and robustness [21,22]. Additionally, approaches utilizing deep learning networks like ResNet-50 process feature maps to output heat maps of hand target joints, yielding commendable results in multi-target recognition [23]. Some methodologies employ two-stage networks, initially predicting hands through hand masks and subsequently performing pose estimation, mutually refining to precisely locate hand key points and bolster robustness amidst complex backgrounds [24].…”

Section: Introductionmentioning

confidence: 99%

Fusion Enhancement of YOLOv5 and Copula Bayesian Classifier for Hand Gesture Recognition in Smart Sports Venues

Han,

Hao,

Yuan

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This study aims addresses the challenge of gesture recognition in smart sports stadiums through the fusion enhancement of YOLOv5 and Copula Bayesian Classifier algorithm. By leveraging FasterNet for YOLOv5 transformation, integrating the Convolutional Block Attention Module (CBAM) as an attention mechanism, and merging it with the Copula Bayesian Classifier, we propose a tandem model for robust gesture recognition in the complex environments of sports venues. Extensive training on a substantial dataset yields an efficient and accurate gesture recognition model. Our findings showcase the effectiveness of the proposed algorithm, achieving a remarkable accuracy rate of 99.2% in identifying gesture categories swiftly. This advancement holds significant implications for future human-computer interaction in smart sports stadiums, offering a more seamless and intelligent mode of interaction for both spectators and athletes. Ultimately, it enhances user experience and management efficiency within smart sports venues. INDEX TERMSCopula Bayesian Classifier, Fusion enhancement of YOLOv5, Human-computer interaction, Hand gesture recognition, Image processing.

show abstract

3D Hand Gestures Segmentation and Optimized Classification Using Deep Learning

Cited by 10 publications

References 31 publications

Vision-Based Hand Detection and Tracking Using Fusion of Kernelized Correlation Filter and Single-Shot Detection

Vision-Based Hand Detection and Tracking Using Fusion of Kernelized Correlation Filter and Single-Shot Detection

Research Progress of Human–Computer Interaction Technology Based on Gesture Recognition

Fusion Enhancement of YOLOv5 and Copula Bayesian Classifier for Hand Gesture Recognition in Smart Sports Venues

Contact Info

Product

Resources

About