3D Hand Gesture Recognition Using a Depth and Skeletal Dataset

Smedt, Quentin De; Wannous, Hazem; Vandeborre, Jean-Philippe; Guerry, Joris; Saux, Bertrand Le; Filliat, David

doi:10.2312/3dor.20171049

Cited by 23 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Briareo [ 51 ] and Multimodal Hand Gesture [ 52 ] datasets were used to evaluate the proposed hand recognition system. They contain RGB and IR images, unlike other related datasets, such as the DHG 14/28 [ 23 ] and SHREC’17 track [ 53 ], easing the integration of the system into a wide range of applications as no specialized sensors are required. The Briareo dataset is focused on dynamic hand gesture recognition and contains 12 different classes of gestures performed with the right hand by 40 different people (33 men and 7 women).…”

Section: Results and Discussionmentioning

confidence: 99%

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Zhong,

del-Blanco,

Berjón

et al. 2023

Sensors

View full text Add to dashboard Cite

Automatic hand gesture recognition in video sequences has widespread applications, ranging from home automation to sign language interpretation and clinical operations. The primary challenge lies in achieving real-time recognition while managing temporal dependencies that can impact performance. Existing methods employ 3D convolutional or Transformer-based architectures with hand skeleton estimation, but both have limitations. To address these challenges, a hybrid approach that combines 3D Convolutional Neural Networks (3D-CNNs) and Transformers is proposed. The method involves using a 3D-CNN to compute high-level semantic skeleton embeddings, capturing local spatial and temporal characteristics of hand gestures. A Transformer network with a self-attention mechanism is then employed to efficiently capture long-range temporal dependencies in the skeleton sequence. Evaluation of the Briareo and Multimodal Hand Gesture datasets resulted in accuracy scores of 95.49% and 97.25%, respectively. Notably, this approach achieves real-time performance using a standard CPU, distinguishing it from methods that require specialized GPUs. The hybrid approach’s real-time efficiency and high accuracy demonstrate its superiority over existing state-of-the-art methods. In summary, the hybrid 3D-CNN and Transformer approach effectively addresses real-time recognition challenges and efficient handling of temporal dependencies, outperforming existing methods in both accuracy and speed.

show abstract

Section: Results and Discussionmentioning

confidence: 99%

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Zhong,

del-Blanco,

Berjón

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Yang et al [31] proposed a hand-crafted location-viewpoint-invariant feature (namely, JCD) which computes Euclidean distances between pairs of joints, and used it together with two scales of scale invariant motion features as input, along with a CNN formed mainly by 1D ConvNet layers for processing. It has a simple and elegant design and is currently the state of the art (SOTA) on two public datasets: JHMDB [28] and SHREC [32]. However, further experiments on real-world applications are still necessary because of strict requirements for accurate and sufficient pose information.…”

Section: Feature Extraction For Skeleton-based Action Recognitionmentioning

confidence: 99%

Towards Building a Visual Behaviour Analysis Pipeline for Suicide Detection and Prevention

Onie

Liang

et al. 2022

Sensors

View full text Add to dashboard Cite

Understanding human behaviours through video analysis has seen significant research progress in recent years with the advancement of deep learning. This topic is of great importance to the next generation of intelligent visual surveillance systems which are capable of real-time detection and analysis of human behaviours. One important application is to automatically monitor and detect individuals who are in crisis at suicide hotspots to facilitate early intervention and prevention. However, there is still a significant gap between research in human action recognition and visual video processing in general, and their application to monitor hotspots for suicide prevention. While complex backgrounds, non-rigid movements of pedestrians and limitations of surveillance cameras and multi-task requirements for a surveillance system all pose challenges to the development of such systems, a further challenge is the detection of crisis behaviours before a suicide attempt is made, and there is a paucity of datasets in this area due to privacy and confidentiality issues. Most relevant research only applies to detecting suicides such as hangings or jumps from bridges, providing no potential for early prevention. In this research, these problems are addressed by proposing a new modular design for an intelligent visual processing pipeline that is capable of pedestrian detection, tracking, pose estimation and recognition of both normal actions and high risk behavioural cues that are important indicators of a suicide attempt. Specifically, based on the key finding that human body gestures can be used for the detection of social signals that potentially precede a suicide attempt, a new 2D skeleton-based action recognition algorithm is proposed. By using a two-branch network that takes advantage of three types of skeleton-based features extracted from a sequence of frames and a stacked LSTM structure, the model predicts the action label at each time step. It achieved good performance on both the public dataset JHMDB and a smaller private CCTV footage collection on action recognition. Moreover, a logical layer, which uses knowledge from a human coding study to recognise pre-suicide behaviour indicators, has been built on top of the action recognition module to compensate for the small dataset size. It enables complex behaviour patterns to be recognised even from smaller datasets. The whole pipeline has been tested in a real-world application of suicide prevention using simulated footage from a surveillance system installed at a suicide hotspot, and preliminary results confirm its effectiveness at capturing crisis behaviour indicators for early detection and prevention of suicide.

show abstract

“…The framework consists of an end-to-end trainable network that exploits both spatial and temporal information on the input data and applies an attention mechanism on both input modalities. This results in a lightweight but accurate gesture recognition system and is evaluated on two publicly available datasets [8,7].…”

Section: Accepted Papersmentioning

confidence: 99%

HANDS18: Methods, Techniques and Applications for Hand Observation

Oikonomidis

Garcia-Hernando

Yao

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This report outlines the proceedings of the Fourth International Workshop on Observing and Understanding Hands in Action (HANDS 2018). The fourth instantiation of this workshop attracted significant interest from both academia and the industry. The program of the workshop included regular papers that are published as the workshop's proceedings, extended abstracts, invited posters, and invited talks. Topics of the submitted works and invited talks and posters included novel methods for hand pose estimation from RGB, depth, or skeletal data, datasets for special cases and real-world applications, and techniques for hand motion re-targeting and hand gesture recognition. The invited speakers are leaders in their respective areas of specialization, coming from both industry and academia. The main conclusions that can be drawn are the turn of the community towards RGB data and the maturation of some methods and techniques, which in turn has led to increasing interest for real-world applications.

show abstract

3D Hand Gesture Recognition Using a Depth and Skeletal Dataset

Cited by 23 publications

References 1 publication

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Towards Building a Visual Behaviour Analysis Pipeline for Suicide Detection and Prevention

HANDS18: Methods, Techniques and Applications for Hand Observation

Contact Info

Product

Resources

About