Survey on depth and RGB image-based 3D hand shape and pose estimation

Huang, Lin; Zhang, Boshen; Guo, Zhilin; Xiao, Yang; Cao, Zhiguo; Yuan, Junsong

doi:10.1016/j.vrih.2021.05.002

Cited by 32 publications

(12 citation statements)

References 115 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the outputs, these methods can be grouped into two categories, i.e., methods for 3D joints prediction [165], [167], [168], [169], methods producing statistical models for a single hand [161], [162], [164], [166], [170], [171], [172] and two hands [173], [174]. For a thorough review of the recent advances in 3D hand pose and shape estimation, please refer to [175], [176].…”

Section: Individual Reconstruction Of Hands and Facementioning

confidence: 99%

Recovering 3D Human Mesh from Monocular Images: A Survey

Tian¹,

Zhang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Estimating human pose and shape from monocular images is a long-standing problem in computer vision. Since the release of statistical body models, 3D human mesh recovery has been drawing broader attention. With the same goal of obtaining well-aligned and physically plausible mesh results, two paradigms have been developed to overcome challenges in the 2D-to-3D lifting process: i) an optimization-based paradigm, where different data terms and regularization terms are exploited as optimization objectives; and ii) a regression-based paradigm, where deep learning techniques are embraced to solve the problem in an end-to-end fashion. Meanwhile, continuous efforts are devoted to improving the quality of 3D mesh labels for a wide range of datasets. Though remarkable progress has been achieved in the past decade, the task is still challenging due to flexible body motions, diverse appearances, complex environments, and insufficient in-the-wild annotations. To the best of our knowledge, this is the first survey to focus on the task of monocular 3D human mesh recovery. We start with the introduction of body models and then elaborate recovery frameworks and training objectives by providing in-depth analyses of their strengths and weaknesses. We also summarize datasets, evaluation metrics, and benchmark results. Open issues and future directions are discussed in the end, hoping to motivate researchers and facilitate their research in this area. A regularly updated project page can be found at https://github.com/tinatiansjz/hmr-survey.

show abstract

Section: Individual Reconstruction Of Hands and Facementioning

confidence: 99%

Recovering 3D Human Mesh from Monocular Images: A Survey

Tian¹,

Zhang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Currently, computer vision-based gesture recognition methods are being widely implemented. This method consists of data collection through cameras, preprocessing, gesture segmentation, gesture analysis and gesture recognition [21]. Multimodal interaction technologies use two or more modalities of communication to recognize the instructions from the user [22].…”

Section: Gesture Interactions and Recognition In Virtual Realitymentioning

confidence: 99%

Automated Recognition of Hand Gestures for Crane Rigging using Data Gloves in Virtual Reality

Harichandran¹,

Teizer²

2022

Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC)

View full text Add to dashboard Cite

Construction safety training through Virtual Reality (VR) environments offers workers an interactive and immersive experience. Many complex interactions and realistic scenarios are possible in VR using accessories such as data gloves and trackers which allow data recording on incidents. For example, misinterpretation of hand signals construction workers give during operations may result in severe accidents on construction sites. This study proposes automatic gesture recognition to identify hand signals for crane rigging operations in VR training. The developed gesture recognition algorithm tracks information from a data glove and a tracker. Preliminary data on movements and orientation of hands and fingers were recorded. Mathematical models of hand gestures were created based on finger movement data. Gesture rules were created based on the rotation and orientation of the hand. The gesture models were combined with the gesture rules to develop the algorithm for automated gesture recognition. Final experiments estimated the efficacy of the proposed method in automatically recognizing crane rigging signals in real-time. The performance is evaluated by comparing the identified hand gestures with independently created ground truth labels. The proposed method identified static hand gestures with an average accuracy of 96.55 percent. This method recognizes the gestures along with the hand movements and displays the results in real-time equivalent to dynamic gesture recognition. More refined dynamic gesture recognition based on this method is in progress.

show abstract

“…[4], present a method for pose estimation of a robotic hand, based on a particle filter and GPU framework. Lin Huang [5], present a comprehensive survey of state-of-the-art 3D hand shape and pose estimation approaches using RGB-D cameras. Related RGB-D cameras, hand datasets, and performance analysis are also discussed to provide a holistic view of recent achievements.…”

Section: Game Developmentmentioning

confidence: 99%

Gesture controlled interaction using hand pose model

Damdoo¹,

Gupta²

2022

ijhs

View full text Add to dashboard Cite

Direct use of the hand as an input device is a smart method for providing natural human-computer interaction (HCI). Hand pose estimation is an attractive topic for research in recent years. It has been widely used in virtual reality. The domain of computer vision-based human hand three-dimensional shape and hand pose estimation has fascinated momentous attention recently due to its key role in various applications, such as natural human-computer interactions. Hand pose estimation is difficult due to some challenges. First, we need to detect the human hand which is very changeable. Second, the high degree of freedom leads to difficulties in pose estimation. In this paper, we aim to build a hand pose estimation system that can correctly detect a human hand and estimate its pose which can be useful in the areas like industrial automation, sign language recognition etc. We integrated a hand pose model with a game named U3D with 6 different gestures. As the object detection methods perform poorly in the palm detection tasks we used 21 hand joints and increased accuracy to approximately 95.63%. We used the Blender 3D computer graphics software toolset for creating animation, visual effects, motion graphics, and interactive unity 3D games.

show abstract

Survey on depth and RGB image-based 3D hand shape and pose estimation

Cited by 32 publications

References 115 publications

Recovering 3D Human Mesh from Monocular Images: A Survey

Recovering 3D Human Mesh from Monocular Images: A Survey

Automated Recognition of Hand Gestures for Crane Rigging using Data Gloves in Virtual Reality

Gesture controlled interaction using hand pose model

Contact Info

Product

Resources

About