Generating Realistic Training Images Based on Tonality-Alignment Generative Adversarial Networks for Hand Pose Estimation

Chen, Liangjian; Lin, Shih-Yao; Xie, Yusheng; Tang, Hui; Xue, Yufan; Xie, Xiaohui; Lin, Yen‐Yu; Wang, Fan

doi:10.48550/arxiv.1811.09916

Cited by 5 publications

(4 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…InterHand2.6M is captured in a multi-camera studio consisting of 80-140 cameras capturing at 30-90 frames-per-second (fps), and 350-450 directional LED point lights directed at the hand to promote uniform illumination 3 . The cameras captured at image resolution 4096 × 2668.…”

Section: Data Capturementioning

confidence: 99%

InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

Moon

et al. 2020

Lecture Notes in Computer Science

174

311

View full text Add to dashboard Cite

Analysis of hand-hand interactions is a crucial step towards better understanding human behavior. However, most researches in 3D hand pose estimation have focused on the isolated single hand case. Therefore, we firstly propose (1) a large-scale dataset, InterHand2.6M, and (2) a baseline network, InterNet, for 3D interacting hand pose estimation from a single RGB image. The proposed InterHand2.6M consists of 2.6M labeled single and interacting hand frames under various poses from multiple subjects. Our InterNet simultaneously performs 3D single and interacting hand pose estimation. In our experiments, we demonstrate big gains in 3D interacting hand pose estimation accuracy when leveraging the interacting hand data in InterHand2.6M. We also report the accuracy of InterNet on InterHand2.6M, which serves as a strong baseline for this new dataset. Finally, we show 3D interacting hand pose estimation results from general images. Our code and dataset are available 1 .

show abstract

Section: Data Capturementioning

confidence: 99%

InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

Moon

et al. 2020

Lecture Notes in Computer Science

174

311

View full text Add to dashboard Cite

show abstract

“…RGB cameras are much more widely used than depth sensors. Estimating 3D hand poses merely from monocu-lar RGB images are more practical and active in the literature [5,10,19,29,37,41,49,24,23,11]. The pioneering work by Zimmermann and Brox [49] utilizes convolutional neural networks (CNN) to extract image feature, and feed camera parameters with these features to a 3D lift network where depth information is then estimated.…”

Section: Rgb Based 3d Hand Pose Estimationmentioning

confidence: 99%

MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation

Chen

Lin

Xie

et al. 2021

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)

Self Cite

View full text Add to dashboard Cite

Estimating 3D hand poses from a single RGB image is challenging because depth ambiguity leads the problem illposed. Training hand pose estimators with 3D hand mesh annotations and multi-view images often results in significant performance gains. However, existing multi-view datasets are relatively small with hand joints annotated by off-the-shelf trackers or automated through model predictions, both of which may be inaccurate and can introduce biases. Collecting a large-scale multi-view 3D hand pose images with accurate mesh and joint annotations is valuable but strenuous. In this paper, we design a spin match algorithm that enables a rigid mesh model matching with any target mesh ground truth. Based on the match algorithm, we propose an efficient pipeline to generate a largescale multi-view hand mesh (MVHM) dataset with accurate 3D hand mesh and joint labels. We further present a multi-view hand pose estimation approach to verify that training a hand pose estimator with our generated dataset greatly enhances the performance. Experimental results show that our approach achieves the performance of 0.990 in AUC 20-50 on the MHP dataset compared to the previous state-of-the-art of 0.939 on this dataset. Our datasset is available at https://github.com/Kuzphi/MVHM.

show abstract

“…Since depth images contain surface geometry information of hands, they are widely used for hand pose estimation in the literature [40,44,11,41,14,16,27,8,9]. Most existing work adopts regression to fit the parameters of a deformed hand model [30,22,24,40].…”

Section: D Hand Pose Estimation From Depth Imagesmentioning

confidence: 99%

Temporal-Aware Self-Supervised Learning for 3D Hand Pose and Mesh Estimation in Videos

Chen

Lin

Xie

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Estimating 3D hand pose directly from RGB images is challenging but has gained steady progress recently by training deep models with annotated 3D poses. However annotating 3D poses is difficult and as such only a few 3D hand pose datasets are available, all with limited sample sizes. In this study, we propose a new framework of training 3D pose estimation models from RGB images without using explicit 3D annotations, i.e., trained with only 2D information. Our framework is motivated by two observations: 1) Videos provide richer information for estimating 3D poses as opposed to static images; 2) Estimated 3D poses ought to be consistent whether the videos are viewed in the forward order or reverse order. We leverage these two observations to develop a self-supervised learning model called temporal-aware self-supervised network (TASSN). By enforcing temporal consistency constraints, TASSN learns 3D hand poses and meshes from videos with only 2D keypoint position annotations. Experiments show that our model achieves surprisingly good results, with 3D estimation accuracy on par with the state-of-the-art models trained with 3D annotations, highlighting the benefit of the temporal consistency in constraining 3D prediction models.

show abstract

Generating Realistic Training Images Based on Tonality-Alignment Generative Adversarial Networks for Hand Pose Estimation

Cited by 5 publications

References 43 publications

InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation

Temporal-Aware Self-Supervised Learning for 3D Hand Pose and Mesh Estimation in Videos

Contact Info

Product

Resources

About