Learning 3D Human Pose from Structure and Motion

Abstract

3D human pose estimation from a single image is a challenging problem, especially for in-the-wild settings due to the lack of 3D annotated data. We propose two anatomically inspired loss functions and use them with the weaklysupervised learning framework of [41] to jointly learn from large-scale in-thewild 2D and indoor/synthetic 3D data. We also present a simple temporal network that exploits temporal and structural cues present in predicted pose sequences to temporally harmonize the pose estimations. We carefully analyze the proposed contributions through loss surface visualizations and sensitivity analysis to facilitate deeper understanding of their working mechanism. Our complete pipeline improves the state-of-the-art by 11.8% and 12% on Human3.6M and MPI-INF-3DHP, respectively, and runs at 30 FPS on a commodity graphics card.

Shape-Aware Human Pose and Shape Reconstruction Using Multi-View Images

“…4.4. Our synthetic dataset provides a supervision to not only the joint positions but also the rotations, hence the Method MPJPE PA-MPJPE Tome et al [43] 88.39 -Rogez et al [38] 87.7 71.6 Mehta et al [26] 80.5 -Pavlakos et al [31] 71.9 51.23 Mehta et al [25] 68.6 -Sun et al [40] 59.1 -Zhou et al [55] 107.26 -Debra et al [9] 55.5 -*Kolotouros et al [20] 74.7 51.9 *Omran et al [30] -59.9 *Pavlakos et al [33] -75.9 *HMR [19] 87.97 58.1 *Ours (single-view) 88.34 58.55 *Ours (multi-view) 79.85 45.13 Table 8: Results on Human3.6M. Our method results in smaller reconstruction errors compared to HMR [19].…”

Distill Knowledge From NRSfM for Weakly Supervised 3D Pose Learning

“…Weakly supervised 3D pose learning Most 3D pose estimation methods [36,30,29,47,45,44,28,8,26,6] are fully supervised. One bottleneck for the supervised methods is that data coming from multi-view motion capture systems [19,18] includes limited number of human subject, and has simple backgrounds.…”

“…Deep-NRSfM: We use dictionaries with 6 levels. The size for the dictionaries from lower level to higher is: 256, 128, 64, 32,16,8. When learning the dictionaries, the sparsity weight (λ in Eq.…”

Not All Parts Are Created Equal: 3D Pose Estimation by Modeling Bi-Directional Dependencies of Body Parts

“…In other words, such dependency is modeled in a reversed direction as the one in Block-I. We write,   Ŷ 21 = G21(X,Ŷ12,Ŷ13; θ21), Y22 = G22(X,Ŷ21,Ŷ13; θ22), Y23 = G23(X,Ŷ21,Ŷ22; θ23), (5) where, again,Ŷ ij , G ij , and θ ij respectively denote recovered pose locations, network module, and learnable parameters.…”