Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation

Abstract

Estimation of 3D human pose from monocular image has gained considerable attention, as a key step to several human-centric applications. However, generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable, as these models often perform unsatisfactorily on unseen in-the-wild environments. Though weakly-supervised models have been proposed to address this shortcoming, performance of such models relies on availability of paired supervision on some related task, such as 2D pose or multi-view image pairs. In contrast, we propose a novel kinematic-structure-preserved unsupervised 3D pose estimation framework, which is not restrained by any paired or unpaired weak supervisions. Our pose estimation framework relies on a minimal set of prior knowledge that defines the underlying kinematic 3D structure, such as skeletal joint connectivity information with bone-length ratios in a fixed canonical scale. The proposed model employs three consecutive differentiable transformations namely forward-kinematics, camera-projection and spatial-map transformation. This design not only acts as a suitable bottleneck stimulating effective pose disentanglement, but also yields interpretable latent pose representations avoiding training of an explicit latent embedding to pose mapper. Furthermore, devoid of unstable adversarial setup, we re-utilize the decoder to formalize an energy-based loss, which enables us to learn from in-the-wild videos, beyond laboratory settings. Comprehensive experiments demonstrate our state-of-the-art unsupervised and weakly-supervised pose estimation performance on both Human3.6M and MPI-INF-3DHP datasets. Qualitative results on unseen environments further establish our superior generalization ability.

Graph and Temporal Convolutional Networks for 3D Multi-person Pose Estimation in Monocular Videos

“…It is observed that our method is comparable with the SOTA methods. In addition, although our method shows improved performance over others that use kinematic constraints (Wandt and Rosenhahn 2019;Cheng et al 2019) because of our GCNs and TCNs, adding kinematic constraints could potentially improve our performance further (Akhter and Black 2015;Kundu et al 2020). Qualitative Results As shown in Figure 3, our full model can better handle occlusions and incorrect detection compared with the baselines and the relative positions among all persons are well captured without camera parameters.…”

Overview of 3D Human Pose Estimation

“…By contrast, unsupervised learning can directly derive the properties of the data from the data itself and then summarize them, thereby enabling researchers to use these properties to make data-driven decisions. Therefore, some unsupervised-learning-based human pose estimation methods have been studied extensively [113][114][115]. Unsupervised learning usually treats pose estimation as a template matching problem that can be learned.…”

Deep Learning-Based Human Pose Estimation: A Survey

“…Observing that human body parts have a distinct degree of freedom (DOF) based on the kinematic structure, Wang et al [151] and Nie et al [154] proposed bidirectional networks to model the kinematic and geometric dependencies of the human skeleton. Kundu et al [152] [157] designed a kinematic structure preservation approach by inferring local-kinematic parameters with energy-based loss and explored 2D part segments based on the parent-relative local limb kinematic model. Xu et al [153] demonstrated that noisy 2D joint is one of the key obstacles for accurate 3D pose estimation.…”