TriPose: A Weakly-Supervised 3D Human Pose Estimation via Triangulation from Video

Abstract

Estimating 3D human poses from video is a challenging problem. The lack of 3D human pose annotations is a major obstacle for supervised training and for generalization to unseen datasets. In this work, we address this problem by proposing a weakly-supervised training scheme that does not require 3D annotations or calibrated cameras. The proposed method relies on temporal information and triangulation. Using 2D poses from multiple views as the input, we first estimate the relative camera orientations and then generate 3D poses via triangulation. The triangulation is only applied to the views with high 2D human joint confidence. The generated 3D poses are then used to train a recurrent lifting network (RLN) that estimates 3D poses from 2D poses. We further apply a multi-view re-projection loss to the estimated 3D poses and enforce the 3D poses estimated from multi-views to be consistent. Therefore, our method relaxes the constraints in practice, only multi-view videos are required for training, and is thus convenient for in-thewild settings. At inference, RLN merely requires single-view videos. The proposed method outperforms previous works on two challenging datasets, Human3.6M and MPI-INF-3DHP. Codes and pretrained models will be publicly available.

PosturePose: Optimized Posture Analysis for Semi-Supervised Monocular 3D Human Pose Estimation

“…The accuracy of our single-frame model trained with 2D GT poses (Ours–MvP&P 🟉) matched that of the single-frame TriPose model. Like our framework, TriPose [ 38 ] is a monocular weakly-supervised training scheme that leverages multi-view 2D poses during training. Unlike our framework, TriPose estimates relative camera orientations, which are combined with input 2D poses from multiple views to triangulate a 3D pose.…”

“…The following self-supervised works proposed different strategies for acquiring 3D pose annotations from multi-view 2D data. Gholami et al (TriPose) [ 38 ] triangulated a 3D pose given 2D poses from multiple views and estimated the relative orientation of poses. The triangulated 3D poses were then used as pseudo-annotations to train their 2D–3D pose lifting network.…”

Multi-Camera-Based Human Activity Recognition for Human–Robot Collaboration in Construction

“…Ref. [ 26 ] have developed a weakly supervised 3D pose estimation approach that combines temporal information and triangulation. They estimate the 3D pose by triangulating the location of body joints in each camera view.…”

“…Researchers have investigated multi-camera pose estimation techniques to overcome the limitations of single-camera and RGB-D camera pose estimation. These techniques, including triangulation and Kalman filtering, present potential solutions [ 24 , 25 , 26 , 27 , 28 ]. However, triangulation heavily relies on precise feature matching and assumes known camera parameters, making it vulnerable to occlusion and complex backgrounds.…”