3D Pictorial Structures Revisited: Multiple Human Pose Estimation

Abstract

We address the problem of 3D pose estimation of multiple humans from multiple views. The transition from single to multiple human pose estimation and from the 2D to 3D space is challenging due to a much larger state space, occlusions and across-view ambiguities when not knowing the identity of the humans in advance. To address these problems, we first create a reduced state space by triangulation of corresponding pairs of body parts obtained by part detectors for each camera view. In order to resolve ambiguities of wrong and mixed parts of multiple humans after triangulation and also those coming from false positive detections, we introduce a 3D pictorial structures (3DPS) model. Our model builds on multi-view unary potentials, while a prior model is integrated into pairwise and ternary potential functions. To balance the potentials' influence, the model parameters are learnt using a Structured SVM (SSVM). The model is generic and applicable to both single and multiple human pose estimation. To evaluate our model on single and multiple human pose estimation, we rely on four different datasets. We first analyse the contribution of the potentials and then compare our results with related work where we demonstrate superior performance.

Parsing human skeletons in an operating room

“…Next, we present the body prior model, as part of the pairwise and ternary potential functions. We follow the same formulation as in [7,8] to present the 3DPS model.…”

“…3). We follow the same formulation with [8] and model the upper human body with n parts such that a 3D body configuration is given by Y = (Y 1 , Y 2 , . .…”

“…In the following, we briefly present the potential functions. A detailed description of the 3DPS model can be found in [7,8].…”

“…Pairwise and ternary potentials The 3D body prior is encoded in pairwise and ternary potential functions. In detail, we model the kinematic body constraints in terms of translation and rotation between physical body parts using pairwise and ternary potentials respectively [8]. The translation potential corresponds to the translation of the part i to the local coordinate system of the part j and it is modeled with a multivariate Gaussian distribution as:…”

“…The parameters w of the model balance the influence of the potential functions to the inference task. We follow the formulation of [8] and learn a weight for each potential function based on a set of training samples S with labels y s ∈ {−1, 1}. A feature vector Φ(φ s , ψ s ) with the concatenation of all potential functions is constructed for every training sample.…”