3D Pictorial Structures for Multiple Human Pose Estimation

Abstract

In this work, we address the problem of 3D pose estimation of multiple humans from multiple views. This is a more challenging problem than single human 3D pose estimation due to the much larger state space, partial occlusions as well as across view ambiguities when not knowing the identity of the humans in advance. To address these problems, we first create a reduced state space by triangulation of corresponding body joints obtained from part detectors in pairs of camera views. In order to resolve the ambiguities of wrong and mixed body parts of multiple humans after triangulation and also those coming from false positive body part detections, we introduce a novel 3D pictorial structures (3DPS) model. Our model infers 3D human body configurations from our reduced state space. The 3DPS model is generic and applicable to both single and multiple human pose estimation.In order to compare to the state-of-the art, we first evaluate our method on single human 3D pose estimation on HumanEva-I [22] and KTH Multiview Football Dataset II [8] datasets. Then, we introduce and evaluate our method on two datasets for multiple human 3D pose estimation.

Multiple Human Pose Estimation with Temporally Consistent 3D Pictorial Structures

“…Given the track of each individual, we signicantly reduce state space of each joint, which leads to much faster inference. On two challenging datasets for multiple human pose estimating from multiple views, we obtain the best results in comparison to previous work [6]. …”

“…The 3DPS model for multiple human pose estimation [6] is similar to our model. In this approach, a model for inferring multiple human body poses without knowing their identity has been proposed.…”

“…This term keeps the inferred poses consistent over time. In the experiments we directly compare to [6] and observe that these differences lead not only to a significantly reduced state space and thus faster inference, but also to significantly improved performance. Finally, our model is advisable for multiple human pose estimation, where there are different individuals.…”

“…In every application, the human motion remains consistent over time. Different approaches have been proposed for multiple human pose estimation on 2D [1,4,10,24] or 3D space [5,6,14,20,21]. Nevertheless, the utility of temporal consistency for pose estimation has not been sufficiently addressed yet.…”

“…The model has been extended to the 3D space, in order to cope with mutli-view camera setups as well [2,9,16]. Recently, pictorial structures have been successfully modelled for multiple human 3D pose estimation [6]. However, the temporal consistency, between the estimated body poses of different individuals among subsequent frames, has never been addressed within the framework of pictorial structures.…”