A Robust Stereo Prior for Human Segmentation

Abstract

Abstract. The emergence of affordable depth cameras has enabled significant advances in human segmentation and pose estimation in recent years. While it leads to impressive results in many tasks, the use of infrared cameras have their drawbacks, in particular the fact that they don't work in direct sunlight. One alternative is to use a stereo pair of cameras to produce a disparity space image. In this work, we propose a robust method of using a disparity space image to create a prior for human segmentation. This new prior leads to greatly improved segmentation results; it can be applied to any task where a stereo pair of cameras is available, and segmentation results are desired. As an application, we show how the prior can be inserted into a dual decomposition formulation for stereo, segmentation and human pose estimation.

Pose Estimation and Segmentation of People in 3D Movies

“…Our new annotated Inria 3DMovie Dataset used for evaluation in this paper is available on the project website [3]. We also compare our method with [30] on the H2view dataset.…”

“…H2view dataset. The H2view dataset [30] Colour only 0.630 Eichner et al [12] 0.662 Table 2. Evaluation of pixel-wise person segmentation.…”

“…It is, however, restricted to a single person setup and hence has no inter-person occlusions. We tested our model (trained on 3D movies) directly on this dataset, without any further tuning, and analyzed the segmentation quality using the evaluation code from [30]. As our method models only the upper body, we cropped the ground truth, our results, and those from [30] just above the hips, and considered only upper body (rather than full body) segmentation.…”

“…In addition to the significant progress in human pose estimation in still images and videos [15,18,28,41], there has been some work in joint pose estimation and segmentation [20,24,30,36]. However, these works are limited to cases involving isolated people, and extending them to situations with multiple interacting people is not straightforward.…”

“…We evaluate the proposed model on the new Inria 3DMovie Dataset with challenging realistic dynamic scenes from two stereoscopic feature-length movies "StreetDance" [Giwa and Pasquini, 2010] and "Pina" [Wenders, 2011]. Additionally, we compare our results on the Humans in Two Views (H2view) dataset [30] (Section 5).…”