Shape-From-Silhouette Across Time Part II: Applications to Human Modeling and Markerless Motion Tracking

Abstract

In Part I of this paper we developed the theory and algorithms for performing Shape-From-Silhouette (SFS) across time. In this second part, we show how our temporal SFS algorithms can be used in the applications of human modeling and markerless motion tracking. First we build a system to acquire human kinematic models consisting of precise shape (constructed using the temporal SFS algorithm for rigid objects), joint locations, and body part segmentation (estimated using the temporal SFS algorithm for articulated objects). Once the kinematic models have been built, we show how they can be used to track the motion of the person in new video sequences. This marker-less tracking algorithm is based on the Visual Hull alignment algorithm used in both temporal SFS algorithms and utilizes both geometric (silhouette) and photometric (color) information.Electronic Supplementary Material: Supplementary material to this paper is avialable in electronic form at http://dx.

Optimization and Filtering for Human Motion Capture

“…Local optimization has been widely used for 3D human motion capture (Bregler and Malik 1998;Cheung et al 2005;Gall et al 2008;Gavrila and Davis 1996;Kakadiaris and Metaxas 1996;Kehl et al 2005;Mundermann et al 2007). It provides very accurate results provided that the state vector is initialized near the global optimum.…”

Intrinsic Characterization of Dynamic Surfaces

“…During the last decade, several multiview video systems and applications have been developed [22,27,17,10,2,34,11,18,19,20,37]. They are able to capture real-world human or animal performances, and produce free-viewpoint video of the subjects in motion in a virtual world (see Fig.…”

On the Evaluation of Scene Flow Estimation

“…Significant progress has been made but the problem has not been fully solved. Much better results have been obtained for humans via the use of articulated models which drastically reduces the number of parameters to be estimated [10][11][12][13]. Due to the very different nature of such strongly model-based estimation, we consider these methods out of scope here.…”