Combining 2d Feature Tracking and Volume Reconstruction for Online Video-Based Human Motion Capture

Abstract

The acquisition of human motion data is of major importance for creating interactive virtual environments, intelligent user interfaces, and realistic computer animations. Today's performance of off-the-shelf computer hardware enables marker-free non-intrusive optical tracking of the human body. In addition, recent research shows that it is possible to efficiently acquire and render volumetric scene representations in real-time. This paper describes a system to capture human motion without the use of markers or sceneintruding devices. Instead, a 2D feature tracking algorithm and a silhouette-based 3D volumetric scene reconstruction method are applied directly to the image data. A person is recorded by multiple synchronized cameras, and a multi-layer hierarchical kinematic skeleton is fitted to each frame in a two-stage process. The pose of a first model layer at every time step is determined from the tracked 3D locations of hands, head and feet. A more sophisticated second skeleton layer is fitted to the motion data by applying a volume registration technique. We present results with a prototype system showing that the approach is capable of running at interactive frame rates.

Real-Time 3D Body Pose Estimation

“…Voxels lying outside of the silhouette in at least one view do not belong to the intersection of the cones and can be discarded. Because of their simplicity, voxel-based procedures have been used for body tracking [26][27][28][29][30]. Their drawback is their tendency to be expensive as a high number must be projected into the image planes.…”

Combining 3D flow fields with silhouette-based human motion capture for immersive video

“…Ellipsoidal body models [6] , kinematic skeletons [20], or skeleton models with attached volume samples [31] are fitted to the volume data.…”

Recovering refined surface normals for relighting clothing in dynamic scenes

“…Over the past decade there has been extensive research in computer vision and graphics on reconstruction of moving people from multiple view video [1][2] [3][4] [5]. These approaches allow performance capture and replay from novel viewpoints.…”