No abstract

Abstract-3D video is a real 3D movie recording the object's full 3D shape, motion, and precise surface texture. This paper first proposes a parallel pipeline processing method for reconstructing dynamic 3D object shape from multi-view video images, by which a temporal series of full 3D voxel representations of the object behavior can be obtained in real-time. To realize the real-time processing, we first introduce a plane-based volume intersection algorithm: represent an observable 3D space by a group of parallel plane slices, back-project observed multi-view object silhouettes onto each slice, and apply 2D silhouette intersection on each slice. Then, we propose a method to parallelize this algorithm using a PC cluster, where we employ 5 stage pipeline processing in each PC as well as slice-by-slice parallel silhouette intersection. Several results of the quantitative performance evaluation are given to demonstrate the effectiveness of the proposed methods. In the latter half of the paper, we present an algorithm of generating video texture on the reconstructed dynamic 3D object surface. We first describe a naive viewindependent rendering method and show its problems. Then, we improve the method by introducing image-based rendering techniques. Experimental results demonstrate the effectiveness of the improved method in generating high fidelity object images from arbitrary viewpoints.

No abstract

Abstract

Abstract-Existing systems for 3D reconstruction from multiple view video use controlled indoor environments with uniform illumination and backgrounds to allow accurate segmentation of dynamic foreground objects. In this paper we present a portable system for 3D reconstruction of dynamic outdoor scenes which require relatively large capture volumes with complex backgrounds and non-uniform illumination. This is motivated by the demand for 3D reconstruction of natural outdoor scenes to support film and broadcast production. Limitations of existing multiple view 3D reconstruction techniques for use in outdoor scenes are identified. Outdoor 3D scene reconstruction is performed in three stages: (1) 3D background scene modelling using spherical stereo image capture; (2) multiple view segmentation of dynamic foreground objects by simultaneous video matting across multiple views; and (3) robust 3D foreground reconstruction and multiple view segmentation refinement in the presence of segmentation and calibration errors. Evaluation is performed on several outdoor productions with complex dynamic scenes including people and animals. Results demonstrate that the proposed approach overcomes limitations of previous indoor multiple view reconstruction approaches enabling high-quality free-viewpoint rendering and 3D reference models for production.