Unification scheme for 3D surface reconstruction using physically based models

Abstract

We propose a unification framework for three-dimensional shape reconstruction using physically based models. A variety of 3D shape reconstruction techniques have been developed in the past two decades, such as shape from stereopsis, from shading, from texture gradient, and from structured lighting. However, the lack of a general theory that unifies these shape reconstruction techniques into one framework hinders the effort of a synergistical image interpretation scheme using multiple sensors/information sources. Most shapefrom-X techniques use an "observable" (e.g., the stereo disparity, intensity, or texture gradient) and a model, which is based on specific domain knowledge (e.g., the triangulation principle, reflectance function, or texture distortion equation) to predict the observable, in 3D shape reconstruction. We show that all these "observable-prediction-model'' types of techniques can be incorporated into our framework of energy constraint on a flexible, deformable image frame. In our algorithm, if the observable does not confirm to the predictions obtained using the corresponding model, a large "error" potential results. The error potential gradient forces the flexible image frame to deform in space. The deformation brings the flexible image frame to "wrap" onto the surface of the imaged 3D object. Surface reconstruction is thus achieved through a "package wrapping" or a "shape deformation" process by minimizing the discrepancy in the observable and the model prediction. The dynamics of such a wrapping process are governed by the least action principle which is physically correct. A physically based model is essential in this general shape reconstruction framework because of its capability to recover the desired 3D shape, to provide an animation sequence of the reconstruction, and to include the regularization principle into the theory of surface reconstruction.

Surface modeling using deformable templates

“…3, 300-310 (1991) 307 shape from structured lighting, is implemented and reported in this article. Currently, we are experimenting using other shape-from techniques to supply the image constraints in a dynamic visual reconstruction process [34]. Also, we intend to use a more structured representation, namely the B-spline patch [35], as our surface data structure.…”