High-quality single-shot capture of facial geometry

Abstract

This technical report describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are sub-millimeter accurate and commensurate with those from stateof-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography.Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.

Binocular Photometric Stereo

“…Nehab et al [14] use a laser-scanned shape to rectify the low frequency component of normals from photometric stereo, and combine the rectified normals with the laser-scanned shape to solve for an optimized reconstruction. Beeler et al [3] apply photometric constraint in a separate refinement step for facial geometry capture. These methods, as well as [9] [1], require an initially computed 3D shape, either from laser or stereo.…”

“…(2,3,8,9), the Normal Objective (NO) is linear with respect to filter entries. For filters with one nonzero entry this approximation is exact, but for general noncompact filters it is a convex combination of the depths corresponding to nonzero filter entries.…”

Deformable image alignment as a source of stereo correspondences on portraits

“…Passive methods on the other hand rely entirely on scene features and require a sufficiently textured surface. With high resolution cameras, the fine details of human skin can be exploited for impressive reconstructions [2]. The other line of classification, calibrated vs. uncalibrated, relates to whether knowledge of scene geometry is used.…”

“…The other line of classification, calibrated vs. uncalibrated, relates to whether knowledge of scene geometry is used. The vast majority of stereo approaches, generic ones such as PMVS [7] as well as specialized methods for the face [2,4], use an extrinsically calibrated setup in order to restrict the search for correspondences to epipolar lines. Volumetric and hull based methods, too, rely on extrinsic camera parameters for ray computation.…”

Wide-Baseline Hair Capture Using Strand-Based Refinement

“…Multi-view stereo methods have been widely used to reconstruct real world objects with ever improving quality [7,1]. However, hair reconstruction remains one of the most challenging tasks due to many unique hair characteristics.…”