High Accuracy and Visibility-Consistent Dense Multiview Stereo

Abstract

Since the initial comparison of Seitz et al., the accuracy of dense multiview stereovision methods has been increasing steadily. A number of limitations, however, make most of these methods not suitable to outdoor scenes taken under uncontrolled imaging conditions. The present work consists of a complete dense multiview stereo pipeline which circumvents these limitations, being able to handle large-scale scenes without sacrificing accuracy. Highly detailed reconstructions are produced within very reasonable time thanks to two key stages in our pipeline: a minimum s-t cut optimization over an adaptive domain that robustly and efficiently filters a quasidense point cloud from outliers and reconstructs an initial surface by integrating visibility constraints, followed by a mesh-based variational refinement that captures small details, smartly handling photo-consistency, regularization, and adaptive resolution. The pipeline has been tested over a wide range of scenes: from classic compact objects taken in a laboratory setting, to outdoor architectural scenes, landscapes, and cultural heritage sites. The accuracy of its reconstructions has also been measured on the dense multiview benchmark proposed by Strecha et al., showing the results to compare more than favorably with the current state-of-the-art methods.

Patchwork Stereo: Scalable, Structure-Aware 3D Reconstruction in Man-Made Environments

“…A number of top-performing general MVS algorithms assume a Delaunay tetrahedralization of an initial 3D point cloud, whose cells are labeled with a discrete occupancy state according to visibility and photometric constraints; the reconstructed surface lies at the interface between empty and non-empty cells [14,39]. Despite mesh refinement, the resulting surface remains a jagged approximation of a locally-smooth geometry, which may then require expensive post-processing to achieve a compact representation, e.g., by fitting 3D geometric primitives [19,15].…”

“…These improvements are achieved thanks to both our patch-based representation and our robust hypothesis extraction from already-available SfM data. The resulting mesh is compact, and aligned with scenes' structure and image gradients by design, which is achieved with no need of later 3D geometry simplification [29], nor additional complex mesh refinement [39], or tedious primitive fitting steps [19].…”

Accurate Multiview Stereo Reconstruction With Fast Visibility Integration and Tight Disparity Bounding

“…Other algorithms make use of polygon meshes to represent the surface as a set of planar polygons. This represen-tation can be used in the central part of a reconstruction pipeline since it is well-suited for visibility computation (Fua and Leclerc, 1995) or it can be computed in the final part, starting from a dense points cloud (Kazhdan et al, 2006, Hiep et al, 2009, Vu et al, 2012. Many modern algorithms employ different representation over their reconstruction pipeline.…”

“…The computation can be carried on the graphics processing unit (GPU) (Vu et al, 2012), knocking down the computing times. Multi-view stereo algorithms are well-suited for general purpose GPU programming (GPGPU), since many of their tasks can be executed in parallel.…”

Consistent 3D Background Model Estimation from Multi-viewpoint Videos

“…While undeniable progress has been made in the past few years in multi-view 3D reconstruction [32,36,41,17,33], the emphasis remains on achieving high accuracy, photorealism and unprecedented size in reconstructed models of static scenes. A much smaller branch of multi-view stereo addresses the modeling of dynamic scenes, in which people, animals, robots or other visible surfaces move.…”