Multi-view stereo for large-scale scene reconstruction with MRF-based depth inference

Sun, Shang; Xu, Dan; Wu, Hao; Ying, Haocong; Mou, Yurui

doi:10.1016/j.cag.2022.06.009

Cited by 6 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the feature extraction and cost optimisation are performed in a data‐driven manner rather than using ad hoc approaches, the cost volume construction process is similar to that of the classical plane sweep algorithm (Collins, 1996). Since the development of MVSNet, more powerful feature extractors and cost regularisers have been proposed (Ding et al, 2021; Luo et al, 2020; Peng et al, 2022; Sun et al, 2022; Yu, Guo et al, 2021) that can yield more valid and reasonable depth hypotheses and estimations.…”

Section: Related Workmentioning

confidence: 99%

Adaptive region aggregation for multi‐view stereo matching using deformable convolutional networks

Hu,

Su,

Mao

et al. 2023

The Photogrammetric Record

View full text Add to dashboard Cite

Deep‐learning methods have demonstrated promising performance in multi‐view stereo (MVS) applications. However, it remains challenging to apply a geometrical prior on the adaptive matching windows to achieve efficient three‐dimensional reconstruction. To address this problem, this paper proposes a learnable adaptive region aggregation method based on deformable convolutional networks (DCNs), which is integrated into the feature extraction workflow for MVSNet method that uses coarse‐to‐fine structure. Following the conventional pipeline of MVSNet, a DCN is used to densely estimate and apply transformations in our feature extractor, which is composed of a deformable feature pyramid network (DFPN). Furthermore, we introduce a dedicated offset regulariser to promote the convergence of the learnable offsets of the DCN. The effectiveness of the proposed DFPN is validated through quantitative and qualitative evaluations on the BlendedMVS and Tanks and Temples benchmark datasets within a cross‐dataset evaluation setting.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive region aggregation for multi‐view stereo matching using deformable convolutional networks

Hu,

Su,

Mao

et al. 2023

The Photogrammetric Record

View full text Add to dashboard Cite

show abstract

“…Fusing depth maps into a point cloud or a volumetric representation of the object is a common approach for MVS-based 3D reconstruction, and has been widely used due to its efficiency and scalability [ 11 , 12 ]. Popular depth estimation methods such as semi-global matching [ 13 ], and PatchMatch-based algorithms [ 14 , 15 ] are relatively mature, but struggle to handle surfaces lacking reliable data for depth estimation, such as smoothness, low-texture, and homogeneous regions. To overcome this barrier, Schops, T., et al [ 12 ] introduced higher-level scene understanding constraints to facilitate the propagation of correct depth estimates between adjacent pixels.…”

Section: Related Workmentioning

confidence: 99%

Plane Fitting in 3D Reconstruction to Preserve Smooth Homogeneous Surfaces

Xu¹,

Woo

2022

Sensors

View full text Add to dashboard Cite

Photogrammetric techniques for weakly-textured surfaces without sufficient information about the R (red), G (green) and B (blue) primary colors of light are challenging. Considering that most urban or indoor object surfaces follow simple geometric shapes, a novel method for reconstructing smooth homogeneous planar surfaces based on MVS (Multi-View Stereo) is proposed. The idea behind it is to extract enough features for the image description, and to refine the dense points generated by the depth values of pixels with plane fitting, to favor the alignment of the surface to the detected planes. The SIFT (Scale Invariant Feature Transform) and AKAZE (Accelerated-KAZE) feature extraction algorithms are combined to ensure robustness and help retrieve connections in small samples. The smoothness of the enclosed watertight Poisson surface can be enhanced by enforcing the 3D points to be projected onto the absolute planes detected by a RANSAC (Random Sample Consensus)-based approach. Experimental evaluations of both cloud-to-mesh comparisons in the per-vertex distances with the ground truth models and visual comparisons with a popular mesh filtering based post-processing method indicate that the proposed method can considerably retain the integrity and smoothness of the reconstruction results. Combined with other primitive fittings, the reconstruction extent of homogeneous surfaces can be further extended, serving as primitive models for 3D building reconstruction, and providing guidance for future works in photogrammetry and 3D surface reconstruction.

show abstract

Editorial note

Jorge¹

2022

Computers & Graphics

View full text Add to dashboard Cite

Multi-view stereo for large-scale scene reconstruction with MRF-based depth inference

Cited by 6 publications

References 36 publications

Adaptive region aggregation for multi‐view stereo matching using deformable convolutional networks

Adaptive region aggregation for multi‐view stereo matching using deformable convolutional networks

Plane Fitting in 3D Reconstruction to Preserve Smooth Homogeneous Surfaces

Editorial note

Contact Info

Product

Resources

About