Planar Surface Reconstruction from Sparse Views

Jin, Linyi; Qian, Shengyi; Owens, Andrew; Fouhey, David F.

doi:10.1109/iccv48922.2021.01275

Cited by 22 publications

(44 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, although being possible to learn 2D plane segmentation with a single image, it is still challenging to learn accurate 3D plane geometry only with single-view regression. Most recently, Jin et al [23] proposes a framework to jointly reconstruct planes and estimate camera poses from sparse views. In our work, we assume the camera poses are obtained from some SLAM systems, and design our plane detection branch based on PlaneRCNN [31], but learn plane geometry in a separate multi-view-stereo (MVS) branch.…”

Section: Related Workmentioning

confidence: 99%

PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo

Liu¹,

Pan²,

Bansal³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present a novel framework named PlaneMVS for 3D plane reconstruction from multiple input views with known camera poses. Most previous learning-based plane reconstruction methods reconstruct 3D planes from single images, which highly rely on single-view regression and suffer from depth scale ambiguity. In contrast, we reconstruct 3D planes with a multi-view-stereo (MVS) pipeline that takes advantage of multi-view geometry. We decouple plane reconstruction into a semantic plane detection branch and a plane MVS branch. The semantic plane detection branch is based on a single-view plane detection framework but with differences. The plane MVS branch adopts a set of slanted plane hypotheses to replace conventional depth hypotheses to perform plane sweeping strategy and finally learns pixel-level plane parameters and its planar depth map. We present how the two branches are learned in a balanced way, and propose a soft-pooling loss to associate the outputs of the two branches and make them benefit from each other. Extensive experiments on various indoor datasets show that PlaneMVS significantly outperforms state-of-theart (SOTA) single-view plane reconstruction methods on both plane detection and 3D geometry metrics. Our method even outperforms a set of SOTA learning-based MVS methods thanks to the learned plane priors. To the best of our knowledge, this is the first work on 3D plane reconstruction within an end-to-end MVS framework.

show abstract

Section: Related Workmentioning

confidence: 99%

PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo

Liu¹,

Pan²,

Bansal³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While some works relax the assumption [30] or avoid it via many images [21], these have not been demonstrated in the few-image, wide baseline case. Most work in the sparse view setting (e.g., [4,8]) does pose estimation but not reconstruction and works that produce reconstructions from sparse views [42,24] come with substantial limitations. Qian et.…”

Section: Introductionmentioning

confidence: 99%

“…Jin et. al [24] apply a complex hand-designed discrete/continuous optimization applied to plane segments found by an extended PlaneRCNN [34] output. This optimization includes bundle-adjustment on SIFT [36] on viewpoint-normalized texture like VIP [61].…”

Section: Introductionmentioning

confidence: 99%

“…We propose an approach ( §3), named the PlaneFormer, that overcomes these limitations. Following existing work in this area [42,24], we construct a scene reconstruction by merging scene elements that are visible in multiple views and estimating relative camera transformations. We build on [24] and construct a piecewise planar reconstruction from the images.…”

Section: Introductionmentioning

confidence: 99%

“…Following existing work in this area [42,24], we construct a scene reconstruction by merging scene elements that are visible in multiple views and estimating relative camera transformations. We build on [24] and construct a piecewise planar reconstruction from the images. However, rather than perform an optimization, we directly train a transformer that ingests the scene components as tokens.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

PlaneFormers: From Sparse View Planes to 3D Reconstruction

Agarwala¹,

Jin²,

Rockwell³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present an approach for the planar surface reconstruction of a scene from images with limited overlap. This reconstruction task is challenging since it requires jointly reasoning about single image 3D reconstruction, correspondence between images, and the relative camera pose between images. Past work has proposed optimization-based approaches. We introduce a simpler approach, the PlaneFormer, that uses a transformer applied to 3D-aware plane tokens to perform 3D reasoning. Our experiments show that our approach is substantially more effective than prior work, and that several 3D-specific design decisions are crucial for its success. Project page: https://samiragarwala.github.io/ PlaneFormers. PlaneFormer Extracted Planes Extracted Cameras Refined Cameras Refined PlanesFig. 1. Given a sparse set of images, our method detects planes and cameras, and produces plane correspondences and refined cameras using a Plane Transformer ), from which it can reconstruct the scene in 3D.

show abstract

SALVe: Semantic Alignment Verification for Floorplan Reconstruction from Sparse Panoramas

Lambert

Boyadzhiev

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Planar Surface Reconstruction from Sparse Views

Cited by 22 publications

References 54 publications

PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo

PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo

PlaneFormers: From Sparse View Planes to 3D Reconstruction

SALVe: Semantic Alignment Verification for Floorplan Reconstruction from Sparse Panoramas

Contact Info

Product

Resources

About