General 3D Room Layout from a Single View by Render-and-Compare

Stekovic, Sinisa; Hampali, Shreyas; Rad, Mahdi; Sarkar, Supriya; Fraundorfer, Friedrich; Lepetit, Vincent

doi:10.1007/978-3-030-58517-4_12

Cited by 18 publications

(32 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When performing layout estimation from point clouds as input data [43,6,20,33,32], one has to deal with incomplete and noisy scans as can be found in the ScanNet dataset [14]. Like previous work [33,49], we first hypothesize layout component proposals, but relying on MCTS for optimization lets us deal with a large number of proposals and be robust to noise and missing data, without special constraints like the Manhattan assumption.…”

Section: Layout Estimationmentioning

confidence: 99%

“…For the layout component proposals, we use the semantic segmentation by MinkowskiNet to extract the 3D points on the layout from the point cloud and rely on a simple RANSAC procedure to fit 3D planes. Like previous works [33,34,61,49], we compute the intersections between these planes to obtain 3D polygons, which we use as layout proposals. We also include the planes of the point cloud's 3D bounding box faces to handle incomplete scans: for example, long corridors are never scanned completely in ScanNet.…”

Section: Generating Proposalsmentioning

confidence: 99%

“…We further adjust this criterion: if multiple predicted corners are within this radius, a single corner that is closest to the ground truth is taken and a predicted corner can be assigned to only one ground truth corner. We also compute the polygons' Intersection-Over-Union (IOU) metric from [49] after projecting the retrieved polygons to their ground truth polygons. Table 1 compares the layouts retrieved by our approach to the SceneCAD annotations.…”

Section: Layoutsmentioning

confidence: 99%

See 2 more Smart Citations

Monte Carlo Scene Search for 3D Scene Understanding

Hampali

Stekovic

Sarkar

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

Section: Layout Estimationmentioning

confidence: 99%

Section: Generating Proposalsmentioning

confidence: 99%

Section: Layoutsmentioning

confidence: 99%

See 1 more Smart Citation

Monte Carlo Scene Search for 3D Scene Understanding

Hampali

Stekovic

Sarkar

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

“…We focus here on the creation of a structured floor plan where each room of an indoor environment is represented as a polygon with one edge per wall. Many types of input have been considered: Monocular perspective color views [18,19,22,31], panoramic views [32,38,40], depth…”

Section: Introductionmentioning

confidence: 99%

MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans

Stekovic

Rad

Fraundorfer

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

We propose a novel method for reconstructing floor plans from noisy 3D point clouds. Our main contribution is a principled approach that relies on the Monte Carlo Tree Search (MCTS) algorithm to maximize a suitable objective function efficiently despite the complexity of the problem. Like previous work, we first project the input point cloud to a top view to create a density map and extract room proposals from it. Our method selects and optimizes the polygonal shapes of these room proposals jointly to fit the density map and outputs an accurate vectorized floor map even for large complex scenes. To do this, we adapt MCTS, an algorithm originally designed to learn to play games, to select the room proposals by maximizing an objective function combining the fitness with the density map as predicted by a deep network and regularizing terms on the room shapes. We also introduce a refinement step to MCTS that adjusts the shape of the room proposals. For this step, we propose a novel differentiable method for rendering the polygonal shapes of these proposals. We evaluate our method on the recent and challenging Structured3D and Floor-SP datasets and show a significant improvement over the state-of-theart, without imposing any hard constraints nor assumptions on the floor plan configurations.

show abstract

“…faces meet (e.g. wall-wall-floor corner) [16,36], segmentation masks of room surfaces [15,2,25] or depth maps of dominant planes [32,27]. In a similar manner, we propose to use room boundaries, corners, and surface labels as features to learn.…”

Section: Introductionmentioning

confidence: 99%

RoomStructNet: Learning to Rank Non-Cuboidal Room Layouts From Single View

Zhang,

Wang,

Deng

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we present a new approach to estimate the layout of a room from its single image. While recent approaches for this task use robust features learnt from data, they resort to optimization for detecting the final layout. In addition to using learnt robust features, our approach learns an additional ranking function to estimate the final layout instead of using optimization. To learn this ranking function, we propose a framework to train a CNN using max-margin structure cost. Also, while most approaches aim at detecting cuboidal layouts, our approach detects non-cuboidal layouts for which we explicitly estimates layout complexity parameters. We use these parameters to propose layout candidates in a novel way. Our approach shows state-of-the-art results on standard datasets with mostly cuboidal layouts and also performs well on a dataset containing rooms with non-cuboidal layouts.

show abstract

General 3D Room Layout from a Single View by Render-and-Compare

Cited by 18 publications

References 32 publications

Monte Carlo Scene Search for 3D Scene Understanding

Monte Carlo Scene Search for 3D Scene Understanding

MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans

RoomStructNet: Learning to Rank Non-Cuboidal Room Layouts From Single View

Contact Info

Product

Resources

About