LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image

Zou, Chuhang; Colburn, Alex; Qi, Shihua; Hoiem, Derek

doi:10.1109/cvpr.2018.00219

Cited by 255 publications

(261 citation statements)

References 33 publications

Supporting

Mentioning

249

Contrasting

Order By: Relevance

“…Recently, there are several other works [22,9,24,21,18] related to room layouts, but they focus on a different problem, i.e., to reconstruct 3D room layouts from photos.…”

Section: Related Workmentioning

confidence: 99%

Deep Floor Plan Recognition Using a Multi-Task Network With Room-Boundary-Guided Attention

Zeng

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

101

View full text Add to dashboard Cite

This paper presents a new approach to recognize elements in floor plan layouts. Besides walls and rooms, we aim to recognize diverse floor plan elements, such as doors, windows and different types of rooms, in the floor layouts. To this end, we model a hierarchy of floor plan elements and design a deep multi-task neural network with two tasks: one to learn to predict room-boundary elements, and the other to predict rooms with types. More importantly, we formulate the room-boundary-guided attention mechanism in our spatial contextual module to carefully take room-boundary features into account to enhance the room-type predictions. Furthermore, we design a cross-and-within-task weighted loss to balance the multi-label tasks and prepare two new datasets for floor plan recognition. Experimental results demonstrate the superiority and effectiveness of our network over the state-of-the-art methods.

show abstract

“…Recently, there are several other works [22,9,24,21,18] related to room layouts, but they focus on a different problem, i.e., to reconstruct 3D room layouts from photos.…”

Section: Related Workmentioning

confidence: 99%

Deep Floor Plan Recognition Using a Multi-Task Network With Room-Boundary-Guided Attention

Zeng

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

101

View full text Add to dashboard Cite

show abstract

“…None of these methods predicts texture behind occlusion, which is subject of our approach. Other methods exploit more extended inputs to predict 3D scene representations, such as a panorama image [51], RGB-D [11] or a depth map [37,44].…”

Section: Related Workmentioning

confidence: 99%

“…On the other hand, having a fully completed 3D model of the scene is often an unnecessary complication, since most of the information present in such model would never be used if the novel vantage points are either nearby the original one and/or small in number. It is worth noting that generating such completed 3D scenes typically comes with high computational and memory cost [51,11,37,44].…”

Section: Introductionmentioning

confidence: 99%

Object-Driven Multi-Layer Scene Decomposition From a Single Image

Dhamo

Navab

Tombari

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

We present a method that tackles the challenge of predicting color and depth behind the visible content of an image. Our approach aims at building up a Layered Depth Image (LDI) from a single RGB input, which is an efficient representation that arranges the scene in layers, including originally occluded regions. Unlike previous work, we enable an adaptive scheme for the number of layers and incorporate semantic encoding for better hallucination of partly occluded objects. Additionally, our approach is object-driven, which especially boosts the accuracy for the occluded intermediate objects. The framework consists of two steps. First, we individually complete each object in terms of color and depth, while estimating the scene layout. Second, we rebuild the scene based on the regressed layers and enforce the recomposed image to resemble the structure of the original input. The learned representation enables various applications, such as 3D photography and diminished reality, all from a single RGB image. 1

show abstract

“…However, it remains challenging for vision algorithms to detect and utilize such global structures from local image features, until recent advances in deep learning which makes learning high-level features possible from labeled data. The examples include detecting planes [30,19], surfaces [10], 2D wireframes [13], room layouts [35], key points for mesh fitting [31,29], and sparse scene representations from multiple images [6].…”

Section: Introductionmentioning

confidence: 99%

Learning to Reconstruct 3D Manhattan Wireframes From a Single Image

Zhou

Zhai

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

In this paper, we propose a method to obtain a compact and accurate 3D wireframe representation from a single image by effectively exploiting global structural regularities. Our method trains a convolutional neural network to simultaneously detect salient junctions and straight lines, as well as predict their 3D depth and vanishing points. Compared with the state-of-the-art learning-based wireframe detection methods, our network is much simpler and more unified, leading to better 2D wireframe detection. With global structural priors such as Manhattan assumption, our method further reconstructs a full 3D wireframe model, a compact vector representation suitable for a variety of high-level vision tasks such as AR and CAD. We conduct extensive evaluations on a large synthetic dataset of urban scenes as well as real images. Our code and datasets will be released.

show abstract

LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image

Cited by 255 publications

References 33 publications

Deep Floor Plan Recognition Using a Multi-Task Network With Room-Boundary-Guided Attention

Deep Floor Plan Recognition Using a Multi-Task Network With Room-Boundary-Guided Attention

Object-Driven Multi-Layer Scene Decomposition From a Single Image

Learning to Reconstruct 3D Manhattan Wireframes From a Single Image

Contact Info

Product

Resources

About