Learning to Reconstruct 3D Manhattan Wireframes From a Single Image

Zhou, Yichao; Qi, Haozhi; Zhai, Yuexiang; Sun, Qi; Chen, Zhili; Wei, Li-Yi; Ma, Yi

doi:10.1109/iccv.2019.00779

Cited by 58 publications

(44 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, if a long line is broken into several short line segments, the resulted heat map is almost the same as the ground truth heat map, as shown in Figure 4. A good performance on the above two properties is vital for downstream tasks that rely on the correctness of line connectivity, such as inferring the 3D geometry through lines [24,36].…”

Section: Precision and Recall Of Line Heat Mapsmentioning

confidence: 99%

End-to-End Wireframe Parsing

Zhou

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

158

213

View full text Add to dashboard Cite

We present a conceptually simple yet effective algorithm to detect wireframes [14] in a given image. Compared to the previous methods [14,33] which first predict an intermediate heat map and then extract straight lines with heuristic algorithms, our method is end-to-end trainable and can directly output a vectorized wireframe that contains semantically meaningful and geometrically salient junctions and lines. To better understand the quality of the outputs, we propose a new metric for wireframe evaluation that penalizes overlapped line segments and incorrect line connectivities. We conduct extensive experiments and show that our method significantly outperforms the previous state-of-the-art wireframe and line extraction algorithms [14,33,32]. We hope our simple approach can be served as a baseline for future wireframe parsing studies. Code has been made publicly available at https://github.com/zhou13/lcnn.

show abstract

Section: Precision and Recall Of Line Heat Mapsmentioning

confidence: 99%

End-to-End Wireframe Parsing

Zhou

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

158

213

View full text Add to dashboard Cite

show abstract

“…Deep learning-based line segment detection methods has attracted great attention due to the remarkable performances [11,12,33,38,39]. AFM [32] presented regional partition maps and attraction field maps of line segment maps, followed by a squeeze module to generate line segments.…”

Section: Line Segment Detectionmentioning

confidence: 99%

ELSD: Efficient Line Segment Detector and Descriptor

Zhang¹,

Luo²,

Qin

et al. 2021

Preprint

View full text Add to dashboard Cite

We present the novel Efficient Line Segment Detector and Descriptor (ELSD) to simultaneously detect line segments and extract their descriptors in an image. Unlike the traditional pipelines that conduct detection and description separately, ELSD utilizes a shared feature extractor for both detection and description, to provide the essential line features to the higher-level tasks like SLAM and image matching in real time. First, we design the one-stage compact model, and propose to use the mid-point, angle and length as the minimal representation of line segment, which also guarantees the center-symmetry. The non-centerness suppression is proposed to filter out the fragmented line segments caused by lines' intersections. The fine offset prediction is designed to refine the mid-point localization. Second, the line descriptor branch is integrated with the detector branch, and the two branches are jointly trained in an end-to-end manner. In the experiments, the proposed ELSD achieves the state-of-the-art performance on the Wireframe dataset and YorkUrban dataset, in both accuracy and efficiency. The line description ability of ELSD also outperforms the previous works on the line matching task.

show abstract

“…We refer the interested reader to the recent study by Gryaditskaya et al [2019], who showed that a state-of-the-art deep network trained on synthetic drawings get confused by the many hidden and construction lines present in the sketches we target. The problem of parsing and reconstructing 3D wireframes has also been considered in computer vision, taking as input one or multiple photographs [Huang et al 2018;Usumezbas et al 2016;Zhou et al 2019], but these methods do not face the challenge of dealing with occlusion between lines since photographs only capture visible surfaces.…”

Section: Inputmentioning

confidence: 99%

Lifting freehand concept sketches into 3D

et al. 2020

View full text Add to dashboard Cite

We present the first algorithm capable of automatically lifting real-world, vector-format, industrial design sketches into 3D. Targeting real-world sketches raises numerous challenges due to inaccuracies, use of overdrawn strokes, and construction lines. In particular, while construction lines convey important 3D information, they add significant clutter and introduce multiple accidental 2D intersections. Our algorithm exploits the geometric cues provided by the construction lines and lifts them to 3D by computing their intended 3D intersections and depths. Once lifted to 3D, these lines provide valuable geometric constraints that we leverage to infer the 3D shape of other artist drawn strokes. The core challenge we address is inferring the 3D connectivity of construction and other lines from their 2D projections by separating 2D intersections into 3D intersections and accidental occlusions. We efficiently address this complex combinatorial problem using a dedicated search algorithm that leverages observations about designer drawing pREFERENCES, and uses those to explore only the most likely solutions of the 3D intersection detection problem. We demonstrate that our separator outputs are of comparable quality to human annotations, and that the 3D structures we recover enable a range of design editing and visualization applications, including novel view synthesis and 3D-aware scaling of the depicted shape.

show abstract

Learning to Reconstruct 3D Manhattan Wireframes From a Single Image

Cited by 58 publications

References 32 publications

End-to-End Wireframe Parsing

End-to-End Wireframe Parsing

ELSD: Efficient Line Segment Detector and Descriptor

Lifting freehand concept sketches into 3D

Contact Info

Product

Resources

About