Procedural Modeling of a Building from a Single Image

Abstract

Figure 1: Procedural modeling generation from a single image. 1) Given an image and a silhouette of a building, 2) our approach estimates the camera parameters and building mass as a first step.Then, 3) the façade image is rectified, and 4) the façade structure and 5) window styles are recognized. 6) Finally the output grammar is constructed and a corresponding 3d geometry is generated.Figure 1: Procedural Modeling from a Single Image. a) Given an image and a silhouette of a building, b) our approach automatically estimates the camera parameters and generates a building mass grammar as a first step. Then, c) the façade image is rectified, and d) the façade grammar is generated. e) For each window non-terminal, the best window grammar is selected by maximum vote. f) Finally the output grammar is constructed and a corresponding 3D geometry is generated. AbstractCreating a virtual city is demanded for computer games, movies, and urban planning, but it takes a lot of time to create numerous 3D building models. Procedural modeling has become popular in recent years to overcome this issue, but creating a grammar to get a desired output is difficult and time consuming even for expert users. In this paper, we present an interactive tool that allows users to automatically generate such a grammar from a single image of a building. The user selects a photograph and highlights the silhouette of the target building as input to our method. Our pipeline automatically generates the building components, from large-scale building mass to fine-scale windows and doors geometry. Each stage of our pipeline combines convolutional neural networks (CNNs) and optimization to select and parameterize procedural grammars that reproduce the building elements of the picture. In the first stage, our method jointly estimates camera parameters and building mass shape. Once known, the building mass enables the rectification of the façades, which are given as input to the second stage that recovers the façade layout. This layout allows us to extract individual windows and doors that are subsequently fed to the last stage of the pipeline that selects procedural grammars for windows and doors. Finally, the grammars are combined to generate a complete procedural building as output. We devise a common methodology to make each stage of this pipeline tractable. This methodology consists in simplifying the input image to match the visual appearance of synthetic training data, and in using optimization to refine the parameters estimated by CNNs. We used our method to generate a variety of procedural models of buildings from existing photographs.

Interactive modeling of lofted shapes from a single image

“…Other methods restrict this problem to a specific class of objects with strong in-class similarities: Refs. [19][20][21] considered architecture, Ref. [22] focused on garments, while Ref.…”

Nostalgin

“…Due to the recent popularity of deep learning, a number of publications have proposed deep models to learn automatic reconstruction. Recent work has improved on extracting intrinsic camera parameters and object poses [7], semantically parsing facades [17], or combining machine learning techniques with procedural grammars for reconstruction [19]. We note that many approaches to deep 3D reconstruction are not scalable and are very difficult to train outside of academic datasets (e.g.…”

“…Recent advances in computer vision have enabled the automatic recovery and extraction of missing information from images. Computers have gained the ability to semantically parse [8], rectify [28], and inpaint [26] images, and extract 3D scene understanding [19] from images. Research has been done to extract city geometries from images as well [18].…”

“…These limitations carry over to 3D city generators. For example, the work of [19] requires a color image with few occlusions and a usergenerated trace of the building model to create a single building. These limitations are not scalable and are unsuited for historical data, which is sparse and has few image guarantees.…”

Saliency-Guided Repetition Detection From Facade Point Clouds

“…Accurate detection of 3D facade elements has become highly important for urban building modeling because the reconstructed models have been widely used for many important applications, such as virtual tourism, urban planning, and entertainment. There is an extensive literature on repetitive structure detection methods, which range from image-based methods [1]- [3] to 3D point-based methods [4]- [6]. Due to the loss of three-dimensional information in two-dimensional imaging and the inevitable influences of illumination, reflections and occlusions, detecting repetitive structures from images remains difficult.…”