Y-Shaped Convolutional Neural Network for 3d Roof Elements Extraction to Reconstruct Building Models From a Single Aerial Image

Abstract

Abstract. Fast and efficient detection and reconstruction of buildings have become essential in real-time applications such as navigation, 3D rendering, augmented reality, and 3D smart cities. In this study, a modern Deep Learning (DL)-based framework is proposed for automatic detection, localization, and height estimation of buildings, simultaneously, from a single aerial image. The proposed framework is based on a Y-shaped Convolutional Neural Network (Y-Net) which includes one encoder and two decoders. The input of the network is a single RGB image, while the outputs are predicted height information of buildings as well as the rooflines in three classes of eave, ridge, and hip lines. The extracted knowledge by the Y-Net (i.e. buildings’ heights and rooflines) is utilized for 3D reconstruction of buildings based on the third Level of Detail (LoD2). The main steps of the proposed approach are data preparation, CNNs training, and 3D reconstruction. For the experimental investigations airborne data from Potsdam are used, which were provided by ISPRS. For the predicted heights, the results show an average Root Mean Square Error (RMSE) and a Normalized Median Absolute Deviation (NMAD) of about 3.8 m and 1.3 m, respectively. Moreover, the overall accuracy of the extracted rooflines is about 86%.

Procedural Roof Generation From a Single Satellite Image

“…For [AAT19], it outputs 43.4% completeness and 4% correctness for just roof ridges (their completeness and correctness is higher when you also consider the building footprint pixels). In [AAH20], they improved results to 57.7% completeness and 81.3% correctness for roof ridges (using the same metrics as [AAT19]). At satellite‐level resolutions, the correctness and completeness term they provide does not seem suitable.…”

“…Ywata et al [YDSdO21] introduce a method to extract building roof boundaries in object space by integrating a high‐resolution aerial images stereo pair and three‐dimensional roof models reconstructed from LiDAR data. In [AAT19; AAH20], they work on a deep learning‐based approach to detect and reconstruct roof parts of buildings from a single image. However, they require high resolution aerial images and annotate the dataset for roof ridges and building boundaries.…”

Building Roof Vectorization With Ppgnet

“…Within the algorithm in (Alidoost et al, 2020), a Y-shaped CNN is applied to detect eave, ridge, and hip lines in a single aerial image. In a post-processing step outside the neural network, eave lines are used to subdivide building footprints into individual roof areas.…”

Intuitive and efficient roof modeling for reconstruction and synthesis

“…Bauchet and Lafarge [2020] adopt a kinetic data structure to partition the 3D space into convex polyhedra from which the underlying surface mesh of the input point cloud can be extracted. Another category of papers use deep learning [Alidoost et al 2020;Yu et al 2021;Zeng et al 2018; to directly output the reconstruction of 3D roof structures. However, these methods do not propose a solution for enforcing geometric constraints, while in our work, we mainly focus on enforcing geometric constraints in an optimization-based formulation during interactive roof reconstruction.…”