Generating 3d City Models Based on the Semantic Segmentation of Lidar Data Using Convolutional Neural Networks

Agoub, Amgad; Schmidt, Valentina; Kada, Martin

doi:10.5194/isprs-annals-iv-4-w8-3-2019

Cited by 4 publications

(4 citation statements)

References 20 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the proposed method, the 3D reconstruction of buildings with a variety of shapes and complexity was achieved with root mean square error (RMSE) values of 3.43 m and 1.13 m for the predicted normalized DSM (nDSM), respectively. To generate block-like city models using depth maps, Agoub et al (2019) developed a pipeline based on multiple CNNs with an encoderdecoder architecture. A view of the reconstructed buildings of Manhattan area in their study is given in Figure 3.…”

Section: Convolutional Neural Networkmentioning

confidence: 99%

Deep Learning for 3d Building Reconstruction: A Review

Buyukdemircioglu

Kocaman

Kada³

2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

Self Cite

View full text Add to dashboard Cite

Abstract. 3D building reconstruction using Earth Observation (EO) data (aerial and satellite imagery, point clouds, etc.) is an important and active research topic in different fields, such as photogrammetry, remote sensing, computer vision and Geographic Information Systems (GIS). Nowadays 3D city models have become an essential part of 3D GIS environments and they can be used in many applications and analyses in urban areas. The conventional 3D building reconstruction methods depend heavily on the data quality and source; and manual efforts are still needed for generating the object models. Several tasks in photogrammetry and remote sensing have been revolutionized by using deep learning (DL) methods, such as image segmentation, classification, and 3D reconstruction. In this study, we provide a review on the state-of-the-art machine learning and in particular the DL methods for 3D building reconstruction for the purpose of city modelling using EO data. This is the first review with a focus on object model generation based on the DL methods and EO data. A brief overview of the recent building reconstruction studies with DL is also given. We have investigated the different DL architectures, such as convolutional neural networks (CNNs), generative adversarial networks (GANs), and the combinations of conventional approaches with DL in this paper and reported their advantages and disadvantages. An outlook on the future developments of 3D building modelling based on DL is also presented.

show abstract

Section: Convolutional Neural Networkmentioning

confidence: 99%

Deep Learning for 3d Building Reconstruction: A Review

Buyukdemircioglu

Kocaman

Kada³

2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…AI-based procedures were recently used to infer buildings' features and characteristics. Machine and deep learning methods were increasingly employed for predicting 3D urban geometries and semantics [29][30][31][32], for energy performances [33][34][35], for models generalisation [36], or to infer some missing information, such as buildings' age [37][38][39][40] and height [28,[41][42][43][44]. Prediction algorithms are generally trained using satellite or aerial images [43,44], LiDAR data [37,42], or 2D data (such as photographs, maps, footprints, and attributes) available from historical archives, cadastre datasets, or volunteered geographic information databases [28,38,39,45].…”

Section: Related Workmentioning

confidence: 99%

4D Building Reconstruction with Machine Learning and Historical Maps

2021

View full text Add to dashboard Cite

The increasing importance of three-dimensional (3D) city modelling is linked to these data’s different applications and advantages in many domains. Images and Light Detection and Ranging (LiDAR) data availability are now an evident and unavoidable prerequisite, not always verified for past scenarios. Indeed, historical maps are often the only source of information when dealing with historical scenarios or multi-temporal (4D) digital representations. The paper presents a methodology to derive 4D building models in the level of detail 1 (LoD1), inferring missing height information through machine learning techniques. The aim is to realise 4D LoD1 buildings for geospatial analyses and visualisation, valorising historical data, and urban studies. Several machine learning regression techniques are analysed and employed for deriving missing height data from digitised multi-temporal maps. The implemented method relies on geometric, neighbours, and categorical attributes for height prediction. Derived elevation data are then used for 4D building reconstructions, offering multi-temporal versions of the considered urban scenarios. Various evaluation metrics are also presented for tackling the common issue of lack of ground-truth information within historical data.

show abstract

“…The first neural network generates the 3D point cloud, e.g., by using the truncated signed distance function (TSDF) [ 16 , 17 ]. Then, a second network is used to identify and extract relevant, representative 3D object content [ 18 , 19 ]. However, there are approaches combining both aforementioned steps in one single neural network [ 20 ].…”

Section: Introductionmentioning

confidence: 99%

3D Visual Reconstruction as Prior Information for First Responder Localization and Visualization

Kaiser,

Linkiewicz,

Meißner

et al. 2023

Sensors

View full text Add to dashboard Cite

In professional use cases like police or fire brigade missions, coordinated and systematic force management is crucial for achieving operational success during intervention by the emergency personnel. A real-time situation picture enhances the coordination of the team. This situation picture includes not only an overview of the environment but also the positions, i.e., localization, of the emergency forces. The overview of the environment can be obtained either from known situation pictures like floorplans or by scanning the environment with the aid of visual sensors. The self-localization problem can be solved outdoors using the Global Navigation Satellite System (GNSS), but it is not fully solved indoors, where the GNSS signal might not be received or might be degraded. In this paper, we propose a novel combination of an inertial localization technique based on simultaneous localization and mapping (SLAM) with 3D building scans, which are used as prior information, for geo-referencing the positions, obtaining a situation picture, and finally visualizing the results with an appropriate visualization tool. We developed a new method for converting point clouds into a hexagonal prism map specifically designed for our SLAM algorithm. With this combination, we could keep the equipment for first responders as lightweight as required. We showed that the positioning led to an average accuracy of less than 1m indoors, and the final visualization including the building layout obtained by the 3D building reconstruction will be advantageous for coordinating first responder operations.

show abstract

Generating 3d City Models Based on the Semantic Segmentation of Lidar Data Using Convolutional Neural Networks

Cited by 4 publications

References 20 publications

Deep Learning for 3d Building Reconstruction: A Review

Deep Learning for 3d Building Reconstruction: A Review

4D Building Reconstruction with Machine Learning and Historical Maps

3D Visual Reconstruction as Prior Information for First Responder Localization and Visualization

Contact Info

Product

Resources

About