Implementing PointNet for point cloud segmentation in the heritage context

Abstract

Automated Heritage Building Information Modelling (HBIM) from the point cloud data has been researched in the last decade as HBIM can be the integrated data model to bring together diverse sources of complex cultural content relating to heritage buildings. However, HBIM modelling from the scan data of heritage buildings is mainly manual and image processing techniques are insufficient for the segmentation of point cloud data to speed up and enhance the current workflow for HBIM modelling. Artificial Intelligence (AI) based deep learning methods such as PointNet are introduced in the literature for point cloud segmentation. Yet, their use is mainly for manufactured and clear geometric shapes and components. To what extent PointNet based segmentation is applicable for heritage buildings and how PointNet can be used for point cloud segmentation with the best possible accuracy (ACC) are tested and analysed in this paper. In this study, classification and segmentation processes are performed on the 3D point cloud data of heritage buildings in Gaziantep, Turkey. Accordingly, it proposes a novel approach of activity workflow for point cloud segmentation with deep learning using PointNet for the heritage buildings. Twenty-eight case study heritage buildings are used, and AI training is performed using five feature labelling for segmentation namely, walls, roofs, floors, doors, and windows for each of these 28 heritage buildings. The dataset is divided into clusters with 80% training dataset and 20% prediction test dataset. PointNet algorithm was unable to provide sufficient accuracy in segmenting the point clouds due to deformation and deterioration on the existing conditions of the heritage case study buildings. However, if PointNet algorithm is trained with the restitution-based heritage data, which is called synthetic data in the research, PointNet algorithm provides high accuracy. Thus, the proposed approach can build the baseline for the accurate classification and segmentation of the heritage buildings.

Deep Learning Models to Predict Finishing Pig Weight Using Point Clouds

“…PointNet is a recently proposed algorithm that directly takes points for training [ 27 , 28 ]. PointNet has shown great potential in 3D classification and segmentation [ 29 , 30 ], but it has never been explored for livestock weight prediction. A significant advantage is that it directly takes a set of points as input and extracts features of the point clouds.…”

Automatic completion of geometric models from point clouds for analyzing historic timber roof structures

“…Semantic segmentation of point clouds using different approaches has been proposed in many studies (Poux and Billen, 2019;Grilli and Remondino, 2020;Haznedar et al, 2023;Song et al, 2023). Yang et al (2023) presented a review of the methods used for semantic segmentation of point clouds for cultural heritage; this classification has lower importance in the present study because beams represent the majority of the data, while the roof cover and floor can be easily filtered.…”

Deep learning based approaches from semantic point clouds to semantic BIM models for heritage digital twin

“…Researchers [22,23,27,[29][30][31][32][33] utilized historical building datasets for classification and segmentation to identify historical architectural elements, and even recognize the styles [34]. Point clouds automatic semantic segmentation is also an important step towards BIM models [17].…”

“…The paper [30] employed DGCNN [35] to segment ArCH datasets, which are churches, chapels, cloisters, porticoes and loggias into 11 different architectural parts. The authors [31] implemented PointNet [36] to segment 3D point cloud data of heritage buildings in Gaziantep, Turkey. Scholars [32] segmented the buildings' bricks based on images convolutional neural networks (CNN) for Basilica of St Anthony, Italy.…”