A Performance Comparison Between Segnet and Deeplabv3+ on the Semantic Segmentation of Heritage Buildings

Abstract

Abstract. During the last decade, the use of machine and deep learning tools to support 3D semantic segmentation of point clouds remarkably increased and their impressive results have led to the application of such methods to the semantic modeling of heritage buildings. Nevertheless, a standard procedure to deal with such problem is still missing, and several significant challenges, caused by the complexity of heritage building scenario, have still to be faced. This paper aims at comparing the overall performance of two convolutional neural network architectures, named SegNet and Deeplabv3+, for the semantic segmentation of heritage point clouds throughout a multiview approach. More specifically, the two architectures have been tested to obtain 2D segmentation maps of the related photogrammetric images of the buildings, and then the output maps have been projected to the photogrammetric point cloud by means of the interior and exterior camera parameters. Experiments to test the effectiveness of the proposed approach have been conducted on the case study of Spedale del Ceppo in Pistoia, Italy. Despite the results shown a remarkable performance of both the architectures, Deeplabv3+ outperformed SegNet in terms of accuracy, memory consumption and training time.