Comparing Machine and Deep Learning Methods for Large 3D Heritage Semantic Segmentation

Matrone, Francesca; Grilli, Eleonora; Martini, Massimo; Paolanti, Marina; Pierdicca, Roberto; Remondino, Fabio

doi:10.3390/ijgi9090535

Cited by 105 publications

(91 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work is based on the tests carried out in (Matrone et al, 2020a), specifically the experiment of the Trompone's scene is reproduced using only the coordinates as a feature. Figure 2 shows this particular scene, representing original features and the relative ground truth.…”

Section: Arch Datasetmentioning

confidence: 99%

“…Then, every single object has been manually divided. ArCH dataset provides the ground truth for 10 classes, but only the classes that are more difficult to recognize has been selected, as indicated in (Matrone et al, 2020a). For this reason, only the objects regarding Column and Window classes have extrapolated from all scenes.…”

Section: Arch Datasetmentioning

confidence: 99%

“…Moreover, to compare the best performances we train a novel Deep Neural Network, namely DGCNN-Mod that classifies the synthetically generate scenes (Pierdicca et al, 2020). The experiments have been performed to ArCH dataset described in (Matrone et al, 2020a). In contrast to many existing datasets, it has been manually labelled by domain experts, thus providing a more precise dataset.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generative Networks for Point Cloud Generation in Cultural Heritage Domain

Martini

Pierdicca

Paolanti

et al. 2021

Proceedings ARQUEOLÓGICA 2.0 - 9th International Congress &Amp; 3rd GEORES - GEOmatics and pREServation

Self Cite

View full text Add to dashboard Cite

In the Cultural Heritage (CH) domain, the semantic segmentation of 3D point clouds with Deep Learning (DL) techniques allows to recognize historical architectural elements, at a suitable level of detail, and hence expedite the process of modelling historical buildings for the development of BIM models from survey data. However, it is more difficult to collect a balanced dataset of labelled architectural elements for training a network. In fact, the CH objects are unique, and it is challenging for the network to recognize this kind of data. In recent years, Generative Networks have proven to be proper for generating new data. Starting from such premises, in this paper Generative Networks have been used for augmenting a CH dataset. In particular, the performances of three state-of-art Generative Networks such as PointGrow, PointFLow and PointGMM have been compared in terms of Jensen-Shannon Divergence (JSD), the Minimum Matching Distance-Chamfer Distance (MMD-CD) and the Minimum Matching Distance-Earth Mover’s Distance (MMD-EMD). The objects generated have been used for augmenting two classes of ArCH dataset, which are columns and windows. Then a DGCNN-Mod network was trained and tested for the semantic segmentation task, comparing the performance in the case of the ArCH dataset without and with augmentation.

show abstract

Section: Arch Datasetmentioning

confidence: 99%

Section: Arch Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generative Networks for Point Cloud Generation in Cultural Heritage Domain

Martini

Pierdicca

Paolanti

et al. 2021

Proceedings ARQUEOLÓGICA 2.0 - 9th International Congress &Amp; 3rd GEORES - GEOmatics and pREServation

Self Cite

View full text Add to dashboard Cite

show abstract

“…We do not know if any transformations place by the Zumbo family, following the concession of 1465. Some aspects were been processed from the clouds obtained with TLS active sensors, including automatic decimation, without losing significant information, segmentation attempts and extraction of profiles with automatic procedures (Matrone et al, 2020).…”

Section: The Versuraementioning

confidence: 99%

The Taormina Theater: The Digital Survey System of Knowledge Open in Time

Gregorio

2021

Proceedings ARQUEOLÓGICA 2.0 - 9th International Congress &Amp; 3rd GEORES - GEOmatics and pREServation

View full text Add to dashboard Cite

In Sicily there are 19 show venues including ancient theaters and theatrical architectures. Many of these structures are fully functional and subject to visitor flows such as the theater of Syracuse and that of Taormina. They are object of interest and curiosity, revealed in the eighteenth century during the grand tour by travelers and landscape painters, in the last twenty years they have become reasons for study in various scientific areas as from acoustics to archeology, always passing through digital surveying. Studied through classical photogrammetry, structure from motion (SFM), 3D laser scanner, their representation as well as by increasingly refined and detailed two-dimensional graphics, makes use of 3D representations and techniques of virtual reality (VR) and augmented reality (AR). Due to their particular geometry, the need for studies and research is considered essential to deepen the methods of the surveys and plan their developments. Examples and problems for the archaeological survey are reported with the aim of critically evaluating the current state of the art of 3D survey, the potential and possible future developments, in the present study the results obtained for the survey of the Taormina theater (ME) and in-depth analysis of the versure environments.

show abstract

“…Due to the difficulties of finding a wide, enough labelled dataset to train a Neural Network for a DL approach and to ML better performance (Matrone et al, 2020), a supervised ML approach based on Grilli, Farella, Torresani & Remondino (2019) has been used to classify Milan Cathedral dataset. The authors train a Random Forest (RF) classifier manually annotating a small portion of the dataset together with geometric features designed for the architecture (Grilli, Farella, Torresani & Remondino, 2019).…”

Section: Preliminary Operation: Point Cloud Segmentationmentioning

confidence: 99%

A Hybrid Model for the Reverse Engineering of the Milan Cathedral. Challenges and Lesson Learnt

Spettu

Teruggi

Canali

et al. 2021

Proceedings ARQUEOLÓGICA 2.0 - 9th International Congress &Amp; 3rd GEORES - GEOmatics and pREServation

View full text Add to dashboard Cite

Cultural Heritage (CH) 3D digitisation is getting increasing attention and importance. Advanced survey techniques provide as output a 3D point cloud, wholly and accurately describing even the most complex architectural geometry with a priori established accuracy. These 3D point models are generally used as the base for the realisation of 2D technical drawings and 3D advanced representations. During the last 12 years, the 3DSurveyGroup (3DSG, Politecnico di Milano) conduced an omni-comprehensive, multi-technique survey, obtaining the full point cloud of Milan Cathedral, from which were produced the 2D technical drawings and the 3D model of the Main Spire used by the Veneranda Fabbrica del Duomo di Milano (VF) to plan its periodic maintenance and inspection activities on the Cathedral. Using the survey product directly to plan VF activities would help to skip a long-lasting, uneconomical and manual process of 2D and 3D technical elaboration extraction. In order to do so, the unstructured point cloud data must be enriched with semantics, providing a hierarchical structure that can communicate with a powerful, flexible information system able to effectively manage both point clouds and 3D geometries as hybrid models. For this purpose, the point cloud was segmented using a machine-learning algorithm with multi-level multi-resolution (MLMR) approach in order to obtain a manageable, reliable and repeatable dataset. This reverse engineering process allowed to identify directly on the point cloud the main architectonic elements that are then re-organised in a logical structure inserted inside the informative system built inside the 3DExperience environment, developed by Dassault Systémes.

show abstract

Comparing Machine and Deep Learning Methods for Large 3D Heritage Semantic Segmentation

Cited by 105 publications

References 48 publications

Generative Networks for Point Cloud Generation in Cultural Heritage Domain

Generative Networks for Point Cloud Generation in Cultural Heritage Domain

The Taormina Theater: The Digital Survey System of Knowledge Open in Time

A Hybrid Model for the Reverse Engineering of the Milan Cathedral. Challenges and Lesson Learnt

Contact Info

Product

Resources

About