Semantic Annotations on Heritage Models: 2d/3d Approaches and Future Research Challenges

Abstract

Abstract. Research in the field of Cultural Heritage is increasingly moving towards the creation of digital information systems, in which the geometric representation of an artifact is linked to some external information, through meaningful tags. The process of attributing additional and structured information to various elements in a given digital model is customarily identified with the term semantic annotation; the added contextual information is associated, for instance, to analysis and conservation terms. Starting from the existing literature, aim of this work is to discuss how semantic annotations are used, in digital architectural heritage models, to link the geometrical representation of an artefact with knowledge-related information. Most consolidated methods -such as traditional mapping on 2D media, are compared with more recent approaches making the most of 3D representation. Reference is made, in particular, to Heritage-BIM techniques and to collaborative reality-based platforms, such as Aïoli (http://aioli.cloud). Potentialities and limits of the different solutions proposed in literature are critically discussed, also addressing future research challenges in Cultural Heritage application fields.

Connecting Geometry and Semantics via Artificial Intelligence: From 3d Classification of Heritage Data to H-Bim Representations

“…Then, by recognizing ideal geometries from these segmented data and by reconstructing each class in a parametric environment, a conceptual representation is derived. This representation can be used as an information system where to store knowledge-related data: the latter may be graphically associated to the whole class of elements, or to the single elements, or to parts of them, as expressed in (Croce et al, 2020).…”

“…In recent years, the field of architectural heritage has benefited from an increasing use of digital information models, such as Heritage-Building Information Modeling (H-BIM) systems, which have enabled the exploitation of digital replicas as integrated tools to archive, retrieve and disseminate the knowledge related to the documentation and preservation of a heritage asset (Croce et al, 2019). In this context, it has become increasingly essential to implement and characterize the different digital 3D surveying outputs, as obtained from laser scanning or photogrammetry (Bevilacqua et al, 2018), in order to make them more easily interpretable and shareable, in terms of both inclusion and visualization of the information related, e.g., to the results of material analysis, to the level of degradation of surfaces, to the state of conservation or to the architectural components represented (Croce et al, 2020). Hence, the correct construction of an information model for the handling of digital heritage data should follow the consecutive phases of: data acquisition; semantic segmentation -i.e.…”

Propagation of semantic information between orthophoto and 3D replica: a H-BIM system for the north transept of Pisa Cathedral

“…The connection of the merely visual and metric representation with structured knowledge-related information is performed thanks to the mechanism of the semantic annotation. The latter can be inserted on the digital media through manual selection techniques, or even through semi-automated or automated procedures (Croce et al 2020;Scalas et al 2020). The associated level of information can result more or less structured, depending on whether the annotation is performed, respectively, by tags, attributes, relations and ontologies (Andrews et al 2012).…”

“…In (Croce et al 2020), the different semantic annotation pipelines are also classified on the basis of the type of digital media where annotations are performed. A distinction can thus be made between: (a) 2D approaches, (b) 3D approaches, and (c) hybrid 2D/3D approaches, respectively.…”

Combining Multiband Imaging, Photogrammetric Techniques, and FOSS GIS for Affordable Degradation Mapping of Stone Monuments

“…An essential issue in semantic annotation of thematically mapped historical surfaces is the automation level [31]. Digital image processing (DIP)-based approaches have been implemented to automatize the mapping process, offering a time-efficient alternative to manual mapping procedures and supporting a more detailed, quantitative, and comprehensive evaluation of the state of stone preservation [32][33][34].…”