Reasoning on Crowd-Sourced Semantic Annotations to Facilitate Cataloguing of 3D Artefacts in the Cultural Heritage Domain

Abstract

The 3D Semantic Annotation (3DSA) system expedites the classification of 3D digital surrogates from the cultural heritage domain, by leveraging crowd-sourced semantic annotations. More specifically, the 3DSA system generates high-level classifications of 3D objects by applying rule-based reasoning across community-generated annotations and low-level shape and size attributes. This paper describes a particular use of the 3DSA system -cataloguing Greek pottery. It also describes our novel approach to rule-based reasoning that is modelled on concepts inspired from Markov logic networks. Our evaluation of this approach demonstrates its efficiency, accuracy and versatility, compared to classical rule-based reasoning.

Semantic Annotations on Heritage Models: 2d/3d Approaches and Future Research Challenges

“…The 3D Semantic Annotation System (3DSA) presented by Yu et al (2013) supports rule-based reasoning across generated annotations to catalogue heritage artifacts. Scalas et al (2020), in the context of the GRAVITATE project, build a 3D semantic annotation system aimed at the virtual assembly of fragmented or broken heritage artifacts.…”

X3D Fragment Identifiers — Extending the Open Annotation Model to Support Semantic Annotation of 3D Cultural Heritage Objects over the Web

“…Other advanced annotation capabilities supported by 3DSA include: the ability to display and annotate more than one 3D object, which enables users to document comparisons between multiple artefacts [7]. It also enables rule-based reasoning across annotations to infer higher level labels that help streamline the classification of 3D museum artefacts and support both keyword and semantic searches [6]. The adoption of WebGL (now a component of HTML5) within the annotation client, means there is no need for users to install any additional 3D rendering software or browser plug-in to display interactive 3D PLY (Polygon File Format) objects through the Browser.…”

“…Users can also specify the type of fragment (vase part) and its attributes (size, shape, curvature, color etc.) by using pull-down menus that display terms from the GVO [6]. Once the user is satisfied with the content in the annotation fields, they can save the annotation.…”

“…The 3DSA system supports three types of search: full-text search, semantic search and spatial search. The full-text search and semantic search capabilities that have been described previously [5][6][7] enable users to search across both institutional metadata or crowd-sourced Figure 6. Example of a vase handle being selected and annotated annotations by entering either free-text or choosing values from pulldown menus to search across specific fields.…”

“…After texture mapping, the medium-resolution PLY file was converted to X3D using the MeshLab software. All of these objects were then ingested into the 3DSA (3D Semantic Annotation) repository which is accessible via the 3DSA Web Portal [5][6][7]. The technical details of the 3DSA framework are described in Section 4.…”