A Hybrid Approach to Reassemble Ancient Decorated Block Fragments through a 3D Puzzling Engine

Abstract

The reassembling of severely damaged tangible heritage is a primordial task for archaeologists who not only aim to further study the past but also to preserve ruined ancient monuments. As a consequence, various researchers have proposed methods to automatically solve this problem by computing and matching geometric properties of counterpart fragments. Although their results are quite promising, experts still carry out this task manually by finding relationships between distinctive matching cues, such as type of decoration, remaining traces, inscriptions’ content, etc. The topic itself poses challenges to both automatic and manual approaches due to the high level of damage ancient broken fragments have undergone over the centuries. Therefore, this paper proposes a Puzzling Engine that combines crucial elements of automatic and manual methodologies to empower experts with registration tools for reassembling fragmented heritage. Unlike similar hybrid human-computer puzzling engines, our approach is capable of automatically proposing matches and rough alignments solely based on the geometry of fractured surfaces. Based on these initial solutions and a set of registration tools, experts can accurately solve the puzzle. The virtual environment has been used and verified to find pairwise puzzle-pieces of actual antique wall decorated fragments, resulting in new discoveries that experts could not have come up with by utilizing classic techniques. Concretely, the contributions are twofold, (i) a feature-based registration pipeline that is able to suggest both matches and alignments to the user and (ii) a virtual interface that integrates automatic and user-assisted techniques to accurately puzzle fragmented surfaces.

Texture-based Clustering of Archaeological Textile Images

“…While a broad range of machine learning literature has addressed puzzle solving problem in the artwork imagery [2][3][4], as well as for 3D archaeological artefacts [5][6][7], to the best of our knowledge none of them proposed a methodology that is robust enough for solving the puzzle of highly degraded, irregularly shaped artefacts that come from unknown number of originals. We believe that identification of the number of originals and grouping the fragments potentially from the same piece of artwork is the first and vital step toward the puzzle solving goal.…”

Computational techniques for virtual reconstruction of fragmented archaeological textiles

“…The availability of the ground truth images gives the authors possibility to evaluate the performance in a reliable manner. On the other hand, the real puzzle problems with already fragmented objects and no ground truth exist in the cultural heritage domain [45][46][47][48][49][50][51] or when dealing with shredded documents, as in [41]. Unlike natural images that are fragmented digitally, digitization of these real-world fragments is needed, which itself adds a complexity to the problem [49,51,52].…”

“…De Lima-Hernandez et al proposed several interesting approaches to puzzling damaged 3D archaeological artifacts [45][46][47]. The 3D Puzzling Engine [45,46] registers the 3D ancient wall-decorated fragments based on surface normal coherence, and offers elements of fullautomation, as well as user-input-based semi-supervised solutions.…”

“…De Lima-Hernandez et al proposed several interesting approaches to puzzling damaged 3D archaeological artifacts [45][46][47]. The 3D Puzzling Engine [45,46] registers the 3D ancient wall-decorated fragments based on surface normal coherence, and offers elements of fullautomation, as well as user-input-based semi-supervised solutions. In their recent study, de Lima-Hernandez et al [47] propose using a Generative Adversarial Network (GAN) to predict the missing decoration traces on broken heritage fragments, extending the texture information of the fragments and allowing for a more accurate estimation of fragment alignment.…”