3D Reconstruction of Incomplete Archaeological Objects Using a Generative Adversarial Network

Abstract

We introduce a data-driven approach to aid the repairing and conservation of archaeological objects: ORGAN, an object reconstruction generative adversarial network (GAN). By using an encoder-decoder 3D deep neural network on a GAN architecture, and combining two loss objectives: a completion loss and an Improved Wasserstein GAN loss, we can train a network to effectively predict the missing geometry of damaged objects. As archaeological objects can greatly differ between them, the network is conditioned on a variable, which can be a culture, a region or any metadata of the object. In our results, we show that our method can recover most of the information from damaged objects, even in cases where more than half of the voxels are missing, without producing many errors.

Findings from machine learning in clinical medical imaging applications – Lessons for translation to the forensic setting

“…In forensic anthropology, tasks that could be addressed using ML include: determination of skeletal completeness after accidents [ 96 ], e.g. plane crashes; 3D reconstruction of incomplete bones, that could be extrapolated from the work by Hermoza and Sipiran [ 97 ] on incomplete archaeological objects; and 3D reconstruction of fractured skulls [ [98] , [99] , [100] ], used to infer a cause of death, or to perform facial reconstruction.…”

When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs)

“…The network has learned multiple objects and internally performs the registration between the image and the conceptual model. In a similar way, Hermoza and Sipiran [39] also used a GAN network for predicting the missing geometry of damaged archaeological objects, indicating the reconstructed object in a voxel grid format and a label designating its class. Its network architecture combines a completion loss and an improved Wasserstein GAN loss.…”

“…The network outputs a transformed image and also a deformation field. Similarly, in three-dimensional space, Hermoza and Sipiran [39], as referenced above, performed the reconstruction of incomplete archaeological models also using GANs, in which the generator network provides a reconstructed model. Zhang et al [79] also proposed a registration method based on a GAN architecture with a gradient loss which can manage local structure information across modalities.…”

“…Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) for predicting missing geometry [39,50] • The use of Adversarial Training (AT) to predict structural deformations on 3D meshes given a multi-modal input (e.g., forces, material properties, visual imaging) [61] features and matching…”

Application of artificially intelligent systems for the identification of discrete fossiliferous levels

“…The pioneering introduction of Artificially Intelligent Algorithms (AIAs) in fields of archaeology and paleoanthropology has revolutionized numerous sub-disciplines such as those related with genetic sequencing (Mondal, Bertranpetit & Lao, 2019), site and object detection (Anemone, Emerson & Conroy, 2011;Conroy et al, 2012;Emerson et al, 2015;Benhabiles & Tabia, 2016;Block et al, 2016;Wills, Choiniere & Barrett, 2018;Anemone & Conroy, 2018;Caspari & Crespo, 2019;Verschoof-van der Vaart & Lambers, 2019), physical anthropology (Bewes et al, 2019), biomechanics (Püschel et al, 2018) restoration (Derech, Tal & Shimshoni, 2018;Hermoza & Sipiarn, 2018), as well as taphonomy (Arriaza & Domínguez-Rodrigo, 2016;Domínguez-Rodrigo, 2019;Egeland et al, 2018;Byeon et al, 2019;Courtenay et al, 2019;Moclán, Domínguez-Rodrigo & Yravedra, 2019).…”