Multi-Scale Upsampling GAN Based Hole-Filling Framework for High-Quality 3D Cultural Heritage Artifacts

Abstract

With the rapid development of 3D scanners, the cultural heritage artifacts can be stored as a point cloud and displayed through the Internet. However, due to natural and human factors, many cultural relics had some surface damage when excavated. As a result, the holes caused by these damages still exist in the generated point cloud model. This work proposes a multi-scale upsampling GAN (MU-GAN) based framework for completing these holes. Firstly, a 3D mesh model based on the original point cloud is reconstructed, and the method of detecting holes is presented. Secondly, the point cloud patch contains hole regions and is extracted from the point cloud. Then the patch is input into the MU-GAN to generate a high-quality dense point cloud. Finally, the empty areas on the original point cloud are filled with the generated dense point cloud patches. A series of real-world experiments are conducted on real scan data to demonstrate that the proposed framework can fill the holes of 3D heritage models with grained details. We hope that our work can provide a useful tool for cultural heritage protection.

Initial Assessment on the Use of State-of-the-Art Nerf Neural Network 3d Reconstruction for Heritage Documentation

“…For example, the generation of dense point cloud for reflective surfaces remain difficult to be addressed using classical dense matching approaches. Recent solutions tend to gear towards the use of artificial intelligence (AI) and neural networks, for example to aid the reconstruction of textureless objects (Stathopoulou et al, 2021), to recover lost heritage from historical archives (Condorelli et al, 2020) or to improve 3D models via upsampling (Ren et al, 2022).…”