Nerf for Heritage 3d Reconstruction

Abstract

Abstract. Conventional or learning-based 3D reconstruction methods from images have clearly shown their potential for 3D heritage documentation. Nevertheless, Neural Radiance Field (NeRF) approaches are recently revolutionising the way a scene can be rendered or reconstructed in 3D from a set of oriented images. Therefore the paper wants to review some of the last NeRF methods applied to various cultural heritage datasets collected with smartphone videos, touristic approaches or reflex cameras. Firstly several NeRF methods are evaluated. It turned out that Instant-NGP and Nerfacto methods achieved the best outcomes, outperforming all other methods significantly. Successively qualitative and quantitative analyses are performed on various datasets, revealing the good performances of NeRF methods, in particular for areas with uniform texture or shining surfaces, as well as for small datasets of lost artefacts. This is for sure opening new frontiers for 3D documentation, visualization and communication purposes of digital heritage.

Evolution of Rendering Based on Radiance Fields. The Palermo Case Study for a Comparison Between Nerf and Gaussian Splatting

“…The advantages of Neural Radiance Field (NeRF) are also being tested in the field of Cultural Heritage. Some research compares and analyses the advantages of using NeRF over the photogrammetric technique or laser scanning, especially in critical conditions such as low number of photos or low image resolution (Croce et al, 2024;Palestini et al, 2021Balloni et al 2023, in the reconstruction of buildings with very difficult conditions from typical ones (Condorelli et al 2021), or also explore the potential of NeRF in the cases of unconventional acquisition conditions (Mazzacca et al 2023).…”

Evaluation of Nerf 3d Reconstruction for Rock Art Documentation

“…Initially introduced by Mildenhall et al (2020), NeRF has revolutionised the synthesis of novel views. NeRF is a specialised technique that puts the trained image into artificial neural networks, a segment within the domain of artificial intelligence (AI) utilised in computer vision and graphics (Mazzacca et al, 2023). Advantageously, NeRF efficiently renders 3D scenes from 2D images without high computing power and memory (Mildenhall et al, 2020;Tancik et al, 2023).…”

“…The evaluation of NeRF output quality mostly assesses the accuracy of the 3D model generated by NeRF by comparing it with the photogrammetry/MVS method. The evaluation involves analysing discrepancies of NeRF-generated point cloud through distance error analysis, point cloud density and completeness Mazzacca et al 2023;Murtiyoso and Grussenmeyer 2023;Pepe, Alfio, and Costantino 2023;Remondino et al 2023). Among the previously mentioned studies, there is limited literature that discusses its potential for the digital documentation of rock art.…”

Evaluating the Point Cloud of Individual Trees Generated from Images Based on Neural Radiance Fields (NeRF) Method

“…NeRF's main task is to synthesize new views based on the known view images, but can also be used to generate mesh using marching cubes [11]; further, in nerfstudio (https://docs.nerf.studio/ index.html, accessed on 22 November 2023), the reconstructed mesh and point cloud can be exported. Since the original NeRF was proposed, the field has grown explosively with hundreds of papers extending or building on it each year, and this method has found new applications in various areas, including autonomous driving [12], medicine [13], digital human body [14], 3D cities [15] and cultural heritage reconstruction [16,17], to name a few.…”