Neural Radiance Fields (Nerf): Review and Potential Applications to Digital Cultural Heritage

Abstract

Abstract. Neural Radiance Fields (NeRF or NeRFs) are to date emerging as a novel method for synthesizing novel views of complex 3D scenes, leveraging an artificial neural network to optimize a volumetric scene function using a set of input views. We conduct a preliminary critical review of the scientific and technical literature on NeRFs, and we highlight possible applications of the latter in the Cultural Heritage domain, for the image-based reconstruction of 3D models of real, multi-scale objects, even in combination with the more well-established photogrammetric techniques. A comparison is made between NeRFs and photogrammetry in terms of operating procedures and outputs (volumetric renderings vs. point clouds or meshes). It is demonstrated that NeRFs could be conveniently used for rendering objects (sculptures, archaeological remains, sites, paintings etc.) that are challenging for photogrammetry, typically: i) metallic, translucent, and/or transparent surfaces; ii) objects that present homogeneous textures; iii) occlusions, vegetation, and elements of very fine detail.

A Critical Analysis of NeRF-Based 3D Reconstruction

“…In the field of cultural heritage, only a limited number of publications have explicitly investigated and recognized the potential of NeRFs for 3D reconstruction, digital preservation and conservation purposes [96,97].…”

Comparative Assessment of NeRF and Photogrammetry in Digital Heritage: Impact of Varying Image Conditions on 3D Reconstruction

“…In the specific field of cultural heritage, the first experiments began with the work by Condorelli and Rinaudo [37], and they have continued over the past three years, incorporating the developments and improvements that NeRFs have undergone since their earliest version. Interest in the application of NeRF to the Cultural Heritage sector is on the rise, as evidenced by a recent session at the CIPA 2023 Conference on Documenting, Understanding, Preserving Cultural Heritage, entirely dedicated to the theme 'AI and NeRF for 3D reconstruction' [36,[38][39][40][41].…”

Automatic Removal of Non-Architectural Elements in 3D Models of Historic Buildings with Language Embedded Radiance Fields

“…Though rival photogrammetric approaches are able to show strong reconstruction performance in many scenarios, they do have multiple drawbacks in their large storage size, lack of novel view synthesis ability, and lack of native methods for manipulation or understanding of the 3D content. To ameliorate these concerns and also to explore the potentialities of more novel methods of 3D reconstruction, we utilized strictly neural radiance field-based approaches following Pepe et al [11], Llull et al [12], and Croce et al [13][14][15] who demonstrate the feasibility of utilizing NeRFs specifically within the cultural heritage domain. In particular, we employed language embedded radiance fields (LERFs) to introduce querying ability to our models and make the identification of extraneous objects possible.…”