A Critical Analysis of NeRF-Based 3D Reconstruction

Remondino, Fabio; Karami, Ali; Yan, Ziyang; Mazzacca, Gabriele; Rigon, Simone; Qin, Rongjun

doi:10.3390/rs15143585

Cited by 17 publications

(9 citation statements)

References 99 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instant-NGP We use the original Instant-NGP framework since it achieves a similar accuracy as Nerfacto [43] but enables faster reconstruction. While the radiance field depends on viewing direction and does not separate color and illumination, the density field represents NeRFs geometry and is only related to query positions [44].…”

Section: Neural Radiance Fieldsmentioning

confidence: 99%

Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs)

Petrovska,

Jutzi

2024

Remote Sensing

View full text Add to dashboard Cite

In this contribution we evaluate the 3D geometry reconstructed by Neural Radiance Fields (NeRFs) of an object’s occluded parts behind obstacles through a point cloud comparison in 3D space against traditional Multi-View Stereo (MVS), addressing the accuracy and completeness. The key challenge lies in recovering the underlying geometry, completing the occluded parts of the object and investigating if NeRFs can compete against traditional MVS for scenarios where the latter falls short. In addition, we introduce a new “obSTaclE, occLusion and visibiLity constrAints” dataset named STELLA concerning transparent and non-transparent obstacles in real-world scenarios since there is no existing dataset dedicated to this problem setting to date. Considering that the density field represents the 3D geometry of NeRFs and is solely position-dependent, we propose an effective approach for extracting the geometry in the form of a point cloud. We voxelize the whole density field and apply a 3D density-gradient based Canny edge detection filter to better represent the object’s geometric features. The qualitative and quantitative results demonstrate NeRFs’ ability to capture geometric details of the occluded parts in all scenarios, thus outperforming in completeness, as our voxel-based point cloud extraction approach achieves point coverage up to 93%. However, MVS remains a more accurate image-based 3D reconstruction method, deviating from the ground truth 2.26 mm and 3.36 mm for each obstacle scenario respectively.

show abstract

Section: Neural Radiance Fieldsmentioning

confidence: 99%

Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs)

Petrovska,

Jutzi

2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Challenges in acquiring 2D images lie in achieving the appropriate resolution and color accuracy, to ensure their most accurate reproduction. In the case of 3D, additional requirements include images leveraging common techniques such as photogrammetry for geometric surveying and photometric stereo [22][23][24][25][26][27] for primarily reconstructing the mesostructure, along with more recent advancements such as neural radiance fields (NeRF) [28][29][30][31].…”

Section: Pipelinesmentioning

confidence: 99%

Evolving Perspectives in Digitizing Cultural Heritage: Novel Paradigms through Self-Built and Adaptable Equipment

Bacci¹,

Bozzola²,

Gaiani³

et al. 2023

Preprint

View full text Add to dashboard Cite

The emergence of new methodologies and tools for digitizing objects belonging to the Cultural Heritage (CH) changed the paradigms adopted so far. Traditionally, the process to acquire artifacts required specialized and often costly equipment tailored to specific purposes. However, the development of more generalized, adaptable, and affordable tools led to novel approaches. This manuscript provides a glimpse into the evolving landscape of custom-made tools for digital documentation, both hardware and software, and their transformative impact on the digitization techniques, built to meet requirements of specific case studies, including ancient drawings, manuscripts, paintings, and museum objects. The advent of self-built instruments has revolutionized the way professionals work today, by leveraging a new generation of low-cost, adaptable gears, leading to unprecedented flexibility and efficiency, while facilitating the capture of high-quality digital representations of objects with minimal damage and preserving their integrity. The outcomes of instruments and tools specifically produced for the contexts described in this paper highlight their potential for promoting interdisciplinary collaboration, facilitating scholarly research, enhancing conservation efforts, and fostering cultural exchange. Ultimately, this research contributes to illustrate how custom software, in combination with recent devices such as smartphones and 3D printers, underscores the importance of adopting these innovative approaches to generate an ecosystem of tools and methods able to preserve, document, and promote the richness of our collective past for future generations.

show abstract

Section: Pipelinesmentioning

confidence: 99%

Novel Paradigms in the Cultural Heritage Digitization with Self and Custom-Built Equipment

Bacci,

Bozzola,

Gaiani

et al. 2023

Heritage

View full text Add to dashboard Cite

In the field of the Cultural Heritage (CH), image-based 2D and 3D digital acquisition is today the most common technique used to create digital replicas of existing artifacts. This is carried out for many reasons, such as the following: research, analysis, preservation, conservation, communication, and valorization. These activities usually require complementary specialized equipment, tailored to specific purposes in order to achieve the desired results. This equipment is not easy-to-find on the market, it is not always affordable for museums operators; it is sometimes expensive, and it usually needs tricky customizations. However, the development in recent years of more generalized, versatile, and affordable instruments and technologies has led to new approaches, leveraging a new generation of low-cost, adaptable equipment. This paper presents custom-made equipment following this new path, designed to provide optimized results through calibrated tools alongside the software to make it work. The essay focuses specifically on the self-production of instruments for the digital reproduction of ancient drawings, manuscripts, paintings, and other museum artifacts and their transformative impact on digitization techniques. The outcomes of self and custom-built equipment specifically produced for the contexts described in this paper highlight their potential to foster interdisciplinary collaboration, facilitate scholarly research, enhance conservation efforts, and promote cultural exchange. The final goal is to propose inexpensive equipment that is easy to use (even by not specifically trained operators) and that provides remarkable quality.

show abstract

A Critical Analysis of NeRF-Based 3D Reconstruction

Cited by 17 publications

References 99 publications

Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs)

Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs)

Evolving Perspectives in Digitizing Cultural Heritage: Novel Paradigms through Self-Built and Adaptable Equipment

Novel Paradigms in the Cultural Heritage Digitization with Self and Custom-Built Equipment

Contact Info

Product

Resources

About