Block-NeRF: Scalable Large Scene Neural View Synthesis

Tancik, Matthew; Casser, Vincent; Yan, Xinchen; Pradhan, Sabeek; Mildenhall, Ben; Srinivasan, Pratul P.; Barron, Jonathan T.; Kretzschmar, Henrik

doi:10.1109/cvpr52688.2022.00807

Cited by 280 publications

(124 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have also demonstrated the ability to perform real-time localization with Loc-NeRF on a real-world robotic platform. Future work includes using adaptive techniques to adjust the number of particles [51] as well as scaling up localization to larger environments using bigger NeRF models such as [26] and [28]. Additionally, computation time can be reduced by leveraging recent work in faster NeRF rendering such as [50].…”

Section: Discussionmentioning

confidence: 99%

“…These methods take several hours or over a day to train and are intended for building a NeRF as opposed to real-time pose estimation with a trained NeRF. NeRF has also been extended to large-scale [26], [27], [28] and unbounded scenes [29], [30], which has the potential to enable neural representations of large-scale scenes such as the ones typically encountered in robotics applications, from drone navigation to self-driving cars.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields

Maggio¹,

Abate²,

Shi³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present Loc-NeRF, a real-time vision-based robot localization approach that combines Monte Carlo localization and Neural Radiance Fields (NeRF). Our system uses a pre-trained NeRF model as the map of an environment and can localize itself in real-time using an RGB camera as the only exteroceptive sensor onboard the robot. While neural radiance fields have seen significant applications for visual rendering in computer vision and graphics, they have found limited use in robotics. Existing approaches for NeRF-based localization require both a good initial pose guess and significant computation, making them impractical for real-time robotics applications. By using Monte Carlo localization as a workhorse to estimate poses using a NeRF map model, Loc-NeRF is able to perform localization faster than the state of the art and without relying on an initial pose estimate. In addition to testing on synthetic data, we also run our system using real data collected by a Clearpath Jackal UGV and demonstrate for the first time the ability to perform real-time global localization with neural radiance fields. We make our code publicly available at https://github.com/MIT-SPARK/Loc-NeRF.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields

Maggio¹,

Abate²,

Shi³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Our algorithm has some limitations, all of which create opportunities for future work. First, we currently focus on single-object reconstruction but plan to expand our method to large-scale reconstruction of marine environments at the scale of harbors by taking inspiration from techniques such as Block-Nerf [45]. Second, our method is currently mostly suited for offline 3D reconstructions but using techniques such as Instant-NGP [46] and Relu-Fields [47] can bring it to real-time performance needed for robotic navigation applications.…”

Section: Discussionmentioning

confidence: 99%

Neural Implicit Surface Reconstruction using Imaging Sonar

Qadri¹,

Kaess²,

Gkioulekas³

2022

Preprint

View full text Add to dashboard Cite

We present a technique for dense 3D reconstruction of objects using an imaging sonar, also known as forward-looking sonar (FLS). Compared to previous methods that model the scene geometry as point clouds or volumetric grids, we represent the geometry as a neural implicit function. Additionally, given such a representation, we use a differentiable volumetric renderer that models the propagation of acoustic waves to synthesize imaging sonar measurements. We perform experiments on real and synthetic datasets and show that our algorithm reconstructs high-fidelity surface geometry from multi-view FLS images at much higher quality than was possible with previous techniques and without suffering from their associated memory overhead.

show abstract

“…Zhang et al [27] solved the imbalance problem of NeRF in foreground and background rendering by training two MLPs respectively. Recently, several approaches [20] [21] [25] have successfully applied NeRF to the implicit reconstruction of city-scale scenes. Among them, Mega-NeRF [21] achieves cutting-edge results in novel view synthesis by dividing the map into cells in the spatial domain and fitting a set of NeRFs in parallel to represent the whole scene.…”

Section: A Nerf-based Representations For Large Scenementioning

confidence: 99%

LATITUDE: Robotic Global Localization with Truncated Dynamic Low-pass Filter in City-scale NeRF

Zhu¹,

Chen²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Block-NeRF: Scalable Large Scene Neural View Synthesis

Cited by 280 publications

References 39 publications

Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields

Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields

Neural Implicit Surface Reconstruction using Imaging Sonar

LATITUDE: Robotic Global Localization with Truncated Dynamic Low-pass Filter in City-scale NeRF

Contact Info

Product

Resources

About