Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields

Barron, Jonathan T.; Mildenhall, Ben; Verbin, Dor; Srinivasan, Pratul P.; Hedman, Peter

doi:10.1109/cvpr52688.2022.00539

Cited by 529 publications

(277 citation statements)

References 46 publications

Supporting

Mentioning

204

Contrasting

Order By: Relevance

“…Thus, our approach yields to the segmentation error and still cannot provide high-quality hair rendering with opacity details. Inspired by recent work [Barron et al 2022;, we plan to explore the explicit modeling of background and opacity hairs into our neural framework. Moreover, our rendering pipeline relies on the animated meshes and is thus limited by the tracking quality.…”

Section: Limitations and Discussionmentioning

confidence: 99%

Human Performance Modeling and Rendering via Neural Animated Mesh

Zhao¹,

Jiang²,

Yao³

et al. 2022

Preprint

View full text Add to dashboard Cite

We have recently seen tremendous progress in the neural advances for photo-real human modeling and rendering. However, it's still challenging to integrate them into an existing mesh-based pipeline for downstream applications. In this paper, we present a comprehensive neural approach for high-quality reconstruction, compression, and rendering of human performances from dense multi-view videos. Our core intuition is to bridge the traditional animated mesh workflow with a new class of highly efficient neural techniques. We first introduce a neural surface reconstructor for highquality surface generation in minutes. It marries the implicit volumetric for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

show abstract

Section: Limitations and Discussionmentioning

confidence: 99%

Human Performance Modeling and Rendering via Neural Animated Mesh

Zhao¹,

Jiang²,

Yao³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…These methods take several hours or over a day to train and are intended for building a NeRF as opposed to real-time pose estimation with a trained NeRF. NeRF has also been extended to large-scale [26], [27], [28] and unbounded scenes [29], [30], which has the potential to enable neural representations of large-scale scenes such as the ones typically encountered in robotics applications, from drone navigation to self-driving cars.…”

Section: Related Workmentioning

confidence: 99%

Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields

Maggio¹,

Abate²,

Shi³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present Loc-NeRF, a real-time vision-based robot localization approach that combines Monte Carlo localization and Neural Radiance Fields (NeRF). Our system uses a pre-trained NeRF model as the map of an environment and can localize itself in real-time using an RGB camera as the only exteroceptive sensor onboard the robot. While neural radiance fields have seen significant applications for visual rendering in computer vision and graphics, they have found limited use in robotics. Existing approaches for NeRF-based localization require both a good initial pose guess and significant computation, making them impractical for real-time robotics applications. By using Monte Carlo localization as a workhorse to estimate poses using a NeRF map model, Loc-NeRF is able to perform localization faster than the state of the art and without relying on an initial pose estimate. In addition to testing on synthetic data, we also run our system using real data collected by a Clearpath Jackal UGV and demonstrate for the first time the ability to perform real-time global localization with neural radiance fields. We make our code publicly available at https://github.com/MIT-SPARK/Loc-NeRF.

show abstract

“…The coordinate MLP M θ with a hidden layer dimension h = 256 maps a 3D position p = (x, y, z) ∈ R 3 into M θ (p) = (ρ, f , s), where ρ stands for volumetric density, f for a feature vector, and s for semantic logits. Before feeding a point p = (x, y, z) into the scene network M θ we apply off-axis positional encoding [16] to ensure rich representational capacity for fitting high-frequency feature maps and mitigate axis-aligned artifacts caused by the standard positional encoding used in NeRFs.…”

Section: A Scene Networkmentioning

confidence: 99%

Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding

Mazur¹,

Sucar²,

Davison³

2022

Preprint

View full text Add to dashboard Cite

General scene understanding for robotics requires flexible semantic representation, so that novel objects and structures which may not have been known at training time can be identified, segmented and grouped. We present an algorithm which fuses general learned features from a standard pretrained network into a highly efficient 3D geometric neural field representation during real-time SLAM. The fused 3D feature maps inherit the coherence of the neural field's geometry representation. This means that tiny amounts of human labelling interacting at runtime enable objects or even parts of objects to be robustly and accurately segmented in an open set manner. Project page: https://makezur.github.io/ FeatureRealisticFusion/

show abstract

Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields

Cited by 529 publications

References 46 publications

Human Performance Modeling and Rendering via Neural Animated Mesh

Human Performance Modeling and Rendering via Neural Animated Mesh

Loc-NeRF: Monte Carlo Localization using Neural Radiance Fields

Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding

Contact Info

Product

Resources

About