NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

Zhu, Zihan; Peng, Songyou; Larsson, Viktor; Xu, Wei; Bao, Hujun; Cui, Zhaopeng; Oswald, Martin R.; Pollefeys, Marc

doi:10.48550/arxiv.2112.12130

Cited by 4 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work is strongly inspired by iMAP and indeed the active sampling and keyframe selection in iSDF are based on iMAP. NICE-SLAM [39] builds on iMAP by proposing to use a voxel grid of neural fields instead of a single global model. Although not a real-time system, Yan et al [36] investigates offline continual learning for neural fields.…”

Section: Related Workmentioning

confidence: 99%

“…Based on a multi-layer perceptron (MLP) that maps a 3D coordinate to occupancy, these models can be optimised from scratch to accurately fit a specific scene without prior training. Recent work has shown that neural fields can reconstruct highly accurate 3D geometry and that they can be trained in real-time as part of a SLAM system [30,39].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

iSDF: Real-Time Neural Signed Distance Fields for Robot Perception

Ortiz¹,

Clegg²,

Dong³

et al. 2022

Preprint

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

iSDF: Real-Time Neural Signed Distance Fields for Robot Perception

Ortiz¹,

Clegg²,

Dong³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…Some works learn a light field for a vivid relighting effect [3,39,65]. In robotics, researchers turn the learning problem inversely to optimize the 6D pose [57] or extend to the environment mapping system [40,67].…”

Section: Novel View Synthesismentioning

confidence: 99%

V4D: Voxel for 4D Novel View Synthesis

Wanshui¹,

Xu²,

Huang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural radiance fields have made a remarkable breakthrough in the novel view synthesis task at the 3D static scene. However, for the 4D circumstance (e.g., dynamic scene), the performance of the existing method is still limited by the capacity of the neural network, typically in a multilayer perceptron network (MLP). In this paper, we present the method to model the 4D neural radiance field by the 3D voxel, short as V4D, where the 3D voxel has two formats. The first one is to regularly model the bounded 3D space and then use the sampled local 3D feature with the time index to model the density field and the texture field. The second one is in look-up tables (LUTs) format that is for the pixel-level refinement, where the pseudo-surface produced by the volume rendering is utilized as the guidance information to learn a 2D pixel-level refinement mapping. The proposed LUTsbased refinement module achieves the performance gain with a little computational cost and could serve as the plug-and-play module in the novel view synthesis task. Moreover, we propose a more effective conditional positional encoding toward the 4D data that achieves performance gain with negligible computational burdens. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance by a large margin. At last, the proposed V4D is also a computational-friendly method in both the training and testing phase, where we achieve 2 times faster in the training phase and 10 times faster in the inference phase compared with the state-of-the-art method. The relevant code will be available in https://github.com/GANWANSHUI/V4D.Preprint. Under review.

show abstract

“…They parameterize viewing rays and points with positional encoding, and need to be re-trained on a scene-by-scene basis. Many recent improvements leverage depth supervision to improve view synthesis in a volume rendering framework [1,5,36,55,62]. An alternative approach replaces volume rendering with a directly learned light field network [44], predicting color values directly from viewing rays.…”

Section: Related Workmentioning

confidence: 99%