vMAP: Vectorised Object Mapping for Neural Field SLAM

Kong, Xiangming; Liu, Shikun; Marwan, Taher,; Davison, Andrew J.

doi:10.48550/arxiv.2302.01838

Cited by 3 publications

(4 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Neural implicit representations infer scene semantics [4], [5], [37]- [42] jointly with geometry using a multi-layer perceptron or similar parametric model. These have been extended to dynamic scenes [43]. Neural feature fields [5], [38], [44], [45] are neural implicit representations which map continuous 3D coordinates to vector-valued features.…”

Section: Semantic Scene Representationsmentioning

confidence: 99%

See 1 more Smart Citation

Neural Implicit Vision-Language Feature Fields

Blomqvist¹,

Milano²,

Chung³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recently, groundbreaking results have been presented on open-vocabulary semantic image segmentation. Such methods segment each pixel in an image into arbitrary categories provided at run-time in the form of text prompts, as opposed to a fixed set of classes defined at training time. In this work, we present a zero-shot volumetric open-vocabulary semantic scene segmentation method. Our method builds on the insight that we can fuse image features from a visionlanguage model into a neural implicit representation. We show that the resulting feature field can be segmented into different classes by assigning points to natural language text prompts. The implicit volumetric representation enables us to segment the scene both in 3D and 2D by rendering feature maps from any given viewpoint of the scene. We show that our method works on noisy real-world data and can run in real-time on live sensor data dynamically adjusting to text prompts. We also present quantitative comparisons on the ScanNet dataset.

show abstract

Section: Semantic Scene Representationsmentioning

confidence: 99%

“…For the time being, our method is limited to static scenes. Dealing with moving objects within scenes remains an open problem, but promising recent research [43] suggests that extending neural implicit representations to dynamic scenes might be feasible.…”

Section: Query Performancementioning

confidence: 99%

Neural Implicit Vision-Language Feature Fields

Blomqvist¹,

Milano²,

Chung³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The work most similar to ours is [37], but it focuses on analyzing the effect of observation quality on reconstruction results. In addition, a concurrent work [38] also models objects separately while reconstructing the scene, but it uses RGB-D observations. III.…”

Section: B Nerfs and Nerf-based Slammentioning

confidence: 99%

MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

Zhang

Wang²,

Yang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

We present MMNeRF, a simple yet powerful learning framework for highly photo-realistic novel view synthesis by learning Multi-modal and Multi-view features to guide neural radiance fields to a generic model. Novel view synthesis has achieved great improvement with the significant success of NeRFseries methods. However, how to make the method generic across scenes has always been a challenging task. A good idea is to introduce 2D image features as prior knowledge for adaptive modeling, yet RGB features lack geometry and 3D spatial information, which causes shape-radiance ambiguity issues and lead to blurry and low-resolution results in the synthesis images. We propose a multi-modal multi-view method to make up for the existing methods. Specifically, we introduce depth features besides RGB features into the model and effectively fuse these multi-modal features by modality-based attention. Furthermore, Our framework innovatively adopts the transformer encoder to fuse multi-view features and uses the transformer decoder to adaptively incorporate the target view with global memory. Extensive experiments are carried out on both categories-specific and category-agnostic benchmarks, and the results demonstrate that our MMNeRF achieves state-of-the-art neural rendering performance. INDEX TERMSNeural rendering, Novel view synthesis, Vision transformer, 3D implicit reconstruction Nowadays, several works (e.g., [7]-[9], [11]-[13]) are committed to the study of generality to solve the vanilla

show abstract

“…Finally, outside of the point cloud reconstruction realm, the recent popularity of Neural Radiance Fields [Mildenhall et al 2021] has also given rise to uncertainty-driven approaches for next-bestview planning in RGB multi-view representations (see, e.g., [Jin et al 2023;Kong et al 2023;Smith et al 2022;Sucar et al 2021]).…”

Section: Next-best-view Planningmentioning

confidence: 99%

Neural Stochastic Poisson Surface Reconstruction

Sellán,

Jacobson

2023

SIGGRAPH Asia 2023 Conference Papers

View full text Add to dashboard Cite

Fine-tune our pretrained model to obtain new reconstruction Statistical formalism provides optimal next camera positions Our updated reconstruction rotated view Figure 1: We use a neural network to quantify the reconstruction uncertainty in Poisson Surface Reconstruction (center left), allowing us to efficiently select next sensor positions (center right) and update the reconstruction upon capturing data (right).

show abstract

vMAP: Vectorised Object Mapping for Neural Field SLAM

Cited by 3 publications

References 35 publications

Neural Implicit Vision-Language Feature Fields

Neural Implicit Vision-Language Feature Fields

MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

Neural Stochastic Poisson Surface Reconstruction

Contact Info

Product

Resources

About