iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Tracking

Ming, Yuhang; Ye, Weicai; Calway, Andrew

doi:10.48550/arxiv.2209.07919

Cited by 3 publications

(4 citation statements)

References 27 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, ScanNet [55] and NYU-Dv2 [56] have RGBD images with semantic segmentation labels, and are commonly used to train deep-learning systems to integrate semantic information into navigation. Whilst NYU-Dv2 lacks the ground-truth camera poses needed for vSLAM benchmarking, ScanNet does have this ground-truth data and has started seeing usage for evaluation purposes, especially for neural implicit SLAM [57,58].…”

Section: Visual Slam Datasetsmentioning

confidence: 99%

Benchmarking visual SLAM methods in mirror environments

Herbert,

Wu,

et al. 2024

Comp. Visual Media

View full text Add to dashboard Cite

Visual simultaneous localisation and mapping (vSLAM) finds applications for indoor and outdoor navigation that routinely subjects it to visual complexities, particularly mirror reflections. The effect of mirror presence (time visible and its average size in the frame) was hypothesised to impact localisation and mapping performance, with systems using direct techniques expected to perform worse. Thus, a dataset, MirrEnv, of image sequences recorded in mirror environments, was collected, and used to evaluate the performance of existing representative methods. RGBD ORB-SLAM3 and BundleFusion appear to show moderate degradation of absolute trajectory error with increasing mirror duration, whilst the remaining results did not show significantly degraded localisation performance. The mesh maps generated proved to be very inaccurate, with real and virtual reflections colliding in the reconstructions. A discussion is given of the likely sources of error and robustness in mirror environments, outlining future directions for validating and improving vSLAM performance in the presence of planar mirrors. The MirrEnv dataset is available at https://doi.org/10.17035/d.2023.0292477898.

show abstract

Section: Visual Slam Datasetsmentioning

confidence: 99%

Benchmarking visual SLAM methods in mirror environments

Herbert,

Wu,

et al. 2024

Comp. Visual Media

View full text Add to dashboard Cite

show abstract

“…scenes, while NICE-SLAM can scale up to much larger indoor environments by applying hierarchical feature grids and tiny MLPs as the scene representation. Many follow-up works improve upon these two works from various perspectives, including efficient scene representation [18,21], fast optimziation [67], add IMU measurements [25], or different shape representations [39,30]. However, all of them require RGB-D inputs, which limits their applications in outdoor scenes or when only RGB sensors are available.…”

Section: Input Rgb Streammentioning

confidence: 99%

“…NICE-SLAM [76] introduces a hierarchical implicit encoding to perform map-ping and camera tracking in much larger indoor scenes. Although follow-up works [67,30,21,18,25,39] try to improve upon NICE-SLAM and iMAP from different perspectives, all of these works still rely on the reliable depth input from RGB-D sensors.…”

Section: Introductionmentioning

confidence: 99%

NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

Zhu

Peng

Larsson

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

330

View full text Add to dashboard Cite

show abstract

“…Compared to iMAP, it updates only the visible grid features at each step, effectively solving the forgetting problem. Subsequent works have made further improvements, including the integration with traditional voxel grids [34] and different shape representations [35], [36]. In contrast to these methods that focus on dense reconstruction of the scene, our approach emphasizes object instances with semantic meaning.…”

Section: B Nerfs and Nerf-based Slammentioning

confidence: 99%

MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

Zhang

Wang²,

Yang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

We present MMNeRF, a simple yet powerful learning framework for highly photo-realistic novel view synthesis by learning Multi-modal and Multi-view features to guide neural radiance fields to a generic model. Novel view synthesis has achieved great improvement with the significant success of NeRFseries methods. However, how to make the method generic across scenes has always been a challenging task. A good idea is to introduce 2D image features as prior knowledge for adaptive modeling, yet RGB features lack geometry and 3D spatial information, which causes shape-radiance ambiguity issues and lead to blurry and low-resolution results in the synthesis images. We propose a multi-modal multi-view method to make up for the existing methods. Specifically, we introduce depth features besides RGB features into the model and effectively fuse these multi-modal features by modality-based attention. Furthermore, Our framework innovatively adopts the transformer encoder to fuse multi-view features and uses the transformer decoder to adaptively incorporate the target view with global memory. Extensive experiments are carried out on both categories-specific and category-agnostic benchmarks, and the results demonstrate that our MMNeRF achieves state-of-the-art neural rendering performance. INDEX TERMSNeural rendering, Novel view synthesis, Vision transformer, 3D implicit reconstruction Nowadays, several works (e.g., [7]-[9], [11]-[13]) are committed to the study of generality to solve the vanilla

show abstract

iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Tracking

Cited by 3 publications

References 27 publications

Benchmarking visual SLAM methods in mirror environments

Benchmarking visual SLAM methods in mirror environments

NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

Contact Info

Product

Resources

About