DSP-SLAM: Object Oriented SLAM with Deep Shape Priors

Visual SLAM (VSLAM) has been developing rapidly due to its advantages of low-cost sensors, the easy fusion of other sensors, and richer environmental information. Traditional visionbased SLAM research has made many achievements, but it may fail to achieve wished results in challenging environments. Deep learning has promoted the development of computer vision, and the combination of deep learning and SLAM has attracted more and more attention. Semantic information, as high-level environmental information, can enable robots to better understand the surrounding environment. This paper introduces the development of VSLAM technology from two aspects: traditional VSLAM and semantic VSLAM combined with deep learning. For traditional VSLAM, we summarize the advantages and disadvantages of indirect and direct methods in detail and give some classical VSLAM open-source algorithms. In addition, we focus on the development of semantic VSLAM based on deep learning. Starting with typical neural networks CNN and RNN, we summarize the improvement of neural networks for the VSLAM system in detail. Later, we focus on the help of target detection and semantic segmentation for VSLAM semantic information introduction. We believe that the development of the future intelligent era cannot be without the help of semantic technology. Introducing deep learning into the VSLAM system to provide semantic information can help robots better perceive the surrounding environment and provide people with higher-level help.

show abstract

“…However, for some large objects, accurate pose estimation is not possible. Similarly, University College London proposed DSP-SLAM [249].…”

Section: For Sencementioning

confidence: 99%

An Overview on Visual SLAM: From Tradition to Semantic

Chen

Shang

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Object poses are then used in the SLAM system to estimate the camera pose. DSP-SLAM [3] optimizes the latent code of a deep learning based SDF [6] and uses it to estimate the pose of the object and to reconstruct the object shape. Object poses are then used to constrain camera pose estimation in the BA.…”

Section: Related Work: Dynamic and Object Based Slammentioning

confidence: 99%

“…We inject both detected and registered poses as constrains in the BA, the first one being free from any drift and the second one more accurate. Third, we follow the work of [3] and use DeepSDF [6] to fit a SDF to objects using LiDAR points. The SDF is then used in the BA to constrain the SLAM map points to lie on the object surface.…”

Section: Twistslam++: Multimodal Object Trackingmentioning

confidence: 99%

“…The geometry of objects is an important property that we can inject in the SLAM to improve the accuracy of 3D mapping. To estimate it we use clustered LiDAR points, that are precise and apply the method developed in DSP-SLAM [3]. DSP-SLAM uses DeepSDF [6] to represent the geometry of an object with a signed distance function generated from its latent code vector:…”

Section: E Estimating Clusters Geometrymentioning

confidence: 99%

“…The computed pose is then used as a constraint in the bundle adjustment (BA). Second, inspired by the work of DSP-SLAM [3], we use scans to fit a deep-learnt signed distance function (SDF) [6] that represents the object geometry. However contrary to DSP-SLAM we do not use the SDF to estimate the object pose as we already have a good estimate of it, but we rather use it to constrain the 3D map points of clusters to lie on the estimated mesh, similarly to our previous work [7] which was restricted to planes.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TwistSLAM++: Fusing multiple modalities for accurate dynamic semantic SLAM

Gonzalez¹,

Marchand²,

Kacete³

et al. 2022

Preprint

View full text Add to dashboard Cite

Most classical SLAM systems rely on the static scene assumption, which limits their applicability in real world scenarios. Recent SLAM frameworks have been proposed to simultaneously track the camera and moving objects. However they are often unable to estimate the canonical pose of the objects and exhibit a low object tracking accuracy. To solve this problem we propose TwistSLAM++, a semantic, dynamic, SLAM system that fuses stereo images and LiDAR information. Using semantic information, we track potentially moving objects and associate them to 3D object detections in LiDAR scans to obtain their pose and size. Then, we perform registration on consecutive object scans to refine object pose estimation. Finally, object scans are used to estimate the shape of the object and constrain map points to lie on the estimated surface within the BA. We show on classical benchmarks that this fusion approach based on multimodal information improves the accuracy of object tracking.

show abstract

Attention-guided LiDAR segmentation and odometry using image-to-point cloud saliency transfer

Ding,

İmamoğlu,

Caglayan

et al. 2024

Multimedia Systems

View full text Add to dashboard Cite

LiDAR odometry estimation and 3D semantic segmentation are crucial for autonomous driving, which has achieved remarkable advances recently. However, these tasks are challenging due to the imbalance of points in different semantic categories for 3D semantic segmentation and the influence of dynamic objects for LiDAR odometry estimation, which increases the importance of using representative/salient landmarks as reference points for robust feature learning. To address these challenges, we propose a saliency-guided approach that leverages attention information to improve the performance of LiDAR odometry estimation and semantic segmentation models. Unlike in the image domain, only a few studies have addressed point cloud saliency information due to the lack of annotated training data. To alleviate this, we first present a universal framework to transfer saliency distribution knowledge from color images to point clouds, and use this to construct a pseudo-saliency dataset (i.e. FordSaliency) for point clouds. Then, we adopt point cloud based backbones to learn saliency distribution from pseudo-saliency labels, which is followed by our proposed SalLiDAR module. SalLiDAR is a saliency-guided 3D semantic segmentation model that integrates saliency information to improve segmentation performance. Finally, we introduce SalLONet, a self-supervised saliency-guided LiDAR odometry network that uses the semantic and saliency predictions of SalLiDAR to achieve better odometry estimation. Our extensive experiments on benchmark datasets demonstrate that the proposed SalLiDAR and SalLONet models achieve state-of-the-art performance against existing methods, highlighting the effectiveness of image-to-LiDAR saliency knowledge transfer. Source code will be available at https://github.com/nevrez/SalLONet

show abstract

DSP-SLAM: Object Oriented SLAM with Deep Shape Priors

Cited by 46 publications

References 37 publications

An Overview on Visual SLAM: From Tradition to Semantic

An Overview on Visual SLAM: From Tradition to Semantic

TwistSLAM++: Fusing multiple modalities for accurate dynamic semantic SLAM

Attention-guided LiDAR segmentation and odometry using image-to-point cloud saliency transfer

Contact Info

Product

Resources

About