Network Uncertainty Informed Semantic Feature Selection for Visual SLAM

Ganti, Pranav; Waslander, Steven L.

doi:10.1109/crv.2019.00024

Cited by 25 publications

(12 citation statements)

References 37 publications

(51 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by the advances of deep learning and Convolutional Neural Networks (CNNs) for scene understanding, there have been many semantic SLAM techniques exploiting this information using cameras [5], [30], cameras + IMU data [4], stereo cameras [9], [14], [17], [32], [37], or RGB-D sensors [3], [18], [19], [25], [26], [28], [38]. Most of these approaches were only applied indoors and use either an object detector or a semantic segmentation of the camera image.…”

Section: Introductionmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

337

192

View full text Add to dashboard Cite

Reliable and accurate localization and mapping are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the mapping step or derail localization. In this paper, we propose an extension of a recently published surfelbased mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach.

show abstract

Section: Introductionmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

337

192

View full text Add to dashboard Cite

show abstract

“…Combining object information into SLAM to form semantic slam is one of the typical representatives of the combination of visual SLAM and deep learning. Ganti et al 70 proposed a feature selection method SIVO (SIVO, semantically informed visual odometry and mapping) based on information theory. This method introduces semantic segmentation and neural network uncertainty into feature selection and generates sparse maps which are conducive to long‐term positioning.…”

Section: Related Workmentioning

confidence: 99%

Target localization in local dense mapping using RGBD SLAM and object detection

Liu

Jiang

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

Summary Target localization in unknown environment is one of the development directions of mobile robots. Simultaneous localization and mapping (SLAM) can be used to build maps in unknown environments, but it has the problem of poor readability and interactivity. In this article, target detection and SLAM are combined to search and locate the target by using rich RGBD images information. The determined position in the global map is conducive to the follow‐up operation of the target by mobile robots. By establishing a local dense point cloud map of the target object, the current state of the target object is directly displayed, the readability of the map is improved, and the disadvantages of difficult understanding of the global sparse map and slow construction of the global dense map are avoided. A target localization algorithm under the framework of yolov4 is designed to apply in the process of SLAM global mapping. Our works are helpful for obtaining positions of objects in three‐dimensional space. The experimental results show that the time‐consuming of this method in dense mapping is reduced by 50%–70%, and the number of point clouds is also reduced by 60%–70%.

show abstract

“…This work achieved highly-accurate and robust visual odometry. Ganti et al [39] incorporated semantic segmentation network uncertainty into the feature point selection. If the mutual information of a feature point above a predefined threshold, the uncertainty of this feature point is considered to be small and this feature point will be easily selected.…”

Section: Semantic Slammentioning

confidence: 99%

A novel vSLAM framework with unsupervised semantic segmentation based on adversarial transfer learning

Jin

Chen

Sun

et al. 2020

Applied Soft Computing

View full text Add to dashboard Cite

Significant progress has been made in the field of visual Simultaneous Localization and Mapping (vSLAM) systems. However, the localization accuracy of vSLAM can be significantly reduced in dynamic applications with mobile robots or passengers. In this paper, a novel semantic SLAM framework in dynamic environments is proposed to improve the localization accuracy. We incorporate a semantic segmentation model into the Oriented FAST and Rotated BRIEF-SLAM2 (ORB-SLAM2) system to filter out dynamic feature points, but we encounter one main challenge, i.e. the performance of a segmentation network well-trained with labeled datasets may decrease seriously in a real application without any labeled data due to the inconsistency between the source domain and the target domain. Therefore, we proposed an unsupervised semantic segmentation model with a Residual Neural Network (ResNet) structure, which is trained by the adversarial transfer learning method in the multi-level feature spaces. This work may be the first to perform multi-level feature space adversarial transfer learning for the semantic SLAM task in dynamic environments. In order to evaluate our method, images of indoor scenes from three datasets are used as the source domain, and the dynamic sequences of the TUM dataset are used as the target domain. The extensive experimental results show favorable performance against the state-of-the-art methods in terms of the absolute trajectory accuracy and image semantic segmentation quality.

show abstract

Network Uncertainty Informed Semantic Feature Selection for Visual SLAM

Cited by 25 publications

References 37 publications

SuMa++: Efficient LiDAR-based Semantic SLAM

SuMa++: Efficient LiDAR-based Semantic SLAM

Target localization in local dense mapping using RGBD SLAM and object detection

A novel vSLAM framework with unsupervised semantic segmentation based on adversarial transfer learning

Contact Info

Product

Resources

About