DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments

Yu, Chao; Liu, Zuxin; Liu, Xinjun; Xie, Fugui; Yang, Yi; Wei, Qi; Qiao, Fei

doi:10.1109/iros.2018.8593691

Cited by 674 publications

(479 citation statements)

References 13 publications

Supporting

Mentioning

399

Contrasting

Unclassified

Order By: Relevance

“…In recent years, several methods have been explored. The first method relies on deep learning techniques, such as semantic segmentation [6] and instance segmentation [7][8][9][10][11]. This method can detect predefined dynamic objects at the pixel level, but it cannot do anything for undefined moving objects or objects with uncertain motion properties.…”

Section: Dynamic Pixels Detectionmentioning

confidence: 99%

“…Moreover, the substantial demand on GPUs limits its application on consumer indoor robots. The second method takes advantage of the constraints introduced by multiview geometry [6,9,17]. It assumes that static pixels satisfy the model of multiview geometry, while dynamic pixels do not.…”

Section: Dynamic Pixels Detectionmentioning

confidence: 99%

“…Localization and dense mapping for dynamic environments require pixelwise dynamic data detection. Semantic segmentation [6] and instance segmentation [7][8][9][10][11] allow predefined dynamic objects to be detected at the pixel level. However, these methods heavily rely on the graphics processing unit (GPU), which is costly for most indoor mobile robots.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DRE-SLAM: Dynamic RGB-D Encoder SLAM for a Differential-Drive Robot

Yang¹,

Bi²,

Wang³

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

The state-of-the-art visual simultaneous localization and mapping (V-SLAM) systems have high accuracy localization capabilities and impressive mapping effects. However, most of these systems assume that the operating environment is static, thereby limiting their application in the real dynamic world. In this paper, by fusing the information of an RGB-D camera and two encoders that are mounted on a differential-drive robot, we aim to estimate the motion of the robot and construct a static background OctoMap in both dynamic and static environments. A tightly coupled feature-based method is proposed to fuse the two types of information based on the optimization. Dynamic pixels occupied by dynamic objects are detected and culled to cope with dynamic environments. The ability to identify the dynamic pixels on both predefined and undefined dynamic objects is available, which is attributed to the combination of the CPU-based object detection method and a multiview constraint-based approach. We first construct local sub-OctoMaps by using the keyframes and then fuse the sub-OctoMaps into a full OctoMap. This submap-based approach gives the OctoMap the ability to deform, and significantly reduces the map updating time and memory costs. We evaluated the proposed system in various dynamic and static scenes. The results show that our system possesses competitive pose accuracy and high robustness, as well as the ability to construct a clean static OctoMap in dynamic scenes.

show abstract

Section: Dynamic Pixels Detectionmentioning

confidence: 99%

Section: Dynamic Pixels Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DRE-SLAM: Dynamic RGB-D Encoder SLAM for a Differential-Drive Robot

Yang¹,

Bi²,

Wang³

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In contrast to our work, they are working with gray value images only, are aiming at real-time navigation and are assuming sensors that have been already calibrated. Also similar to the proposed method, and (Yu et al, 2018) are using knowledge about the presence of dynamic objects in images to remove feature outliers to make the 3D reconstruction more robust. Both methods are treating movable objects by special processing steps, underlining their potential for causing problems when using 3D reconstruction and localization approaches for self-calibration in dynamic environments like public road scenes.…”

Section: Semantic 3d Reconstruction and Localizationmentioning

confidence: 99%

Semantic Road Scene Knowledge for Robust Self-Calibration of Environment-Observing Vehicle Cameras

Hanel

Stilla

2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Environment-observing vehicle camera self-calibration using a structure from motion (SfM) algorithm allows calibration over vehicle lifetime without the need of special calibration objects being present in the calibration images. Scene-specific problems with featurebased correspondence search and reconstruction during the SfM pipeline might be caused by critical objects like moving objects, poor-texture objects or reflecting objects and might have negative influence on camera calibration. In this contribution, a method to use semantic road scene knowledge by means of semantic masks for a semantic-guided SfM algorithm is proposed to make the calibration more robust. Semantic masks are used to exclude image parts showing critical objects from feature extraction, whereby semantic knowledge is obtained by semantic segmentation of the road scene images. The proposed method is tested with an image sequence recorded in a suburban road scene. It has been shown that semantic guidance leads to smaller deviations of the estimated interior orientation and distortion parameters from reference values obtained by test field calibration compared to a standard SfM algorithm.

show abstract

“…In [8], a segment based map representation was proposed for 3D point clouds with extracted semantic information for localization only against static objects. To increase the robustness in dynamic environments, semantic masks were used to distinguish the static feature points and remove the outliers on moving objects [9]. Our proposed VO system also benefits from an extra processing with semantic segmentation results.…”

Section: Introductionmentioning

confidence: 99%

ViLiVO: Virtual LiDAR-Visual Odometry for an Autonomous Vehicle with a Multi-Camera System

Xiang

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

In this paper, we present a multi-camera visual odometry (VO) system for an autonomous vehicle. Our system mainly consists of a virtual LiDAR and a pose tracker. We use a perspective transformation method to synthesize a surroundview image from undistorted fisheye camera images. With a semantic segmentation model, the free space can be extracted. The scans of the virtual LiDAR are generated by discretizing the contours of the free space. As for the pose tracker, we propose a visual odometry system fusing both the feature matching and the virtual LiDAR scan matching results. Only those feature points located in the free space area are utilized to ensure the 2D-2D matching for pose estimation. Furthermore, bundle adjustment (BA) is performed to minimize the feature points reprojection error and scan matching error. We apply our system to an autonomous vehicle equipped with four fisheye cameras. The testing scenarios include an outdoor parking lot as well as an indoor garage. Experimental results demonstrate that our system achieves a more robust and accurate performance comparing with a fisheye camera based monocular visual odometry system.

show abstract

DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments

Cited by 674 publications

References 13 publications

DRE-SLAM: Dynamic RGB-D Encoder SLAM for a Differential-Drive Robot

DRE-SLAM: Dynamic RGB-D Encoder SLAM for a Differential-Drive Robot

Semantic Road Scene Knowledge for Robust Self-Calibration of Environment-Observing Vehicle Cameras

ViLiVO: Virtual LiDAR-Visual Odometry for an Autonomous Vehicle with a Multi-Camera System

Contact Info

Product

Resources

About