Semantic SLAM Based on Improved DeepLabv3⁺ in Dynamic Scenarios

Hu, Zhangfang; Zhao, Jiang; Luo, Yuan; Ou, Junxiong

doi:10.1109/access.2022.3154086

Cited by 22 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ORB-SLAM3 is shown by O3. Meanwhile, the RDS represents RDS-SLAM [26], and the DeepLab represents the method proposed by Hu et al [24]. As presented in table 1, in ATE, the method proposed by us has nearly achieved the best performance in all sequences except the w/rpy sequence.…”

Section: Evaluation Metricsmentioning

confidence: 91%

“…Many studies recently focused on the ORB-SLAM3, a stable SLAM system proposed recently. Based on the ORB-SLAM3, Hu et al [24] use the DeepLab v3+ [25] to dynamic segment objects and filter them with multi-view geometry, which is not capable of handling unknown objects. Liu and Miura [26] use moving probability to update and propagate semantic information to filter out moving points in tracking.…”

Section: Dynamic Slam Improved By the Semantic Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Semantic SLAM for mobile robots in dynamic environments based on visual camera sensors

Zhang

2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

Visual SLAM is constrained by a strong static world assumption,making it hardly succeed when dynamic objects appear. Although some methodseliminate dynamic objects by combining semantic and geometric information, thefewer classes detected by the semantic method can not benefit these systems byapplying more scenes and constructing a richer semantic map. Our paper raises asemantic visual SLAM system for a motion scene, which can run in real-time. Weuse object detection to detect 80 classes in the scene and the moving consistencychecking to find outliers in every image. We propose different methods to checkthe motion state of humans and other objects separately. For the detected human,we propose an algorithm that judges if a person is sitting and divides the boundingbox belonging to the sitting person into two parts according to the proportion ofthe human body. Then we use the same threshold as checking the boxes of thestanding person to determine the state of the two boxes belonging to the sittingperson, respectively. For other different objects except for humans, we propose analgorithm that automatically adjusts the threshold of different bounding boxes.This way, it can have the same detection performance for different objects. Finally,we retain the static box points contained in the dynamic box and eliminate otherpoints in the dynamic box to benefit system performance from more detectedclasses. We evaluated our SLAM on the TUM RGB-D dataset. The performanceof our SLAM exceeds most SLAM systems in a dynamic environment. Our systemis also tested in a real environment with a monocular camera to show its robustnessand universality.

show abstract

Section: Evaluation Metricsmentioning

confidence: 91%

Section: Dynamic Slam Improved By the Semantic Methodsmentioning

confidence: 99%

Semantic SLAM for mobile robots in dynamic environments based on visual camera sensors

Zhang

2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Deep learning-based approaches: To address the limitations of geometric constraint-based dynamic feature point filtering methods, many researchers use prior knowledge from deep learning to assist SLAM systems in pre-detecting dynamic objects in the scene [29][30][31]. Hu et al [32] proposed an enhanced DeepLabv3 semantic segmentation network to segment potential prior dynamic objects. They then applied multi-view geometry methods within geometric threads to detect the motion state of dynamic objects.…”

Section: Related Workmentioning

confidence: 99%

Real-time visual SLAM based YOLO-Fastest for dynamic scenes

Gong,

Sun,

Zou

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

Within the realm of autonomous robotic navigation, Simultaneous Localization and Mapping (SLAM) serves as a critical perception technology, drawing heightened attention in contemporary research. The traditional SLAM systems perform well in static environments, but in the real physical world, dynamic objects can destroy the static geometric constraints of the SLAM system, further limiting its practical application in the real world. In this paper, a robust dynamic RGB-D SLAM system is proposed to expand the number of static points in the scene by combining with YOLO-Fastest to ensure the effectiveness of the geometric constraints model construction, and then based on that, a new thresholding model is designed to differentiate the dynamic features in the objection bounding box, which takes advantage of the double polyline constraints and the residuals after reprojection to filter the dynamic feature points. In addition, two Gaussian models are constructed to segment the moving objects in the bounding box in the depth image to achieve the effect similar to the instance segmentation under the premise of ensuring the computational speed. In this paper, experiments are conducted on dynamic sequences provided by the TUM dataset to evaluate the performance of the proposed method, and the results show that the RMSE (Root mean squared error) metric of the absolute trajectory error of the algorithm of this paper has at least 80% improvement compared to ORB-SLAM2. Higher robustness in dynamic environments with both high and low dynamic sequences compared to DS-SLAM and Dynaslam, and can effectively provide intelligent localization and navigation for mobile robots.

show abstract

“…Kaneko et al (2018) proposed a semantic V-SLAM framework using masks generated by Deeplabv2 to exclude dynamic feature points. Hu et al (2022) introduced an improved Deeplabv3+ network combined with multi-view geometry to detect dynamic objects. A new Ant colony algorithm was applied to search dynamic points quickly to reduce the computation time.…”

Section: Related Workmentioning

confidence: 99%

Semantic stereo visual SLAM toward outdoor dynamic environments based on ORB-SLAM2

Song

Hao

et al. 2023

View full text Add to dashboard Cite

Purpose The prerequisite for most traditional visual simultaneous localization and mapping (V-SLAM) algorithms is that most objects in the environment should be static or in low-speed locomotion. These algorithms rely on geometric information of the environment and restrict the application scenarios with dynamic objects. Semantic segmentation can be used to extract deep features from images to identify dynamic objects in the real world. Therefore, V-SLAM fused with semantic information can reduce the influence from dynamic objects and achieve higher accuracy. This paper aims to present a new semantic stereo V-SLAM method toward outdoor dynamic environments for more accurate pose estimation. Design/methodology/approach First, the Deeplabv3+ semantic segmentation model is adopted to recognize semantic information about dynamic objects in the outdoor scenes. Second, an approach that combines prior knowledge to determine the dynamic hierarchy of moveable objects is proposed, which depends on the pixel movement between frames. Finally, a semantic stereo V-SLAM based on ORB-SLAM2 to calculate accurate trajectory in dynamic environments is presented, which selects corresponding feature points on static regions and eliminates useless feature points on dynamic regions. Findings The proposed method is successfully verified on the public data set KITTI and ZED2 self-collected data set in the real world. The proposed V-SLAM system can extract the semantic information and track feature points steadily in dynamic environments. Absolute pose error and relative pose error are used to evaluate the feasibility of the proposed method. Experimental results show significant improvements in root mean square error and standard deviation error on both the KITTI data set and an unmanned aerial vehicle. That indicates this method can be effectively applied to outdoor environments. Originality/value The main contribution of this study is that a new semantic stereo V-SLAM method is proposed with greater robustness and stability, which reduces the impact of moving objects in dynamic scenes.

show abstract

Semantic SLAM Based on Improved DeepLabv3⁺ in Dynamic Scenarios

Cited by 22 publications

References 23 publications

Semantic SLAM for mobile robots in dynamic environments based on visual camera sensors

Semantic SLAM for mobile robots in dynamic environments based on visual camera sensors

Real-time visual SLAM based YOLO-Fastest for dynamic scenes

Semantic stereo visual SLAM toward outdoor dynamic environments based on ORB-SLAM2

Contact Info

Product

Resources

About