A real-time semantic visual SLAM for dynamic environment based on deep learning and dynamic probabilistic propagation

Abstract

Most existing visual simultaneous localization and mapping (SLAM) algorithms rely heavily on the static world assumption. Combined with deep learning, semantic SLAM has become a popular solution for dynamic scenes. However, most semantic SLAM methods show poor real-time performance when dealing with dynamic scenes. To handle this problem, a real-time semantic SLAM method is proposed in this paper, combining knowledge distillation and dynamic probability propagation strategy. First, to improve the execution speed, a multi-level knowledge distillation method is adopted to obtain a lightweight segmentation model, which is more suitable for continuous frames to create an independent semantic segmentation thread. This segmentation thread only accepts keyframes as input so that the system can avoid time delay caused by processing each frame. Second, a static semantic keyframe selection strategy is proposed based on the segmentation results. In this way, those keyframes containing more static information will be selected to reduce the participation of dynamic objects. By combining segmentation results and data matching algorithm, our system can realize the update and propagation of dynamic probability, reducing the influence of dynamic points in the pose optimization process. Validation results based on the KITTI and TUM datasets show that our method can effectively deal with dynamic feature points and improve running speed simultaneously.

TDO-SLAM: Traffic Sign and Dynamic Object Based Visual SLAM

“…Chen L. et al [25] introduces a semantic SLAM method to address the real time performance in dynamic environment. PSPNet18 [26].…”

“…RDS-SLAM [27] is similar with the method proposed by Chen L. et al [25] in that the moving probability is used to detect and remove outliers from feature tracking. The initial probability of a map point is assigned as 0.5 and it is updated by the Bayesian filtering [28].…”

“…The Jacobian of the map points for the dynamic object are defined in (25).   represents unknown parameters for a dynamic map point.…”

“…TDO-SLAM is compared with the baseline method ORB-SLAM2 and DynaSLAM. RMSE ATE of ORB-SLAM3 is also compared by referring two papers [25], [51].…”