Geometric Constraint-Based and Improved YOLOv5 Semantic SLAM for Dynamic Scenes

Abstract

When using deep learning networks for dynamic feature rejection in SLAM systems, problems such as a priori static object motion leading to disturbed build quality and accuracy and slow system runtime are prone to occur. In this paper, based on the ORB-SLAM2 system, we propose a method based on improved YOLOv5 networks combined with geometric constraint methods for SLAM map building in dynamic environments. First, this paper uses ShuffleNetV2 to lighten the YOLOv5 network, which increases the improved network’s operation speed without reducing the accuracy. At the same time, a pyramidal scene parsing network segmentation head is added to the head part of the YOLOv5 network to achieve semantic extraction in the environment, so that the improved YOLOv5 network has both target detection and semantic segmentation functions. In order to eliminate the objects with low dynamic features in the environment, this paper adopts the method of geometric constraints to extract and eliminate the dynamic features of the low dynamic objects. By combining the improved YOLOv5 network with the geometric constraint method, the robustness of the system is improved and the interference of dynamic targets in the construction of the SLAM system map is eliminated. The test results on the TUM dataset show that, when constructing a map in a dynamic environment, compared with the traditional ORB-SLAM2 algorithm, the accuracy of map construction in a dynamic environment is significantly improved. The absolute trajectory error is reduced by 97.7% compared with ORB-SLAM2, and the relative position error is reduced by 59.7% compared with ORB-SLAM2. Compared with DynaSLAM for dynamic scenes of the same type, the accuracy of map construction is slightly improved, but the maximum increase in keyframe processing time is 94.7%.

SLM-SLAM: a visual SLAM system based on segmented large-scale model in dynamic scenes and zero-shot conditions

“…In YOLO-SLAM proposed by Wu et al [28], a lightweight object detection network, Darknet19-YOLOv3, is designed to generate the essential semantic information required by the SLAM system with low latency. Zhang and Zhang [29] employed ShuffleNetV2 to lightweight YOLOv5 and added a pyramid-shaped scene parsing network segmentation head, simultaneously achieving target detection and semantic segmentation functionalities. However, these methods based on deep learning typically necessitate prior information and training for specific dynamic objects, leading to insufficient generalization [20].…”

DLD-SLAM: RGB-D Visual Simultaneous Localisation and Mapping in Indoor Dynamic Environments Based on Deep Learning

“…A number of approaches have also emerged to improve the performance of SLAM algorithms by improving deep learning networks. Zhang, R [17] used ShuffleNetV2 to improve the YOLOv5 network. Meanwhile, to achieve semantic extraction in the environment, the segmentation head of the pyramid scene analysis network is added to the head of the YOLOv5 network, giving the improved YOLOv5 network both target detection and semantic segmentation capabilities.…”

AFO-SLAM: an improved visual SLAM in dynamic scenes using acceleration of feature extraction and object detection

“…Min et al [37] not only combined the semantic information obtained by YOLOv5 with epipolar geometry constraint, but also introduced blur filtering to solve the problem of blurred image motion caused by capturing rapidly moving objects. Zhang and Zhang [38] used ShuffleNetV2 to lighten the YOLOv5 network, thereby increasing network speed without compromising system accuracy. Song et al [39] employed the newer and faster YOLOv7 network to extract semantic information, which they closely integrated with geometric information to filter dynamic points, achieving high localization accuracy and robustness in both high and low dynamic environments.…”