Semantic SLAM for mobile robots in dynamic environments based on visual camera sensors

Abstract

Visual SLAM is constrained by a strong static world assumption,
making it hardly succeed when dynamic objects appear. Although some methods
eliminate dynamic objects by combining semantic and geometric information, the
fewer classes detected by the semantic method can not benefit these systems by
applying more scenes and constructing a richer semantic map. Our paper raises a
semantic visual SLAM system for a motion scene, which can run in real-time. We
use object detection to detect 80 classes in the scene and the moving consistency
checking to find outliers in every image. We propose different methods to check
the motion state of humans and other objects separately. For the detected human,
we propose an algorithm that judges if a person is sitting and divides the bounding
box belonging to the sitting person into two parts according to the proportion of
the human body. Then we use the same threshold as checking the boxes of the
standing person to determine the state of the two boxes belonging to the sitting
person, respectively. For other different objects except for humans, we propose an
algorithm that automatically adjusts the threshold of different bounding boxes.
This way, it can have the same detection performance for different objects. Finally,
we retain the static box points contained in the dynamic box and eliminate other
points in the dynamic box to benefit system performance from more detected
classes. We evaluated our SLAM on the TUM RGB-D dataset. The performance
of our SLAM exceeds most SLAM systems in a dynamic environment. Our system
is also tested in a real environment with a monocular camera to show its robustness
and universality.

A review of visual SLAM for robotics: evolution, properties, and future applications

“…Using these data types enhances and provides precise SLAM results ( End et al, 2012 ; Li Q. et al, 2022a ). It has the ability to create registered point clouds or OctoMaps for the purpose that can be used for robotic systems ( Zhang and Li 2023 ; Ren et al, 2022 ). In robotics applications, RGB-D SLAM, specifically V-SLAM, excels in both robustness and accuracy.…”

“…This challenge was tackled using TensorRT, optimized by YOLOX for high-precision real-time object recognition ( Chang et al, 2023 ; Martínez-Otzeta et al, 2022 ). It has versatile applications in real-world robotics scenarios, including autonomous driving cars, mobile robotics, and augmented reality ( Zhang and Li, 2023 ; Bahraini et al, 2018 ); see Table 2 .…”

Development of an automatic reading recognition system for liquid-in-glass thermometers based on deep learning

“…Multi-task learning methods [8][9][10][11][12][13][14] generally use multi-task networks to complete multiple related tasks (such as recognition [15,16], classification [17], detection [18][19][20][21], segmentation [22,23], denoising [24], etc). The multiple tasks are trained together to improve network performance by sharing associated features [25].…”

OMS-SLAM: dynamic scene visual SLAM based on object detection with multiple geometric feature constraints and statistical threshold segmentation

“…Subsequently, the dynamic status of the feature points is determined by combining epipolar geometry and dynamic weights, ultimately excluding any feature points identified as dynamic. In literature [42], the author initially employs object detection technology to investigate possible mobile objects in the environment. Then, they utilize the principle of dissected geometry to identify outliers, subsequently employing geometric techniques to ascertain human posture.…”