Dynamic Object-Aware Visual Odometry (VO) Estimation Based on Optical Flow Matching

Abstract

In this work, we propose a new visual odometry (VO) system that exploits the dynamic parts of an image. The key idea of our method is to identify the dynamic parts by combining semantic segmentation and optical flow and to suppress the dynamic parts in the process of VO estimation. First, movable objects are detected using the semantic segmentation. If an object contains many pixels of inconsistent optical flow, the object is considered as dynamic and merged with other dynamic objects to create a dynamic mask. Next, the ego-motion of the camera is estimated by using only the remaining static parts of the image and suppressing its dynamic parts. Unlike the other popular deep learning-based VO approaches, our method uses geometric approach based on optimization to achieve high performance. That is, the ego-motion of the camera is obtained by optimizing the correspondences obtained using the dynamic mask and optical flow consistency. Finally, the proposed method is applied to the KITTI Odometry benchmark dataset, and its performance is compared with that of the previous VO methods. Our method achieves an average of 7.67(%) translation error and 2.186 rotation error( • /100m), which imply the performance improvement by 21% and 11% from the state-of-the-art baseline, DF-VO, respectively. In addition, our method yields the RPE (Relative Pose Error) of 0.186m, which is the performance improvement by 52% against ORB-SLAM2, a popular geometry-based method. The experimental results show that the proposed VO method reliably predicts excellent visual odometry in the presence of dynamic objects.INDEX TERMS Visual odometry, dynamic environment, correspondences.

Self-supervised monocular visual odometry based on cross-correlation

“…ORB [22,23] efficiently combines the advantages of a fast segment test [24] and binary robust independent elementary features [25], providing strong features in terms of scale, rotation and brightness, making it one of the best real-time solutions currently available. Optical flow methods [26] exhibit point tracking characteristics while reducing computations to a certain extent. The Lucas-Kanade method [27] assumes brightness constancy and small motion between consecutive frames, estimating dense motion fields by solving linear equations in local neighborhoods.…”

Correlation Image Sensor-Assisted Single-Frame Optical Flow Estimation in Motion-Blurred Scenes

“…It can be generated by overlaying a transformed foreground object f k fg and a transformed background f k bg using a transformed foreground mask f k mask , as shown in (7). The transformed image (object, background or mask) can be obtained by (8). In this equation, f k refers to the kth transformed image(object, background, or mask).…”

“…O PTICAL flow is a two-dimensional velocity field that describes the apparent motion of image patterns. It has various applications in computer vision, including action recognition [1]- [3], video inpainting [4]- [6], and motion tracking [7], [8]. Despite the widespread use of twoframe methods [9]- [14], which use two consecutive frames in a video to estimate optical flow, these methods encounter limitations when dealing with scenes afflicted by motion blur.…”

SP-VO: RGB-D Visual Odometry Using Static Parts Toward Dynamic Environments

“…Kuo et al [27] proposed the framework that assigns an attention weight to semantic labels detected by mono camera, using an attention module based on neural network for pose estimation. Cho and Kim [28] utilized optical flow and semantic segmentation models to calculate the changes of objects that move within a scene, and generated a dynamic mask. Kim et al [29] proposed the SimVODIS++ network, which utilizes a self-supervised approach to select salient regions while excluding moving objects.…”