Multimotion Visual Odometry (MVO): Simultaneous Estimation of Camera and Third-Party Motions

Abstract

Estimating motion from images is a well-studied problem in computer vision and robotics. Previous work has developed techniques to estimate the motion of a moving camera in a largely static environment (e.g., visual odometry) and to segment or track motions in a dynamic scene using known camera motions (e.g., multiple object tracking).It is more challenging to estimate the unknown motion of the camera and the dynamic scene simultaneously. Most previous work requires a priori object models (e.g., trackingby-detection), motion constraints (e.g., planar motion), or fails to estimate the full SE (3) motions of the scene (e.g., scene flow). While these approaches work well in specific application domains, they are not generalizable to unconstrained motions. This paper extends the traditional visual odometry (VO) pipeline to estimate the full SE (3) motion of both a stereo/RGB-D camera and the dynamic scene. This multimotion visual odometry (MVO) pipeline requires no a priori knowledge of the environment or the dynamic objects. Its performance is evaluated on a real-world dynamic dataset with ground truth for all motions from a motion capture system.

ClusterSLAM: A SLAM backend for simultaneous rigid body clustering and motion estimation

“…Alternatively, segmentation methods like Refs. [19][20][21] use tracked sparse features to perform motion consistency analysis and motion segmentation; dense approaches taking RGBD input [5][6][7][8]22] combine the registration residual of dense model alignment and geometric features for enhanced segmentation and tracking. Further techniques for dynamic SLAM are summarized in Ref.…”

“…Previous methods for motion segmentation are mainly based on subspace factorization techniques [24,25], statistical modeling, and sampling [26][27][28], epipolar or trilinear constraints [29,30], object or scene flow [14,17,31,32], energy minimization [20,33,34], and deep learning based instance-level detection [7,10,12,21,35] (i.e., tracking-by-detection). Our strategy for segmenting multiple instances differs from previous approaches.…”

“…A similar step as used in Ref. [20] was performed in the frontend to find associations and generate landmark tracklets.…”

The Oxford Multimotion Dataset: Multiple SE(3) Motions With Ground Truth

“…The utility of this dataset is demonstrated by using MVO [7] to estimate the motions in the unconstrained SE (3) occlusion segment ( Figs. 7 and 8).…”

“…Isack and Boykov [22] apply an α-expansion and model-refitting framework to segment motions by first sampling the data to estimate a large number of motion models (similarly to [20]) and then refining the models and segmentation. Judd et al [7] introduce Multimotion Visual Odometry (MVO), which uses energy minimization to extend the traditional VO pipeline to simultaneously segment and estimate the SE (3) trajectory of every motion in the scene.…”