Realtime multibody visual SLAM with a smoothly moving monocular camera

Abstract

This paper presents a realtime, incremental multibody visual SLAM system that allows choosing between full 3D reconstruction or simply tracking of the moving objects. Motion reconstruction of dynamic points or objects from a monocular camera is considered very hard due to well known problems of observability. We attempt to solve the problem with a Bearing only Tracking (BOT) and by integrating multiple cues to avoid observability issues. The BOT is accomplished through a particle filter, and by integrating multiple cues from the reconstruction pipeline. With the help of these cues, many real world scenarios which are considered unobservable with a monocular camera is solved to reasonable accuracy. This enables building of a unified dynamic 3D map of scenes involving multiple moving objects. Tracking and reconstruction is preceded by motion segmentation and detection which makes use of efficient geometric constraints to avoid difficult degenerate motions, where objects move in the epipolar plane. Results reported on multiple challenging real world image sequences verify the efficacy of the proposed framework.

Dynamic body VSLAM with semantic constraints

“…Generally, motion segmentation has been approached using geometric constraints [8] or by using affine trajectory clustering into subspaces [9]. In our approach we use motion along with semantic cues to segment the scene into static and dynamic objects, which allows us to work with fast moving cars, occlusions and disparity failure.…”

“…Traditional SLAM approaches with single motion model fail in such cases, as moving bodies cause reconstruction errors. Our approach employs Multi Body vSLAM framework [8] where we propose a novel trajectory optimization to with semantic constraints to show dense reconstruction results of moving objects.…”

TMAGIC: A Model-Free 3D Tracker

“…The use of the mean surface corresponds to a threshold such that there is an equal probability of false positives (a detection on the background was added to the feature set) and false negatives (a point on the object surface was rejected). If there is prior knowledge about the respective robustness of other components available, this can be exploited by adding the appropriate factor (multiple of σZ (α) ) to rZ (α) in Equation (17). It should be noted that as in parts of the previous section, the shape centre Y • is assumed to be the origin of the coordinate system, to keep the notation uncluttered.…”

“…In the area of visual object tracking, 3D based approaches must extract 3D shape and trajectory when objects are represented only by a minority of the video frame. For this reason, motion segmentation is sometimes used, such as to track and 1 http://cvssp.org/data/3DCars/ reconstruct moving bodies in the SLAM pipeline of Kundu et al [17].…”

Incremental real-time multibody VSLAM with trajectory optimization using stereo camera

“…Joint approach like [3] use monocular cameras to jointly estimate the depth maps, do motion segmentation and motion estimation of multiple bodies. Decoupled approaches like [4], [5] have a sequential pipeline where they segment motion and independently reconstruct the moving and static scenes. Our approach is a real-time incremental approach, and differs from the other methods due to the simultaneous optimization The top images depicts the real time segmentation of moving objects.…”