Geometric Consistency for Self-Supervised End-to-End Visual Odometry

Abstract

With the success of deep learning based approaches in tackling challenging problems in computer vision, a wide range of deep architectures have recently been proposed for the task of visual odometry (VO) estimation. Most of these proposed solutions rely on supervision, which requires the acquisition of precise ground-truth camera pose information, collected using expensive motion capture systems or high-precision IMU/GPS sensor rigs. In this work, we propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a selfsupervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate. We also present a method of characterizing the uncertainty in VO estimates thus obtained. To evaluate our VO pipeline, we present exhaustive ablation studies that demonstrate the efficacy of end-to-end, self-supervised methodologies to train deep models for monocular VO. We show that leveraging concepts from geometry and incorporating them into the training of a recurrent neural network results in performance competitive to supervised deep VO methods.

Recurrent Neural Network for (Un-)Supervised Learning of Monocular Video Visual Odometry and Depth

“…However, such a pairwise photometric consistency constraint is very noisy due to illumination variation, low texture, occlusion, etc. Recently, Iyer et al [13] proposed a composite transformation constraint for self-supervised visual odometry learning. By combining the pairwise image reconstruction constraint with the composite transformation constraint, we propose a multi-view image reprojection constraint that is robust to noise and provides strong self-supervision for our multi-view depth and visual odometry learning.…”

Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry

“…( Seq 11,15), it bears large scale drift in complicated scenes (Seq 13,14,16,18,19). The requirement of sophisticate map initialization degrades its ability to handle situations such as high speeds (Seq 12, 17).…”

SelfSphNet: Motion Estimation of a Spherical Camera via Self-Supervised Learning

“…These approaches attempted to provide supervision signals to train their networks using pseudo-labels generated from unlabeled training data. To obtain the required supervision signals, [33] generated synthetic images by using an image warping, [34] geometrically constrained a transformation matrix along multiple frames, and [35] introduced pose consistency, which is unique to a cubemap projection. However, to the best of our knowledge, only a few studies concerned with the self-supervised motion estimation of spherical cameras, such as [35], have been reported to date.…”