Deep Auxiliary Learning for Visual Localization and Odometry

Abstract

Localization is an indispensable component of a robot's autonomy stack that enables it to determine where it is in the environment, essentially making it a precursor for any action execution or planning. Although convolutional neural networks have shown promising results for visual localization, they are still grossly outperformed by state-of-the-art local feature-based techniques. In this work, we propose VLocNet, a new convolutional neural network architecture for 6-DoF global pose regression and odometry estimation from consecutive monocular images. Our multitask model incorporates hard parameter sharing, thus being compact and enabling real-time inference, in addition to being end-to-end trainable. We propose a novel loss function that utilizes auxiliary learning to leverage relative pose information during training, thereby constraining the search space to obtain consistent pose estimates. We evaluate our proposed VLocNet on indoor as well as outdoor datasets and show that even our single task model exceeds the performance of state-of-the-art deep architectures for global localization, while achieving competitive performance for visual odometry estimation. Furthermore, we present extensive experimental evaluations utilizing our proposed Geometric Consistency Loss that show the effectiveness of multitask learning and demonstrate that our model is the first deep learning technique to be on par with, and in some cases outperforms state-of-theart SIFT-based approaches.

Understanding the Limitations of CNN-Based Absolute Camera Pose Regression

“…All three methods operate on image sequences and thus use more information compared to Den-seVLAD, which only uses a single image for localization. Still, DenseVLAD outperforms VLocNet [74].…”

“…The table also compares DenseVLAD and Den-seVLAD+Inter. against three sequence-based approaches, VLoc-Net [74], VLocNet++STL [53], and VLocNet++MTL [53]. All three directly fuse feature map responses from the previous time step t−1 into the CNN that predicts the pose at time t. VLocNet++MTL also integrates some form of higher-level scene understanding through semantic segmentation.…”

“…This embedding is then used to regress the camera pose in the scene. Existing approaches mainly differ in the underlying base architecture and the loss function used for training, e.g., using a weighted combination of position and orientation errors [11,30,76], geometric reprojection errors [29], or adding visual odometry constraints [11,53,74]. [50,78] extend the set of training images with synthetic data.…”

“…[12,28] also reason about the uncertainty of the estimated poses. Rather than using a single image, [11,19,53,74] propose methods based on localizing sequences of images.…”

DeepLocalization: Landmark-based Self-Localization with Deep Neural Networks

“…Kendall et al proposed PoseNet [22], a convolutional neural network that regresses the camera's 6 degrees of freedom pose relative to the scene from a single RGB image. A similar approach has been introduced by Valada et al with VLocNet [23]. The goal is to regress the global pose and simultaneously estimate the odometry between two frames.…”