Balanced Depth Completion between Dense Depth Inference and Sparse Range Measurements via KISS-GP

Abstract

Estimating a dense and accurate depth map is the key requirement for autonomous driving and robotics. Recent advances in deep learning have allowed depth estimation in full resolution from a single image. Despite this impressive result, many deep-learning-based monocular depth estimation (MDE) algorithms have failed to keep their accuracy yielding a meter-level estimation error. In many robotics applications, accurate but sparse measurements are readily available from Light Detection and Ranging (LiDAR). Although they are highly accurate, the sparsity limits full resolution depth map reconstruction. Targeting the problem of dense and accurate depth map recovery, this paper introduces the fusion of these two modalities as a depth completion (DC) problem by dividing the role of depth inference and depth regression. Utilizing the state-of-the-art MDE and our Gaussian process (GP) based depth-regression method, we propose a general solution that can flexibly work with various MDE modules by enhancing its depth with sparse range measurements. To overcome the major limitation of GP, we adopt Kernel Interpolation for Scalable Structured (KISS)-GP and mitigate the computational complexity from O(N 3 ) to O(N ). Our experiments demonstrate that the accuracy and robustness of our method outperform state-of-the-art unsupervised methods for sparse and biased measurements.

SelfDeco: Self-Supervised Monocular Depth Completion in Challenging Indoor Environments

“…Zhang et al [18] propose an end-to-end framework to jointly estimate pose and depth without the PnP method. Yoon et al [26] combine monocular depth estimation and Gaussian processbased depth regression for depth completion.…”

Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR

“…Most depth completion models [23,24,25,26,27,28] are trained with labeled data which require intensive human labors. To utilize the massive unlabeled data, self-supervised depth completion methods were developed in recent years [29,30,31,32,33] to generate depth maps from 64-beams dense LiDAR points. Our proposed method is designed to predict depth maps from 4-beams sparse LiDAR points, however, with the generalizability, our method is also applicable for the depth completion task.…”

“…Most (over 95%) of the methods in the KITTI depth completion benchmark is under supervised training. Following other state-of-theart self-supervised methods [29,30,31,32,33], we test our model on the KITTI validation set, as shown in Table 6. By effectively utilizing the LiDAR features, our model outperforms all other self-supervised methods by a large gap with all the metrics demonstrating the generalizability of our proposed method.…”

“…Method RMSE iRMSE iMAE KISS-GP [33] 1593.37 27.98 2.36 Sparse to dense [30] 1342.33 4.28 1.64 DDP [32] 1310.03 --VOICED [31] 1230.85 We evaluate our method on the KITTI depth completion task validation set, comparing it to the state-of-the-art self-supervised methods. All metrics are the lower, the better.…”