Confidence Propagation through CNNs for Guided Sparse Depth Regression

Abstract

Generally, convolutional neural networks (CNNs) process data on a regular grid, e.g. data generated by ordinary cameras. Designing CNNs for sparse and irregularly spaced input data is still an open research problem with numerous applications in autonomous driving, robotics, and surveillance. In this paper, we propose an algebraically-constrained normalized convolution layer for CNNs with highly sparse input that has a smaller number of network parameters compared to related work. We propose novel strategies for determining the confidence from the convolution operation and propagating it to consecutive layers. We also propose an objective function that simultaneously minimizes the data error while maximizing the output confidence. To integrate structural information, we also investigate fusion strategies to combine depth and RGB information in our normalized convolution network framework. In addition, we introduce the use of output confidence as an auxiliary information to improve the results. The capabilities of our normalized convolution network framework are demonstrated for the problem of scene depth completion. Comprehensive experiments are performed on the KITTI-Depth and the NYU-Depth-v2 datasets. The results clearly demonstrate that the proposed approach achieves superior performance while requiring only about 1-5% of the number of parameters compared to the state-of-the-art methods.

Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision

“…Deep Neural Networks (DNNs) have become the standard paradigm within most computer vision problems due to their astonishing predictive power compared to previous alternatives. Current applications include many safety-critical tasks, such as street-scene semantic segmentation [10,50,6,52], automotive 3D object detection [48,44,29,42,55] and depth completion [46,33,12]. Since erroneous predictions can have disastrous consequences, such applications require an accurate measure of the predictive uncertainty.…”

To Complete or to Estimate, That is the Question: A Multi-Task Approach to Depth Completion and Monocular Depth Estimation

“…Error Metrics (lower, better) [10,15,16,39,50,54,55] using the validation set in [54]. Despite not being the primary focus, our completion approach remains competitive with the state of the art.…”

“…Our sparse generator follows an encoder/decoder architecture with every layer containing modules of convolution, BatchNorm and leaky ReLU (slope = 0.2) with skip connections [46] between every pair of corresponding layers in the encoder and the decoder (Figure 2 -SG). The Figure 5: Comparing our depth completion results against [15,40,54,50]. The depth images have been adjusted for better visualization.…”

“…The depth images have been adjusted for better visualization. RGB: input colour image; LiDAR: sparse depth input; SIC: sparsity invariant CNN [54]; StD: sparse to dense completion [40]; DFN: deep fusion network [50]; GSR: guided sparse regression [15]; Dense: dense ground truth from [54].…”

S&CNet: Monocular Depth Completion for Autonomous Systems and 3D Reconstruction

“…However, their performance on the ranking metric (RMSE ) is much worse than other state-ofthe-art approaches. The approaches in [33,34] run much faster than other state-of-the-art methods, but their performance on metrics are much worse. Both the proposed method and the method in [17] achieve a good balance between accuracy and speed.…”