Deep Convolutional Compressed Sensing for LiDAR Depth Completion

Abstract

In this paper we consider the problem of estimating a dense depth map from a set of sparse LiDAR points. We use techniques from compressed sensing and the recently developed Alternating Direction Neural Networks (ADNNs) to create a deep recurrent auto-encoder for this task. Our architecture internally performs an algorithm for extracting multi-level convolutional sparse codes from the input which are then used to make a prediction. Our results demonstrate that with only two layers and 1800 parameters we are able to out perform all previously published results, including deep networks with orders of magnitude more parameters.

To Complete or to Estimate, That is the Question: A Multi-Task Approach to Depth Completion and Monocular Depth Estimation

“…Error Our Approach 892 243 Table 2: Comparison of our depth completion approach against [10,15,16,39,50,54,55] using the validation set in [54]. Despite not being the primary focus, our completion approach remains competitive with the state of the art.…”

“…[16] proposes a constrained convolution operation from which confidence values are propagated through the network. A compressed sensing approach in [10] utilises a binary mask to filter out unmeasured values in a depth completion framework. The approach in [39] addresses depth completion by employing a self-supervised training procedure based on sequential RGB and sparse depth images.…”

“…Even though sparse depth completion is not the primary objective of this work, our approach is capable of generating dense depth from a sparse input along with its primary function (monocular depth estimation) and can outperform a variety of prior related work [10,16,40,50,54].…”

“…• Monocular depth estimation via adversarial training, a deep architecture with skip connections and a robust compound objective function directly supervised using this framework to outperform prior contemporary work [5,7,14,20,31,36,62,66]. • Sparse to dense depth completion via the same multitask model, capable of generating a dense depth output given a sparse depth input captured via a LiDAR sensor with results superior to prior contemporary work [10,16,40,50,54]. • Unique leverage of both synthetic [17] and real-world datasets [54] to ensure high-density complete depth outputs, despite such levels of density not existing in any real-world training images.…”

“…However, pervasive issues and complications ever-present in depth-reliant vision systems (e.g. missing depth, temporal and in-scene consistency, intensive computational and calibration requirements and alike), have led to the emergence of entire areas of research focusing on refinement procedures post estimation or capture [3,8,10,16,39,55] to render scene depth more useful for downstream applications.…”