DAN‐Conv: Depth aware non‐local convolution for LiDAR depth completion

Abstract

Sparse LiDAR depth completion is a beneficial task for many robotic applications. It commonly generates a dense depth prediction from a sparse depth map and its corresponding aligned RGB image. This image-guided depth completion task mainly has two challenges: sparse data processing and multi-modality data fusion. In this letter, they are dealt with by two novel solutions: ( 1) To efficiently process sparse depth input, a Depth Aware Non-local Convolution (DAN-Conv) is proposed. It augments the spatial sampling locations of a convolution operation. Specifically, DAN-Conv constructs a non-local neighbourhood relationship by exploiting the intrinsic correlation among observable depth pixels. In particular, it can readily replace standard convolution without introducing additional network parameters. (2) A Symmetric Co-attention Module (SCM) is proposed to fuse and enhance features from depth and image domain. It estimates the importance of complementary features by the co-attention mechanism. Finally, a neural network built on DAN-Conv and SCM is proposed. It achieves competitive performance on the challenging KITTI depth completion benchmark. Comparing to approaches with approximate accuracy, this lightweight network requires significantly fewer learnable parameters.

NNNet: New Normal Guided Depth Completion From Sparse LiDAR Data and Single Color Image

“…RMSE and MAE are defined as follows: [41] 730.08 210.55 2.17 0.94 0.032s ACMNet [42] 732.99 206.80 2.08 0.90 0.080s FCFR-Net [43] 735.81 217.15 2.20 0.98 0.130s GuideNet [44] 736.24 218.83 2.25 0.99 0.140s NLSPN [47] 741.68 199.59 1.99 0.84 0.220s CSPN++ [45] 743.69 209.28 2.07 0.90 0.200s UberATG-FuseNet [48] 752.88 221.19 2.34 1.14 0.090s DenseLiDAR [49] 755.41 214.13 2.25 0.96 0.020s DeepLiDAR [10] 758.38 226.50 2.56 1.15 0.070s DANConv [50] 759.65 213.68 2.17 0.92 0.050s…”