Deformable Spatial Propagation Networks For Depth Completion

Xu, Zheyuan; Yin, Hongche; Yao, Jian

doi:10.1109/icip40778.2020.9191138

Cited by 57 publications

(33 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The state-of-the-art methods such as SPN [19], DSPN [6], CSPN [20], CSPN++ [21], and NLSPN [22] have mainly selected the spatial propagation strategy for depth completion. However, the proposed AGNet adopts attention learning to learn guidance information for depth regression from RGB image.…”

Section: Related Work a Depth Completionmentioning

confidence: 99%

“…Therefore, their predictions were affected by unwanted artifacts such as blurring and blending depth values. Since RGB images represent subtle changes in color and texture, many recent works [4]- [6] take deep learning as a computing method and exploit an additional synchronized RGB image for depth completion. Although they have achieved a significant improvement in performance [7]- [10], the ordinary CNN structure is actually…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AGNet: Attention Guided Sparse Depth Completion Using Convolutional Neural Networks

Liang

Jung

2022

IEEE Access

View full text Add to dashboard Cite

Sparse depth completion generates a dense depth image from its sparse measurement with the guidance of RGB image. In this paper, we propose attention guided sparse depth completion using convolutional neural networks, called AGNet. We adopt attention learning to get geometric cues for depth regression from RGB image and capture multi-scale depth structures. First, we use RGB image and valid binary mask from the input sparse depth image as input to generate an initial coarse depth image and its confidence map. Then, we generate attention map for depth refinement using a cross spatial attention module (CSAM). CSAM separately takes the RGB image with valid mask and invalid mask as input to make full use of color information in attention map. Next, we build a multi-scale learning network to encode the sparse depth image with different scales, thus leading to accurate depth completion. AGNet takes advantage of the input sparse depth image for encoding coarse features with a moderate model size. Experimental results show that AGNet achieves comparable performance with state-of-the-art methods for depth completion in NYU v2 dataset.

show abstract

Section: Related Work a Depth Completionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

AGNet: Attention Guided Sparse Depth Completion Using Convolutional Neural Networks

Liang

Jung

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Xu et al . [6] utilize Deformable ConvNets [5] to process depth map. It utilizes an auxiliary network to predict non‐local neighbours.…”

Section: Introductionmentioning

confidence: 99%

DAN‐Conv: Depth aware non‐local convolution for LiDAR depth completion

Lin

Gao

2021

Electronics Letters

View full text Add to dashboard Cite

Sparse LiDAR depth completion is a beneficial task for many robotic applications. It commonly generates a dense depth prediction from a sparse depth map and its corresponding aligned RGB image. This image-guided depth completion task mainly has two challenges: sparse data processing and multi-modality data fusion. In this letter, they are dealt with by two novel solutions: ( 1) To efficiently process sparse depth input, a Depth Aware Non-local Convolution (DAN-Conv) is proposed. It augments the spatial sampling locations of a convolution operation. Specifically, DAN-Conv constructs a non-local neighbourhood relationship by exploiting the intrinsic correlation among observable depth pixels. In particular, it can readily replace standard convolution without introducing additional network parameters. (2) A Symmetric Co-attention Module (SCM) is proposed to fuse and enhance features from depth and image domain. It estimates the importance of complementary features by the co-attention mechanism. Finally, a neural network built on DAN-Conv and SCM is proposed. It achieves competitive performance on the challenging KITTI depth completion benchmark. Comparing to approaches with approximate accuracy, this lightweight network requires significantly fewer learnable parameters.

show abstract

“…Lu et al [5] combined SLAM and depth estimation in an unsupervised way. CSPN [6] had a contentdependent module to improve the accuracy and DSPN [7] refined a deformable spatial propagation network based on CSPN. Tang et al [8] created a multi-task learning module to learn the grayscale image and depth map simultaneously.…”

Section: Introductionmentioning

confidence: 99%

Grayscale And Normal Guided Depth Completion With A Low-Cost Lidar

Chu

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

In this paper, we introduce DenseLivox, a dataset with dense and accurate depth as ground truth. To our best knowledge, it is the first dataset with dense ground truth designed for LiDAR depth completion using a low-cost LiDAR. Also, we develop a simple yet effective multi-task learning network to tackle the problem of depth completion. Compared to the works in the literature, our model's uniqueness is that it completes a depth map, a normal map, and a grayscale image simultaneously. To address the area with heavy noises, we use modified Huber loss to smooth these outliers' effect. We evaluate our method on DenseLivox and show that accuracy is greatly improved with the grayscale and normal guidance. Our method outperforms other depth-only methods and is comparable to the methods that take RGB and depth as input.

show abstract

Deformable Spatial Propagation Networks For Depth Completion

Cited by 57 publications

References 15 publications

AGNet: Attention Guided Sparse Depth Completion Using Convolutional Neural Networks

AGNet: Attention Guided Sparse Depth Completion Using Convolutional Neural Networks

DAN‐Conv: Depth aware non‐local convolution for LiDAR depth completion

Grayscale And Normal Guided Depth Completion With A Low-Cost Lidar

Contact Info

Product

Resources

About