Guided Depth Enhancement via Anisotropic Diffusion

Liu, Junyi; Gong, Xiaojin

doi:10.1007/978-3-319-03731-8_38

Cited by 69 publications

(45 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, the refinement network, a diffusion model, recurrently refines plane-origin distance, which enforces the piecewise plane constraints and regularizes the depth completion. Compared with many previous works [21,2] that assume piecewise constant depth, our method utilizes the geometric constraints between depth and surface normal, and performs better and more stably in the missing regions. Finally, the refined depth can be obtained via the inverse transformation without losing accuracy when the refinement is finished.…”

Section: Methodsmentioning

confidence: 97%

“…Then, we transform the predicted depth and normal to the planeorigin distance space, and conduct a refinement process in this space via a diffusion model to enforce the geometric constraints. Compared with previous works [21,2] that model the depth variation in 2D space and assume piecewise constant depth, we model the geometric constraints in 3D space based on the assumption that 3D structures are constituted by piecewise planes and the plane-origin distances are therefore piecewise constant. The transformation to plane-origin distance enforces constraints between depth and normal during training, and improves the completion accuracy and stability in inference.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Depth Completion From Sparse LiDAR Data With Depth-Normal Constraints

Zhu

Shi

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

226

142

View full text Add to dashboard Cite

Depth completion aims to recover dense depth maps from sparse depth measurements. It is of increasing importance for autonomous driving and draws increasing attention from the vision community. Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises. In this paper, to regularize the depth completion and improve the robustness against noise, we propose a unified CNN framework that 1) models the geometric constraints between depth and surface normal in a diffusion module and 2) predicts the confidence of sparse Li-DAR measurements to mitigate the impact of noise. Specifically, our encoder-decoder backbone predicts surface normals, coarse depth and confidence of LiDAR inputs simultaneously, which are subsequently inputted into our diffusion refinement module to obtain the final completion results. Extensive experiments on KITTI depth completion dataset and NYU-Depth-V2 dataset demonstrate that our method achieves state-of-the-art performance. Further ablation study and analysis give more insights into the proposed method and demonstrate the generalization capability and stability of our model. AbstractSorry about this little trick and hope it would work, since I do not have much time to neatten my original source code. depth completion aims to recover dense depth maps from sparse depth measurements. It is of increasing importance for autonomous driving and draws increasing attention from the vision community. Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises. In this paper, to regularize the depth completion and improve the robustness against noise, we propose a unified CNN framework that 1) models the geometric constraints between depth and surface normal in a

show abstract

Section: Methodsmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

Depth Completion From Sparse LiDAR Data With Depth-Normal Constraints

Zhu

Shi

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

226

142

View full text Add to dashboard Cite

show abstract

“…Depth inpainting. Many methods have been proposed for filling holes in depth channels of RGB-D images, including ones that employ smoothness priors [30], fast marching methods [25,42], Navier-Stokes [6], anisotropic diffusion [41], background surface extrapolation [51,54,68], colordepth edge alignment [10,77,81], low-rank matrix completion [75], tensor voting [36], Mumford-Shah functional optimization [44], joint optimization with other properties of intrinsic images [4], and patch-based image synthesis [11,16,24]. Recently, methods have been proposed for inpainting color images with auto-encoders [70] and GAN architectures [58].…”

Section: Related Workmentioning

confidence: 99%

Deep Depth Completion of a Single RGB-D Image

Zhang

Funkhouser

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

370

336

View full text Add to dashboard Cite

The goal of our work is to complete the depth channel of an RGB-D image. Commodity-grade depth cameras often fail to sense depth for shiny, bright, transparent, and distant surfaces. To address this problem, we train a deep network that takes an RGB image as input and predicts dense surface normals and occlusion boundaries. Those predictions are then combined with raw depth observations provided by the RGB-D camera to solve for depths for all pixels, including those missing in the original observation. This method was chosen over others (e.g., inpainting depths directly) as the result of extensive experiments with a new depth completion benchmark dataset, where holes are filled in training data through the rendering of surface reconstructions created from multiview RGB-D scans. Experiments with different network inputs, depth representations, loss functions, optimization methods, inpainting methods, and deep depth estimation networks show that our proposed approach provides better depth completions than these alternatives. Depth representation:The obvious approach to address our problem is to use the new dataset as supervision to train a fully convolutional network to regress depth directly from RGB-D. However, that approach does not work very well, especially for large holes like the one shown in the bottom row of Figure 1. Estimating absolute depths from a monocular color image is difficult even for people [53]. Rather, we train the network to predict only local differential properties of depth (surface normals and occlusion boundaries), which are much easier to estimate [35]. We then solve for the absolute depths with a global optimization. Deep network design:There is no previous work on studying how best to design and train an end-to-end deep network for completing depth images from RGB-D inputs. At first glance, it seems straight-forward to extend previous net-1 arXiv:1803.09326v2 [cs.CV]

show abstract

“…We also compare against an Anisotropic Diffusion based approach to depth enhancement [25]. It must be noted that this method is independent of stereoscopic information and takes as its input a single image and a set of disparity points and generates a dense disparity image by diffusing these disparity points, using the input image as a guide.…”

Section: Results and Analysismentioning

confidence: 99%

Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements

Shivakumar

Mohta

Pfrommer

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We present an approach to depth estimation that fuses information from a stereo pair with sparse range measurements derived from a LIDAR sensor or a range camera. The goal of this work is to exploit the complementary strengths of the two sensor modalities, the accurate but sparse range measurements and the ambiguous but dense stereo information. These two sources are effectively and efficiently fused by combining ideas from anisotropic diffusion and semi-global matching.We evaluate our approach on the KITTI 2015 and Middlebury 2014 datasets, using randomly sampled ground truth range measurements as our sparse depth input. We achieve significant performance improvements with a small fraction of range measurements on both datasets. We also provide qualitative results from our platform using the PMDTec Monstar sensor. Our entire pipeline runs on an NVIDIA TX-2 platform at 5Hz on 1280×1024 stereo images with 128 disparity levels.

show abstract

Guided Depth Enhancement via Anisotropic Diffusion

Cited by 69 publications

References 19 publications

Depth Completion From Sparse LiDAR Data With Depth-Normal Constraints

Depth Completion From Sparse LiDAR Data With Depth-Normal Constraints

Deep Depth Completion of a Single RGB-D Image

Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements

Contact Info

Product

Resources

About