DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation

Abstract

Monocular depth estimation is a challenging task that predicts the pixel-wise depth from a single 2D image. Current methods typically model this problem as a regression or classification task. We propose DiffusionDepth, a new approach that reformulates monocular depth estimation as a denoising diffusion process. It learns an iterative denoising process to 'denoise' random depth distribution into a depth map with the guidance of monocular visual conditions. The process is performed in the latent space encoded by a dedicated depth encoder and decoder. Instead of diffusing ground truth (GT) depth, the model learns to reverse the process of diffusing the refined depth of itself into random depth distribution. This self-diffusion formulation overcomes the difficulty of applying generative models to sparse GT depth scenarios. The proposed approach benefits this task by refining depth estimation step by step, which is superior for generating accurate and highly detailed depth maps. Experimental results on KITTI and NYU-Depth-V2 datasets suggest that a simple yet efficient diffusion approach could reach state-of-the-art performance in both indoor and outdoor scenarios with acceptable inference time. Codes are available through link 3 .

Self-Supervised Learning for Monocular Depth Estimation on Minimally Invasive Surgery Scenes

“…[42]- [44] extended diffusion model in image segmentation and Chen et al [21] leveraged diffusion model to generate detection box proposals. There are also attempts at applying diffusion models to MDE [23], [24], but they focus on a fully supervised setting. To alleviate the performance degradation induced by diffusing on sparse ground-truth, Saxena et al [23] and Duan et al [24] both proposed to diffuse on the denoised output of network to assist training.…”

“…There are also attempts at applying diffusion models to MDE [23], [24], but they focus on a fully supervised setting. To alleviate the performance degradation induced by diffusing on sparse ground-truth, Saxena et al [23] and Duan et al [24] both proposed to diffuse on the denoised output of network to assist training. Self-supervised MDE with the diffusion model is more challenging due to the lack of depth ground-truth.…”

“…The teacher model will be discarded once the training is completed. Compared with diffusion on the denoised output [23], [24], the pseudo ground-truth process is able to avoid the noisy denoised output at the early training stage from deteriorating the entire training process. As a by-product, the teacher model allows imposing a knowledge distillation loss on the denoised depth.…”

“…The implementation of diffusion is hard for self-supervised MDE due to the lack of depth groundtruth. To allow the diffusion, we utilize the self-diffusion [24] and the proposed pseudo ground-truth diffusion, respectively and find that the self-diffusion does not even make the model converge in the self-supervised setting (ID 2). By contrast, the pseudo ground-truth diffusion enables the model to achieve good performance (ID 3).…”

“…Diffusion models have attracted a widespread attention due to its efficacy in generative tasks [19], [20], detection [21] and segmentation [22]. More recently, Saxena et al [23] and Duan et al [24] applied the diffusion model to MDE in a full supervised setting. However, the lack of depth ground-truth in self-supervised MDE poses a severe challenge to the diffusion process that requires groundtruth in diffusion model.…”