Self-Supervised Deep Depth Denoising

Abstract

Depth perception is considered an invaluable source of information for various vision tasks. However, depth maps acquired using consumer-level sensors still suffer from non-negligible noise. This fact has recently motivated researchers to exploit traditional filters, as well as the deep learning paradigm, in order to suppress the aforementioned non-uniform noise, while preserving geometric details. Despite the effort, deep depth denoising is still an open challenge mainly due to the lack of clean data that could be used as ground truth. In this paper, we propose a fully convolutional deep autoencoder that learns to denoise depth maps, surpassing the lack of ground truth data. Specifically, the proposed autoencoder exploits multiple views of the same scene from different points of view in order to learn to suppress noise in a self-supervised endto-end manner using depth and color information during training, yet only depth during inference. To enforce selfsupervision, we leverage a differentiable rendering technique to exploit photometric supervision, which is further regularized using geometric and surface priors. As the proposed approach relies on raw data acquisition, a large RGB-D corpus is collected using Intel RealSense sensors. Complementary to a quantitative evaluation, we demonstrate the effectiveness of the proposed self-supervised denoising approach on established 3D reconstruction applications. Code is avalable at https://github.com/ VCL3D/DeepDepthDenoising

Feature Pyramid Network Based Efficient Normal Estimation and Filtering for Time-of-Flight Depth Cameras

“…For these raw depth images, we added Gaussian noise with a standard deviation of 5 cm and 10 cm. Then, we ran two traditional denoiser, PCL-SOR [ 50 ], and MatLab denoise function, in addition to some learning-based methods, PointCleanNet [ 44 ] and DeepDepthDenoising (DDD) [ 54 ], to obtain the predictions.…”

“…We also analyzed the runtime of these methods. We gathered the testing results, and compared the performance of our method against the PCL-Sor [ 50 ], Matlab [ 55 ], PointCleanNet (or PCN) [ 44 ], and DDD [ 54 ] in Table 4 for the Chamfer distances, and in Table 5 for runtime comparison. Here, we note that although for the ground truth we used the PointCleanNet [ 44 ] noise removal module when we were comparing this method to others, we also performed the outlier removal module in tandem with the noise removal module.…”

High-quality indoor scene 3D reconstruction with RGB-D cameras: A brief review

“…In recent years, deep depth denoising techniques (e.g., Ref. [58]) which can better capture the global context of each scene have attracted more attention.…”

Image Denoising With Generative Adversarial Networks and its Application to Cell Image Enhancement

“…The task of image denoising is to recover the noisefree image x by removing the noise v from the given noisy image y. The existing denoising methods can be divided into two categories: image prior based methods [1]- [12], which obtain denoised image by processing the noisy image according to some prior knowledge about image; discriminative learning methods [13]- [24], which train a model to learn the mapping relationship from given noisy image to the denoised The associate editor coordinating the review of this manuscript and approving it for publication was Min Xia .…”