In Defense of Classical Image Processing: Fast Depth Completion on the CPU

Abstract

With the rise of data driven deep neural networks as a realization of universal function approximators, most research on computer vision problems has moved away from hand crafted classical image processing algorithms. This paper shows that with a well designed algorithm, we are capable of outperforming neural network based methods on the task of depth completion. The proposed algorithm is simple and fast, runs on the CPU, and relies only on basic image processing operations to perform depth completion of sparse LIDAR depth data. We evaluate our algorithm on the challenging KITTI depth completion benchmark [1], and at the time of submission, our method ranks f irst on the KITTI test server among all published methods. Furthermore, our algorithm is data independent, requiring no training data to perform the task at hand. The code written in Python will be made publicly available at https://github.com/kujason/ip basic.

DFuseNet: Deep Fusion of RGB and Sparse Depth Information for Image Guided Dense Depth Completion

“…Ku et al propose a non-learning based approach to this problem to highlight the effectiveness of well crafted classical methods, using only commonly available morphological operations to produce dense depth information [14]. Their proposed method currently out-performs multiple deep learning based methods on the KITTI depth completion benchmark.…”

“…Our color and depth branches begin with an initial depth filling step, similar to the approach of Ku et al [14]. We use a simple sequence of morphological operations and Gaussian blurring operations to fill the holes in the sparse depth image with depth values from nearby valid points such that no holes remain.…”

“…Traditionally, interpolation and diffusion based schemes have been used to up-sample sparse points into a smooth dense depth image, often using the corresponding color image as a guide [1]. Convolutional neural networks (CNN) have had tremendous success in depth estimation tasks using monocular image data [2], [3], [4], [5], [6], [7], stereo image data [8], [9], [10], [11] and sparse depth data on it's own [12], [13], [14], [15].…”

Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction

“…Generating 3D Instance Training Data The sparse Li-DAR scan is first interpolated using [19] and converted into a 3 channel tensor J using the provided camera calibration, with each pixel representing a point (x, y, z). Points within ground truth boxes are considered as the set of points for an instance.…”

Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation

“…However the sparsity of LiDAR data only provides a low resolution perspective of the scene. We therefore take the portion of the LiDAR scan corresponding to the visible portion of the scene in the RGB image, and process it through a structure preserving depth completion algorithm [37]. In particular, the LiDAR points are projected into the image using the provided 3 × 4 camera projection matrix P cam creating a sparse depth map D s .…”