High-resolution LIDAR-based depth mapping using bilateral filter

Abstract

Abstract-High resolution depth-maps, obtained by upsampling sparse range data from a 3D-LIDAR, find applications in many fields ranging from sensory perception to semantic segmentation and object detection. Upsampling is often based on combining data from a monocular camera to compensate the lowresolution of a LIDAR. This paper, on the other hand, introduces a novel framework to obtain dense depth-map solely from a single LIDAR point cloud; which is a research direction that has been barely explored. The formulation behind the proposed depth-mapping process relies on local spatial interpolation, using sliding-window (mask) technique, and on the Bilateral Filter (BF) where the variable of interest, the distance from the sensor, is considered in the interpolation problem. In particular, the BF is conveniently modified to perform depth-map upsampling such that the edges (foreground-background discontinuities) are better preserved by means of a proposed method which influences the range-based weighting term. Other methods for spatial upsampling are discussed, evaluated and compared in terms of different error measures. This paper also researches the role of the mask's size in the performance of the implemented methods. Quantitative and qualitative results from experiments on the KITTI Database, using LIDAR point clouds only, show very satisfactory performance of the approach introduced in this work.

Semantically aware multilateral filter for depth upsampling in automotive LiDAR point clouds

“…Currently there seems to be no consensus on how to quantitatively evaluate the reconstruction of dense depth images coming from a LiDAR point cloud in automotive scenarios [12]. One possibly relevant dataset is the KITTI Stereo 2012 and Stereo 2015 [8] and [18] which uses dense depth maps from LiDAR as the ground truth to compare depth map reconstructions of various stereo algorithms.…”

“…Accumulated point clouds are projected onto the camera image and then all ambiguous image regions such as windows and fences are manually removed. Using an exhaustive search through the provided raw data, [12] have found a practical sub-set of the Stereo 2015 ground truth images that we will also use to quantitatively measure our dense depth map reconstructions against. The dataset consists of 100 original point clouds and 100 corresponding dense point clouds considered as ground truth.…”

“…We conducted the measurements using the provided MATLAB scripts in KITTI Stereo 2015 and obtained the accuracies which we compare to the work discussed in [12]. We are able to outperform the best algorithm in [12] by a significant margin. Interestingly, when we switch off the term f 3 in (8), again, we observe top performance.…”

“…The classifier we chose is a multi-stage cascade of weak decision trees. Once the models are trained, the detection of pedestrians is [12] 46.67% 30s ACF [20] 39.81% 0.2s performed on the combined RGB and our reconstructed depth images. In order to measure and compare our results to the literature, we uploaded the detected bounding boxes to the KITTI evaluation server where we obtained the results presented on a short extract in Table II.…”

“…We found out, however, that even though these resulting images look appealing to the eye, the actual values around object boundaries are far from their correct values. An adaptation to the bi-lateral filter approach has been suggested [12] where points within each local neighbourhood are clustered and only points belonging to the cluster with the closest distance Semantically aware multilateral filter for depth upsampling in automotive LiDAR point clouds Martin Dimitrievski, Peter Veelaert, and Wilfried Philips to the local point are considered for up-sampling. This technique has promising results, however it uses a very small clustering window which can miss the underlying semantics and the object structure.…”