Semantically aware multilateral filter for depth upsampling in automotive LiDAR point clouds

Abstract

Abstract-We present a novel technique for fast and accurate reconstruction of depth images from 3D point clouds acquired in urban and rural driving environments. Our approach focuses entirely on the sparse distance and reflectance measurements generated by a LiDAR sensor. The main contribution of this paper is a combined segmentation and upsampling technique that preserves the important semantical structure of the scene. Data from the point cloud is segmented and projected onto a virtual camera image where a series of image processing steps are applied in order to reconstruct a fully sampled depth image. We achieve this by means of a multilateral filter that is guided into regions of distinct objects in the segmented point cloud. Thus, the gains of the proposed approach are two-fold: measurement noise in the original data is suppressed and missing depth values are reconstructed to arbitrary resolution. Objective evaluation in an automotive application shows state-of-the-art accuracy of our reconstructed depth images. Finally, we show the qualitative value of our images by training and evaluating a RGBD pedestrian detection system. By reinforcing the RGB pixels with our reconstructed depth values in the learning stage, a significant increase in detection rates can be realized while the model complexity remains comparable to the baseline.

Double-Threshold Segmentation of Panicle and Clustering Adaptive Density Estimation for Mature Rice Plants Based on 3D Point Cloud

“…Due to the variable environment, inherent error of the equipment, operating error of personnel and other factors, the acquired point-cloud data will inevitably produce extreme points. As usual, extreme points are far away from the main part of target point cloud so that they are also called noise points, leading to the inaccurate segmentation of cloud data [24]. Therefore, statistical outlier removal (SOR) algorithm was adopted to remove the noise points ahead of the segmentation process.…”

Weakly Supervised Deep Learning Method for Vulnerable Road User Detection in FMCW Radar

“…The lidar observation Z l is a set of objects which we compute by segmenting the point cloud into disjoint objects, Z l = {l 1 ,l 2 , ...,l j } using [14]. Each object is a vector consisting of its ground plane center and a unique instance identifier:…”

Learning Morphological Operators for Depth Completion

“…Following the success of the bi-lateral filter, we have proposed a semantically aware multi-lateral filter, [1] that is guided by a segmentation image. The segmentation image is computed by segmenting the LiDAR point cloud in a pre-processing step and is independent on the filtering window size.…”

“…A non-sparse data cube consisting of reconstructed depth pixels, middle image on figure 1, can be easily interpreted by classical computer vision algorithms. We have shown that pedestrian detection, in particular, can achieve much higher performance when operating on a RGB-D data reconstructed from a camera-LiDAR pair [1]. Even though many of the proposed depth completion methods produce dense and visually pleasing depth images, the depth completion problem is not entirely solved.…”

“…Even though, stereo reconstruction has been widely researched and many high performance solutions do exist , e.g. close to 100 submissions on the KITTI Stereo 2015 benchmark 1 , it still suffers from the effect of measurement correlated noise. As the 3D point is further away from the sensor, the perceived image disparity between the stereo cameras drops exponentially and so does the accuracy of it's estimate.…”