SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences

Abstract

Figure 1: Our dataset provides dense annotations for each scan of all sequences from the KITTI Odometry Benchmark [19]. Here, we show multiple scans aggregated using pose information estimated by a SLAM approach. AbstractSemantic scene understanding is important for various applications. In particular, self-driving cars need a finegrained understanding of the surfaces and objects in their vicinity. Light detection and ranging (LiDAR) provides precise geometric information about the environment and is thus a part of the sensor suites of almost all self-driving cars. Despite the relevance of semantic scene understanding for this application, there is a lack of a large dataset for this task which is based on an automotive LiDAR.In this paper, we introduce a large dataset to propel research on laser-based semantic segmentation. We annotated all sequences of the KITTI Vision Odometry Benchmark and provide dense point-wise annotations for the complete 360 o field-of-view of the employed automotive LiDAR. We propose three benchmark tasks based on this dataset: (i) semantic segmentation of point clouds using a single scan, (ii) semantic segmentation using multiple past scans, and (iii) semantic scene completion, which requires to anticipate the semantic scene in the future. We provide baseline experiments and show that there is a need for more sophisticated models to efficiently tackle these tasks. Our dataset opens the door for the development of more advanced methods, but also provides plentiful data to investigate new research directions. * indicates equal contribution

LiDAR-Based Multi-Task Road Perception Network for Autonomous Vehicles

“…To training and test our algorithm, we built a multi-task dataset named MultiRoad based on the SemanticKITTI dataset [33], which provides dense semantic annotations for each individual LiDAR scan of sequences 00-10 of the odometry task of the large-scale KITTI Vision Benchmark [2]. It enables the usage of multiple sequential scans for semantic scene interpretation, like semantic segmentation and semantic scene completion.…”

Semantic Point Cloud Mapping of LiDAR Based on Probabilistic Uncertainty Modeling for Autonomous Driving

“…In this paper, three deep neural networks are used; SqueezeSeg [ 25 ], SqueezeSegV2 [ 26 ], and RangeNet++ [ 8 ]. The reason why these networks are chosen is that they were demonstrated by the SemanticKITTI [ 41 ], however, it is not an important topic in this paper, because the main purpose of this paper is to consider the uncertainty of arbitrary semantic segmentation algorithm for semantic mapping. Those models project 3-D point cloud into 2-D image and apply 2-D convolutional neural network for semantic segmentation.…”

Learning-Based Human Segmentation and Velocity Estimation Using Automatic Labeled LiDAR Sequence for Training

“…Automatic Labeled LiDAR Data may cover demands in the frame domain; however, its application is not sufficient to address the requirements of the sequence domain. The SementicKITTI dataset [23] provides manually labeled segmentation information of the entire KITTI dataset [18]. However, the amount of KITTI data is limited and the labeling tasks need massive efforts.…”