Augmented LiDAR Simulator for Autonomous Driving

Abstract

In Autonomous Driving (AD), detection and tracking of obstacles on the roads is a critical task. Deep-learning based methods using annotated LiDAR data have been the most widely adopted approach for this. Unfortunately, annotating 3D point cloud is a very challenging, time-and money-consuming task. In this paper, we propose a novel LiDAR simulator that augments real point cloud with synthetic obstacles (e.g., cars, pedestrians, and other movable objects). Unlike previous simulators that entirely rely on CG models and game engines, our augmented simulator bypasses the requirement to create high-fidelity background CAD models. Instead, we can simply deploy a vehicle with a LiDAR scanner to sweep the street of interests to obtain the background point cloud, based on which annotated point cloud can be automatically generated. This unique "scanand-simulate" capability makes our approach scalable and practical, ready for large-scale industrial applications. In this paper, we describe our simulator in detail, in particular the placement of obstacles that is critical for performance enhancement. We show that detectors with our simulated LiDAR point cloud alone can perform comparably (within two percentage points) with these trained with real data. Mixing real and simulated data can achieve over 95% accuracy.

Development and Validation of LiDAR Sensor Simulators Based on Parallel Raycasting

“…It is necessary to have a platform where the user can build a scene based on the desired target. As seen earlier in this paper, most of these platforms are long-established simulators, especially in the automobilistic area, wherein the sensor simulator is created [ 4 , 18 , 23 , 24 , 25 , 26 , 27 , 28 ]. Here we propose a different approach: to develop or adapt a virtual scene builder so the user can build a scene and choose the sensor scanning locations.…”

“…A similar method was developed by Wang et al [ 18 ], where synthetic point clouds with semantic labels of detected objects were generated with a raycasting LiDAR simulator using CARLA [ 18 ] autonomous driving simulator, and later experiments show the effectiveness of mixed datasets for the deep learning method training. Fang et al [ 23 ], unlike previous simulators that entirely rely on Computer Graphics models and game engines, proposed a novel LiDAR simulator that augments real point clouds with synthetic obstacles (e.g., vehicles, pedestrians, and other movable objects). Manivasagam et al [ 24 ] built a large dataset of 3D static maps and 3D dynamic objects by scanning several cities with LiDAR.…”

“…During the period of this research, it was evident that the sensor simulators in the current literature were built from the perspective of the target problem, and this forces new research projects to develop the simulator from scratch [ 4 , 6 , 15 , 16 , 18 , 23 , 24 , 25 , 26 , 27 , 28 ]. So, the main collaborations of the present work to the field are: A guideline for LiDAR simulator development for generating synthetic dataset with semiautomatic annotation to easily adapt for different fields of research; The development of a sensor simulator based on both sensor models mentioned in [ 4 ], therefore called in this paper as sensor hybrid model [ 29 ]; and A method for calibration of the sensor simulator by comparison with a real sensor.…”

“…During the period of this research, it was evident that the sensor simulators in the current literature were built from the perspective of the target problem, and this forces new research projects to develop the simulator from scratch [4,6,15,16,18,[23][24][25][26][27][28]. So, the main collaborations of the present work to the field are:…”

Development of ADAS perception applications in ROS and "Software-In-the-Loop" validation with CARLA simulator

“…obstacles [8]. Based on this, modern day simulators like rFpro [9], LGSVL [10], AVS [11] by Uber, CARLA [12], DRIVE CONSTELLATION Simulator [13] by NVIDIA, Gazebo [14] and others are able to create very realistic scenes and complex layouts like road paint or road separation that can be difficult to discern even for humans.…”