CarFree: Hassle-Free Object Detection Dataset Generation Using Carla Autonomous Driving Simulator

Abstract

For safe autonomous driving, deep neural network (DNN)-based perception systems play essential roles, where a vast amount of driving images should be manually collected and labeled with ground truth (GT) for training and validation purposes. After observing the manual GT generation’s high cost and unavoidable human errors, this study presents an open-source automatic GT generation tool, CarFree, based on the Carla autonomous driving simulator. By that, we aim to democratize the daunting task of (in particular) object detection dataset generation, which was only possible by big companies or institutes due to its high cost. CarFree comprises (i) a data extraction client that automatically collects relevant information from the Carla simulator’s server and (ii) a post-processing software that produces precise 2D bounding boxes of vehicles and pedestrians on the gathered driving images. Our evaluation results show that CarFree can generate a considerable amount of realistic driving images along with their GTs in a reasonable time. Moreover, using the synthesized training images with artificially made unusual weather and lighting conditions, which are difficult to obtain in real-world driving scenarios, CarFree significantly improves the object detection accuracy in the real world, particularly in the case of harsh environments. With CarFree, we expect its users to generate a variety of object detection datasets in hassle-free ways.

Representative Real-Time Dataset Generation Based on Automated Fault Injection and HIL Simulation for ML-Assisted Validation of Automotive Software Systems

“…By doing so, the real industrial conditions, such as influences, noise and uncertainties, are ignored [34]. In the context of autonomous driving, the Carla simulator has been used to generate object detection datasets in [35], including realistic driving images. The key feature of the generated dataset is the consideration of abnormal weather and lighting conditions, which improves the performance of the developed intelligent object detection system.…”

Investigation of collision estimation with vehicle and pedestrian using CARLA simulation software

“…Their work provided detailed analysis and advancements in these areas. In 2022, Jang et al [15] revolutionised object detection dataset generation, traditionally a costly task limited to large entities, through their CarFree system. The system integrated with the CARLA simulator and created accurate 2D bounding boxes for vehicles and pedestrians in driving images.…”

“…The study also utilised RViz for visualising LiDAR scans, joint movements, and RGB camera images generated by CARLA. Additionally, Jang et al [15] discuss the complexities involved in developing camera-based perception systems for autonomous vehicles. The authors highlight the need for a vast dataset of training images, which are usually collected and labelled through a labour-intensive and error-prone manual process.…”

An Urban Traffic Dataset Composed of Visible Images and Their Semantic Segmentation Generated by the CARLA Simulator

“…Its primary goal is to increase the quantity of available data to enhance the training of computer vision models in autonomous vehicles. The second initiative, named "CarFree" [10], focuses on generating datasets for object detection in autonomous vehicles using CARLA. A noteworthy aspect of this project is its consideration of factors such as weather conditions and the time of day, contributing to the creation of more realistic data for future training.…”