AADS: Augmented autonomous driving simulation using data-driven algorithms

Abstract

Simulation systems have become an essential component in the development and validation of autonomous driving technologies. The prevailing state-of-the-art approach for simulation is to use game engines or high-fidelity computer graphics (CG) models to create driving scenarios. However, creating CG models and vehicle movements (a.k.a. the assets for simulation) remains a manual task that can be costly and time-consuming. In addition, the fidelity of CG images still lacks the richness and authenticity of real-world images and using these CG images for training leads to degraded performance.In this paper we present a novel approach to address these issues: Augmented Autonomous Driving Simulation (AADS). Our formulation augments real-world pictures with a simulated traffic flow to create photo-realistic simulation images and renderings. More specifically, we use LiDAR and cameras to scan street scenes. From the acquired trajectory data, we generate highly plausible traffic flows for cars and pedestrians and compose them into the background. The composite images can be re-synthesized with different viewpoints and sensor models (camera or LiDAR). The resulting images are photo-realistic, fully annotated, and ready for end-to-end training and testing of autonomous driving systems from perception to planning. We explain our system design and validate our algorithms with a number of autonomous driving tasks from detection to segmentation and predictions.Compared to traditional approaches, our method offers unmatched scalability and realism. Scalability is particularly important for AD simulation and we believe the complexity and diversity of the real world cannot be realistically captured in a virtual environment. Our augmented approach combines the flexibility of a virtual environment (e.g., vehicle movements) with the richness of the real world to allow effective simulation of any location on earth. Summary

Neural Network Generalization: The Impact of Camera Parameters

“…One method uses computer graphics to augment camera images, superimposing virtual objects of the target category onto the image [1], [23]. This approach is simpler than synthesizing entire scenes and improves generalization.…”

A Survey on Visual Traffic Simulation: Models, Evaluations, and Applications in Autonomous Driving

“…Recently, Li et al . [LPZ*19] have developed a simulation framework, AADS, which can augment real images with simulated traffic flows for generating realistic‐looking images. Using data from LiDAR and cameras, the framework can compose simulated traffic flows, based on actual vehicle trajectories, into the background.…”

“…Autonomous driving has the potential to revolutionize our transportation systems. However, recent failures in testing have emphasized the training of these automated machines in simulated environments before deploying them to the real world [BNP*18, LWL19, LPZ*19].…”

Deep Learning on Computational-Resource-Limited Platforms: A Survey

“…By contrast with traditional machine learning solutions, deep learning techniques are undergoing rapid development. Applications of deep learning involve information retrieval [4], natural language processing [5], human voice recognition [6], computer vision [7], anomaly detection [8], recommendation systems [9], bioinformatics [10], medicine [11,12], crop science [13], earth science [14], robotics [15][16][17][18], transportation engineering [19], communication technologies [20][21][22], and system simulation [23,24].…”