Guided Conditional Diffusion for Controllable Traffic Simulation

Abstract

Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Typical heuristic-based traffic models offer flexible control to make vehicles follow specific trajectories and traffic rules. On the other hand, data-driven approaches generate realistic and human-like behaviors, improving transfer from simulated to real-world traffic. However, to the best of our knowledge, no traffic model offers both controllability and realism. In this work, we develop a conditional diffusion model for controllable traffic generation (CTG) that allows users to control desired properties of trajectories at test time (e.g., reach a goal or follow a speed limit) while maintaining realism and physical feasibility through enforced dynamics. The key technical idea is to leverage recent advances from diffusion modeling and differentiable logic to guide generated trajectories to meet rules defined using signal temporal logic (STL). We further extend guidance to multi-agent settings and enable interaction-based rules like collision avoidance. CTG is extensively evaluated on the nuScenes dataset for diverse and composite rules, demonstrating improvement over strong baselines in terms of the controllability-realism tradeoff.

Simulation-Based Validation for Autonomous Driving Systems

“…In addition to generating scenarios, many works use linear temporal logic [5,11,16], or signal temporal logic [1,27,33], or metric temporal logic [9] to describe traffic rules or safety properties for the validation of ADS. We also adopt linear temporal logic to describe traffic rules.…”

Wireless Edge-Empowered Metaverse: A Learning-Based Incentive Mechanism for Virtual Reality

“…The proposed data-driven data camera generation scheme demonstrates that the generated data can not only be visualized as high-quality data but also can be utilized as training datasets to improve the performance of AI algorithms in AVs. Furthermore, Zhong et al [21] propose a generative diffusion model to synthesize controllable and realistic traffic simulations in autonomous driving systems. However, these simulation platforms cannot synthesize controllable and realistic driving experiences and sensor data.…”

“…In detail, to improve reliability in DT task execution, we propose a multi-task DT offloading model where AVs can offload heterogeneous DT tasks with different deadlines to RSUs for real-time execution. To improve reliability in driving decision-making, virtual simulators can utilize the information in DTs, such as current location, historical trajectory, and user preferences, for online traffic simulations [21], [22]. Moreover, based on the collected sensing data in the physical world and user preferences in DTs, virtual simulators can synthesize massive and conditioned driving data for AV training of virtual simulators via running generative AI models.…”

Vehicle Type Specific Waypoint Generation

“…Current methods to multi-agent modeling approach the problem by jointly predicting future trajectories using deep probabilistic models such as conditional variational autoencoders (CVAEs) [12,[25][26][27]33], normalizing flows [18,21] and more recently diffusion models [31]. This family of multi-agent trajectory prediction models relies heavily on obtaining road context from HD maps, which requires manual annotations of lane center and boundary lines.…”