DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networks

Abstract

Autonomous driving in multi-agent and dynamic traffic scenarios is challenging, where the behaviors of other road agents are uncertain and hard to model explicitly, and the egovehicle should apply complicated negotiation skills with them to achieve both safe and efficient driving in various settings, such as giving way, merging and taking turns. Traditional planning methods are largely rule-based and scale poorly in these complex dynamic scenarios, often leading to reactive or even overly conservative behaviors. Therefore, they require tedious human efforts to maintain workability. Recently, deep learning-based methods have shown promising results with better generalization capability but less hand engineering effort. However, they are either implemented with supervised imitation learning (IL) that suffers from the dataset bias and distribution mismatch problems, or trained with deep reinforcement learning (DRL) but focus on one specific traffic scenario. In this work, we propose DQ-GAT to achieve scalable and proactive autonomous driving, where graph attention-based networks are used to implicitly model interactions, and asynchronous deep Q-learning is employed to train the network end-to-end in an unsupervised manner. Extensive experiments through a high-fidelity driving simulation show that our method can better trade-off safety and efficiency in both seen and unseen scenarios, achieving higher goal success rates than the baselines (at most 4.7×) with comparable task completion time. Demonstration videos are available at https: //caipeide.github.io/dq-gat/.

Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation

“…Conditional imitation learning [2] is the most widely used framework, which learns to map raw sensor data, such as LIDAR data [15] and image data [5], to output control signals directly. To improve the generalization performance, semantic information (such as BEVs) [7], [8] is widely used as input of the end-to-end framework.…”

“…Our system input consists of three parts {B, C , N }, where B is the historical trajectory information of all vehicles in a local observation range, C is the scenarios information and N is the desired route information of ego-vehicle. Similar to [7], [8], the BEV information is considered. More specifically, we consider the historical trajectory information of n vehicles surround the ego vehicle, {x t i , y t i , vx t i , vy t i , θ t i }, where x t i and y t i denote the relative position to the ego vehicle in lateral and longitudinal directions respectively, vx t i and vy t i denote the relative velocity information, and θ t i denotes the relative heading.…”

“…Several researches aim to learn driving policy directly from the raw image data [5] and LIDAR point cloud [6], however, their performance in the complex scenarios cannot be ensured, as it is very difficult to learn the semantic relations among objects in complex traffic scenario (which is very important for robust navigation). Since the semantic information can provide informative abstraction of the perceptual results and can be easily accessed, it is commonly utilized in existing end-to-end driving models to improve transferability, examples include the birdeye-views (BEVs) [7], [8] and high definition (HD) maps [9]). Thanks to such semantic data and abstract vehicle states, the scalability to unseen scenarios can be easily achieved [8].…”