Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection

Mokhtari, Kasra; Wagner, Alan R.

doi:10.48550/arxiv.2106.04561

Cited by 3 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, experiments were conducted to verify the effectiveness of the proposed driving behavior decision-making method, compared with DQN [14], the combination of DQN and priority experience replay method (Prioritized-DQN) [28], DDQN [36], and D3QN [35]. Firstly, the environment and parameter settings are described.…”

Section: Simulation Results and Discussionmentioning

confidence: 99%

“…Then, the module is integrated into the dueling double deep Q network (D3QN) to make safer and more efficient decisions for autonomous driving [35]. Mokhtari et al [36] utilized two long-term short-term memory (LSTM) models based on a double deep Q network (DDQN) and the priority experience replay method to reconstruct the perception state of the environment and the future trajectories of pedestrians.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Collision Relationship-Based Driving Behavior Decision-Making Method for an Intelligent Land Vehicle at a Disorderly Intersection via DRQN

Huo

et al. 2022

Sensors

View full text Add to dashboard Cite

An intelligent land vehicle utilizes onboard sensors to acquire observed states at a disorderly intersection. However, partial observation of the environment occurs due to sensor noise. This causes decision failure easily. A collision relationship-based driving behavior decision-making method via deep recurrent Q network (CR-DRQN) is proposed for intelligent land vehicles. First, the collision relationship between the intelligent land vehicle and surrounding vehicles is designed as the input. The collision relationship is extracted from the observed states with the sensor noise. This avoids a CR-DRQN dimension explosion and speeds up the network training. Then, DRQN is utilized to attenuate the impact of the input noise and achieve driving behavior decision-making. Finally, some comparative experiments are conducted to verify the effectiveness of the proposed method. CR-DRQN maintains a high decision success rate at a disorderly intersection with partially observable states. In addition, the proposed method is outstanding in the aspects of safety, the ability of collision risk prediction, and comfort.

show abstract

Section: Simulation Results and Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

A Collision Relationship-Based Driving Behavior Decision-Making Method for an Intelligent Land Vehicle at a Disorderly Intersection via DRQN

Huo

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Lastly, DDQN was used in [133] to explore the task of navigating unsignalized intersections crowded with pedestrians. A belief update is employed to obtain the perceived state of the environment and the future trajectories of pedestrians given the noisy observations.…”

Section: Intersection Assistance Systemsmentioning

confidence: 99%

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

Khalil

Mouftah

2023

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

show abstract

“…In contrast, the learning-based method is more flexible in execution. The dominant method in learning-based AIM is multi-agent reinforcement learning (MARL), including Deep Q Learning (DQN) [39][40][41][42][43], Dueling Deep Q Learning (DDQN) [44], Deep Deterministic Policy Gradient (DDPG) [45], Proximal Policy Optimization (PPO) [46][47][48], Twin Delayed Deep Deterministic Policy Gradient (TD3) [49][50][51], and Soft Actor-Critic (SAC) [52]. Besides, DCL-AIM introduced coordinate state and independent state for CAVs to react in different scenarios [53], RAIM [54] and adv.RAIM [55] applied encoder-decoder structure with LSTM cell, AIM5LA further considered communication delay based on the adv.RAIM [56], and game theory was utilized to determine the leader-follower to enhance the performance of reinforcement learning in [52,57].…”

Section: Literature Reviewmentioning

confidence: 99%

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

Li¹,

Wu²,

He³

2023

Preprint

View full text Add to dashboard Cite

Autonomous Intersection Management (AIM) provides a signal-free intersection scheduling paradigm for Connected Autonomous Vehicles (CAVs). Distributed learning method has emerged as an attractive branch of AIM research. Compared with centralized AIM, distributed AIM can be deployed to CAVs at a lower cost, and compared with rule-based and optimization-based method, learningbased method can treat various complicated real-time intersection scenarios more flexibly. Deep reinforcement learning (DRL) is the mainstream approach in distributed learning to address AIM problems. However, the large-scale simultaneous interactive decision of multiple agents and the rapid changes of environment caused by interactions pose challenges for DRL, making its reward curve oscillating and hard to converge, and ultimately leading to a compromise in safety and computing efficiency. For this, we propose a non-RL learning framework, called Distributed Hierarchical Adversarial Learning (D-HAL). The framework includes an actor network that generates the actions of each CAV at each step. The immediate discriminator evaluates the interaction performance of the actor network at the current step, while the final discriminator makes the final evaluation of the overall trajectory from a series of interactions. In this framework, the long-term outcome of the behavior no longer motivates the actor network in terms of discounted rewards, but rather through a designed adversarial loss function with discriminative labels. The proposed model is evaluated at a four-way-six-lane intersection, and outperforms several state-of-the-art methods on ensuring safety and reducing travel time.

show abstract

Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection

Cited by 3 publications

References 18 publications

A Collision Relationship-Based Driving Behavior Decision-Making Method for an Intelligent Land Vehicle at a Disorderly Intersection via DRQN

A Collision Relationship-Based Driving Behavior Decision-Making Method for an Intelligent Land Vehicle at a Disorderly Intersection via DRQN

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

Contact Info

Product

Resources

About