Decision making of autonomous vehicles in lane change scenarios: Deep reinforcement learning approaches with risk awareness

Li, Guofa; Yang, Yifan; Shen, Li; Qu, Xingda; Lyu, Nengchao; Li, Shengbo Eben

doi:10.1016/j.trc.2021.103452

Cited by 134 publications

(58 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One popular paradigm is the lateral decision making schemes with the deep Qnetwork (DQN) or its variants. A lane change decision-making framework for autonomous vehicles is developed to learn risk sensitive driving policies using risk-awareness prioritized replay DQN in [12]. A lane change decision making method is presented for autonomous vehicles through DQN with safety verification in [19].…”

Section: A Reinforcement Learning Based Lateral Decision Making For A...mentioning

confidence: 99%

“…While existing RL based decision making methods of autonomous vehicles have achieved many compelling results [10], [11], [12], [13], the real-world driving tasks involve unavoidable measurement errors or sensor noises which may mislead an autonomous vehicle into making suboptimal decisions, even cause catastrophic failures. In light of these risks, autonomous vehicles are required to ensure that their decision making systems can handle the natural observation uncertainties from sensing and perception system, especially adversarial perturbations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robust Lane Change Decision Making for Autonomous Vehicles: An Observation Adversarial Reinforcement Learning Approach

Yang

et al. 2023

IEEE Trans. Intell. Veh.

View full text Add to dashboard Cite

Section: A Reinforcement Learning Based Lateral Decision Making For A...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Robust Lane Change Decision Making for Autonomous Vehicles: An Observation Adversarial Reinforcement Learning Approach

Yang

et al. 2023

IEEE Trans. Intell. Veh.

View full text Add to dashboard Cite

“…The corresponding reward and penalty functions were designed to curb speeding, encourage high-speed driving, and punish low-speed blocking for these three-speed ranges. In order to make CAVs faster and more stable to explore the optimal speed, we used the exponential function to design a soft reward function [28].…”

Section: Parameters Incentive Punishedmentioning

confidence: 99%

Multi-Agent Decision-Making Modes in Uncertain Interactive Traffic Scenarios via Graph Convolution-Based Deep Reinforcement Learning

Gao

Liu

et al. 2022

Sensors

View full text Add to dashboard Cite

As one of the main elements of reinforcement learning, the design of the reward function is often not given enough attention when reinforcement learning is used in concrete applications, which leads to unsatisfactory performances. In this study, a reward function matrix is proposed for training various decision-making modes with emphasis on decision-making styles and further emphasis on incentives and punishments. Additionally, we model a traffic scene via graph model to better represent the interaction between vehicles, and adopt the graph convolutional network (GCN) to extract the features of the graph structure to help the connected autonomous vehicles perform decision-making directly. Furthermore, we combine GCN with deep Q-learning and multi-step double deep Q-learning to train four decision-making modes, which are named the graph convolutional deep Q-network (GQN) and the multi-step double graph convolutional deep Q-network (MDGQN). In the simulation, the superiority of the reward function matrix is proved by comparing it with the baseline, and evaluation metrics are proposed to verify the performance differences among decision-making modes. Results show that the trained decision-making modes can satisfy various driving requirements, including task completion rate, safety requirements, comfort level, and completion efficiency, by adjusting the weight values in the reward function matrix. Finally, the decision-making modes trained by MDGQN had better performance in an uncertain highway exit scene than those trained by GQN.

show abstract

“…Through RL, the network parameters of DL are tuned to continuously optimize their own behavior strategies; its framework is shown in Figure 2. This method has become a new research hotspot in the field of artificial intelligence and has been applied in fields such as robot control [30][31][32], autonomous driving [33], and machine vision [34][35][36][37][38][39][40][41]. In this paper, the vehicle autonomous driving decision problem is modeled with a partially observable Markov decision process (POMDP) [42], and the autonomous driving strategy optimization problem is solved by identifying the optimal driving strategy of the POMDP.…”

Section: Modeling Of the Automatic Driving Strategy Optimization Problemmentioning

confidence: 99%

APF-DPPO: An Automatic Driving Policy Learning Method Based on the Artificial Potential Field Method to Optimize the Reward Function

Lin

Zhang

et al. 2022

Machines

View full text Add to dashboard Cite

To address the difficulty of obtaining the optimal driving strategy under the condition of a complex environment and changeable tasks of vehicle autonomous driving, this paper proposes an end-to-end autonomous driving strategy learning method based on deep reinforcement learning. The ideas of target attraction and obstacle rejection of the artificial potential field method are introduced into the distributed proximal policy optimization algorithm, and the APF-DPPO learning model is established. To solve the range repulsion problem of the artificial potential field method, which affects the optimal driving strategy, this paper proposes a directional penalty function method that combines collision penalty and yaw penalty to convert the range penalty of obstacles into a single directional penalty, and establishes the vehicle motion collision model. Finally, the APF-DPPO learning model is selected to train the driving strategy for the virtual vehicle, and the transfer learning method is selected to verify the comparison experiment. The simulation results show that the completion rate of the virtual vehicle in the obstacle environment that generates penalty feedback is as high as 96.3%, which is 3.8% higher than the completion rate in the environment that does not generate penalty feedback. Under different reward functions, the method in this paper obtains the highest cumulative reward value within 500 s, which improves 69 points compared with the reward function method based on the artificial potential field method, and has higher adaptability and robustness in different environments. The experimental results show that this method can effectively improve the efficiency of autonomous driving strategy learning and control the virtual vehicle for autonomous driving behavior decisions, and provide reliable theoretical and technical support for real vehicles in autonomous driving decision-making.

show abstract

Decision making of autonomous vehicles in lane change scenarios: Deep reinforcement learning approaches with risk awareness

Cited by 134 publications

References 51 publications

Robust Lane Change Decision Making for Autonomous Vehicles: An Observation Adversarial Reinforcement Learning Approach

Robust Lane Change Decision Making for Autonomous Vehicles: An Observation Adversarial Reinforcement Learning Approach

Multi-Agent Decision-Making Modes in Uncertain Interactive Traffic Scenarios via Graph Convolution-Based Deep Reinforcement Learning

APF-DPPO: An Automatic Driving Policy Learning Method Based on the Artificial Potential Field Method to Optimize the Reward Function

Contact Info

Product

Resources

About