Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints

Wang, Junjie; Zhang, Qichao; Zhao, Dan; Chen, Yaran

doi:10.1109/ijcnn.2019.8852110

Cited by 107 publications

(39 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Udacity, used in [49], is a simulator that was built for Udacity's Self-Driving Car Nanodegree [50] provides various sensors, such as high quality rendered camera image LIDAR and Infrared information, and also has capabilities to model other traffic participants.…”

Section: B Simulatorsmentioning

confidence: 99%

“…[80] is to fed multiple consecutive traffic snapshots into the underlying CNN structure, which inherently extracts the velocity of the moving objects. Representing speed in grid cells is also possible in this setup, for that example can be found in [49], where the authors converted the traffic extracted from the Udacity simulator to the lane-based grid.…”

Section: E Observation Spacementioning

confidence: 99%

See 1 more Smart Citation

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

243

View full text Add to dashboard Cite

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

Section: B Simulatorsmentioning

confidence: 99%

Section: E Observation Spacementioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

243

View full text Add to dashboard Cite

show abstract

“…Another group of researches focus on strategic decisions, where the agent determines high-level actions, such as lane-change, follow, etc. These researches usually use microscopic simulations for the environment, such as Vissim [23], Udacity, [24], SUMO [25], or several self-made models [26]. Though hybrid solutions exist, where the strategic and direct control meets [27], only a few papers deal with defining a path by some geometric approach an RL and then drive through it with a controller [28], [29].…”

Section: A Related Workmentioning

confidence: 99%

Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

2020

View full text Add to dashboard Cite

Motion planning plays an essential role in designing self-driving functions for connected and autonomous vehicles. The methods need to provide a feasible trajectory for the vehicle to follow, fulfilling different requirements, such as safety, efficiency, and passenger comfort. In this area, algorithms must also meet strict real-time expectations, since, especially in an emergency, the decision time is limited, which raises a trade-off for the feasibility requirements. This article proposes a hierarchical path planning solution for evasive maneuvering, where a Twin Delayed DDPG reinforcement learning agent generates the parameters of a geometric path consisting of chlotoids and straight sections, and an underlying model predictive control loop fulfills the trajectory following tasks. The method is applied to the automotive double lane-change test, a common emergency situation, comparing its results with human drivers' performance using a dynamic simulation environment. Besides the test's standardized parameters, a broader range of topological layouts is chosen, both for the training and performance evaluation. The results show that the proposed method highly outperforms human drivers, especially in challenging situations, while meeting the computational requirements, as the pre-trained neural network and path generation algorithm can provide a solution in an instant, based on the experience gained during the training process.

show abstract

“…The model formulated a set of LC rules which mainly considered the lane utility and LC risk. Wang et al [3] proposed a reinforcement learning method based on the LC rules, which was used to provide support for autonomous driving decision. This model mainly considered the safety of self-driving vehicles.…”

Section: Related Workmentioning

confidence: 99%

“…Currently, many methods have been presented to predict LCD (e.g. rule-based [1]- [3], fuzzy reasoning [4]- [6], datadriven algorithm [7]- [10], etc.). However, these studies focused on modeling the surrounding traffic conditions on LC vehicles, and lacked consideration of DP and DS.…”

Section: Introductionmentioning

confidence: 99%

DP and DS-LCD: A New Lane Change Decision Model Coupling Driver’s Psychology and Driving Style

Tao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Lane-Changing Decision (LCD) behavior is complex, dangerous, and varied by the Driver's Psychology (DP) and Driving Style (DS). For lacking the consideration of DP and DS, the existing LCD models cannot predict the LCD process varying with the drivers and traffic conditions accurately. To deal with the problems, a new LCD model coupling DP and DS, named DP&DS-LCD, is put forward. In the model, a psychological field model is constructed to represent the DP effect on the scene vehicles. And then, K-means and K-Nearest Neighbor (KNN) algorithm are respectively adopted in learning and recognition phases to recognize the current driving style pattern. Finally, based on the DP and DS, the multi-Grained Cascade Forest (gcForest) algorithm is applied to predict the LCD behavior. In experiments, DP&DS-LCD is compared with other three LCD models by using the opening I-80 database from Next Generation Simulation project (NGSIM). And the results showed that the DP&DS-LCD model achieved the best performance. Therefore, the DP&DS-LCD model is effective and could provide support for the decision of autonomous vehicles by predicting the surrounding vehicles' Lane-Changing (LC) behavior. INDEX TERMS Autonomous vehicles, driver's psychology, driving style, lane change decision.

show abstract

Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints

Cited by 107 publications

References 22 publications

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

DP and DS-LCD: A New Lane Change Decision Model Coupling Driver’s Psychology and Driving Style

Contact Info

Product

Resources

About