Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated Driving

An, Hongil; Jung, Ji‐Hye

doi:10.3390/electronics8050543

Cited by 47 publications

(23 citation statements)

References 21 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AirSim, used by a recent research in [48], is a simulator initially developed for drones built on Unreal Engine now has a vehicle extension with different weather conditions and scenarios.…”

Section: B Simulatorsmentioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

305

View full text Add to dashboard Cite

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

Section: B Simulatorsmentioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

305

View full text Add to dashboard Cite

show abstract

“…In order to further improve the robustness, reference [34] used a probabilistic map represented as Gaussian distribution over remittance values instead of the previous ground map represented as fixed infrared remittance values. It enables the stationary objects and consistent angular reflectivity in the map to be quickly ~0.5 [23], [24] Localisation Expected millisecondlevel [3] Judgement Planning and decision making ~0.1-0.2 [25], [26] Reaction Execution ~0.1 [27], [28] identified by Bayesian inference. Then they used offline SLAM to align the overlapping trajectories in previous sequential map, which makes the localisation system keep learning and improving maps.…”

Section: A Lidar-based Localisationmentioning

confidence: 99%

Real-Time Performance-Focused Localization Techniques for Autonomous Vehicle: A Review

Lü

Smart

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Real-time, accurate, and robust localisation is critical for autonomous vehicles (AVs) to achieve safe, efficient driving, whilst real-time performance is essential for AVs to achieve their current position in time for decision making. To date, no review paper has quantitatively compared the real-time performance between different localisation techniques based on various hardware platforms and programming languages and analysed the relations among localisation methodologies, real-time performance and accuracy. Therefore, this paper discusses the state-of-the-art localisation techniques and analyses their overall performance in AV application. For further analysis, this paper firstly proposes a localisation algorithm operations capability (LAOC)-based equivalent comparison method to compare the relative computational complexity of different localisation techniques; then, it comprehensively discusses the relations among methodologies, computational complexity, and accuracy. Analysis results show that the computational complexity of localisation approaches differs by a maximum of about times, whilst accuracy varies by about 100 times. Vision-and data fusion-based localisation techniques have about 2-5 times potential for improving accuracy compared with lidar-based localisation. Lidar-and vision-based localisation can reduce computational complexity by improving image registration method efficiency. Data fusion-based localisation can achieve better real-time performance compared with lidar-and vision-based localisation because each standalone sensor does not need to develop a complex algorithm to achieve its best localisation potential. Vehicle-toeverything (V2X) technology can improve positioning robustness. Finally, the potential solutions and future orientations of AVs' localisation based on the quantitative comparison results are discussed.

show abstract

“…The steering, throttle, foot brake, and handbrake were controlled using the image information obtained by the front camera, and the asynchronous advantage actor–critic (A3C) algorithm was used for training in the game World Rally Championship 6 (WRC6). In [ 27 ], An et al considered the position coordinates and speeds of the agent and other vehicles as the input to the state space, for training. In their study on the following behavior of two vehicles, Zhu et al selected the speed, relative distance, and relative speed of two vehicles in the state space [ 28 ].…”

Section: Related Workmentioning

confidence: 99%

End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm

Wang

et al. 2020

Sensors

View full text Add to dashboard Cite

Changing lanes while driving requires coordinating the lateral and longitudinal controls of a vehicle, considering its running state and the surrounding environment. Although the existing rule-based automated lane-changing method is simple, it is unsuitable for unpredictable scenarios encountered in practice. Therefore, using a deep deterministic policy gradient (DDPG) algorithm, we propose an end-to-end method for automated lane changing based on lidar data. The distance state information of the lane boundary and the surrounding vehicles obtained by the agent in a simulation environment is denoted as the state space for an automated lane-change problem based on reinforcement learning. The steering wheel angle and longitudinal acceleration are used as the action space, and both the state and action spaces are continuous. In terms of the reward function, avoiding collision and setting different expected lane-changing distances that represent different driving styles are considered for security, and the angular velocity of the steering wheel and jerk are considered for comfort. The minimum speed limit for lane changing and the control of the agent for a quick lane change are considered for efficiency. For a one-way two-lane road, a visual simulation environment scene is constructed using Pyglet. By comparing the lane-changing process tracks of two driving styles in a simplified traffic flow scene, we study the influence of driving style on the lane-changing process and lane-changing time. Through the training and adjustment of the combined lateral and longitudinal control of autonomous vehicles with different driving styles in complex traffic scenes, the vehicles could complete a series of driving tasks while considering driving-style differences. The experimental results show that autonomous vehicles can reflect the differences in the driving styles at the time of lane change at the same speed. Under the combined lateral and longitudinal control, the autonomous vehicles exhibit good robustness to different speeds and traffic density in different road sections. Thus, autonomous vehicles trained using the proposed method can learn an automated lane-changing policy while considering safety, comfort, and efficiency.

show abstract

Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated Driving

Cited by 47 publications

References 21 publications

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Real-Time Performance-Focused Localization Techniques for Autonomous Vehicle: A Review

End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm

Contact Info

Product

Resources

About