Heuristics‐oriented overtaking decision making for autonomous vehicles using reinforcement learning

Sell

Journal of Field Robotics

et al. 2022

This paper proposes a reliable optimized sigmoid‐based path planning algorithm that ensures smooth, fast and safe overtaking maneuver, while maintaining the necessary safety distance. In the proposed method, the desired smoothness of trajectories, the changes in steering angle and the lateral acceleration are controlled in a robust way. This paper describes the simulations, and the confirming real‐world experiments, conducted using the autonomous shuttle iseAuto. Our results suggest that the sigmoid A‐star algorithm leads to a smoother and more reliable motion when compared to other two standard methods. Specifically, the abruptness of necessary steering angle changes is reduced by factor of 4, and approaching the level of an experienced driver‐like maneuver.

Section: Related Workmentioning

confidence: 99%

Practical path planning techniques in overtaking for autonomous shuttles

Sell

Journal of Field Robotics

et al. 2022

IET Intelligent Trans Sys

“…The STDP module encourages canceling the lane change decision by imposing the boundary to the control points of the Bézier curve. is set as the original lateral coordinate d t 0 as in (7) (width of the vehicle L w ). The purpose is that we hope that the ego vehicle at least maintains the current lateral position and suspends changing lanes.…”

Section: The Trajectory Boundary In D Directionmentioning

confidence: 99%

“…In the backward direction, the strategy and planning results of the ego vehicle also influence the future trajectory of other vehicles, causing the prediction results to vary. Therefore, the models of prediction like [3][4][5], the design of decision-making algorithms [6,7] and the development of planning schemes [8][9][10] should be fully incorporated to realize safe navigation in a dynamic environment.…”

Section: Introductionmentioning

confidence: 99%

Spatio‐temporal decision‐making and trajectory planning framework with flexible constraints in closed‐loop dynamic traffic

Zhang

Liu

et al. 2022

Driving in interactive dynamic traffic is a huge challenge for autonomous vehicles, especially for motion planning. The autonomous vehicle not only needs to predict the future states of the social vehicles to avoid a collision but also realizes the comfort and continuous plan. To deal with the problem, a spatio‐temporal decision‐making and motion planning framework with flexible constraints based on previous work is proposed. Improvements can be highlighted in three aspects. First, a neural network for trajectory prediction is trained and it is integrated into the framework, which can learn the social vehicles' reactions in the negotiation better and give more accurate predictions. Second, instead of setting the fixed local targets, the flexible‐constraint mechanism is introduced to increase the success rate of continuous planning, specifically, the local planning target is determined conditioned on the situation with an elastic factor. Third, two popular simulators with randomly disposed traffic participants are exploited, where vehicles are mutually influenced, more similar to real‐world traffic. The closed‐loop tests with zero crashes and the comparative experiments with traditional models demonstrate the superior performance and practical significance of the proposed framework.

“…Furthermore, a hierarchical motion controller is described to manage the lateral and longitudinal movements of the ego and surrounding vehicles. The upper-level contains two models, which are intelligent driver model (IDM) and minimize overall braking induced by lane changes (MOBIL) [20]. The lower-level focuses on regulating vehicle velocity and acceleration.…”

Section: Driving Environment and Control Modulementioning

confidence: 99%

“…The duration of one episode is 100s, and the simulation frequency is 20 Hz. The initial velocity of the surrounding vehicles is randomly chosen from [20,23] m/s, and their behaviors are manipulated by IDM and MOBIL. The next section will discuss these two models in detail.…”

Section: A Highway Driving Scenariomentioning

confidence: 99%

Decision-Making Strategy on Highway for Autonomous Vehicles Using Deep Reinforcement Learning

Liao

Tang

et al. 2020

IEEE Access

Self Cite

Autonomous driving is a promising technology to reduce traffic accidents and improve driving efficiency. In this work, a deep reinforcement learning (DRL)-enabled decision-making policy is constructed for autonomous vehicles to address the overtaking behaviors on the highway. First, a highway driving environment is founded, wherein the ego vehicle aims to pass through the surrounding vehicles with an efficient and safe maneuver. A hierarchical control framework is presented to control these vehicles, which indicates the upper-level manages the driving decisions, and the lower-level cares about the supervision of vehicle speed and acceleration. Then, the particular DRL method named dueling deep Qnetwork (DDQN) algorithm is applied to derive the highway decision-making strategy. The exhaustive calculative procedures of deep Q-network and DDQN algorithms are discussed and compared. Finally, a series of estimation simulation experiments are conducted to evaluate the effectiveness of the proposed highway decision-making policy. The advantages of the proposed framework in convergence rate and control performance are illuminated. Simulation results reveal that the DDQN-based overtaking policy could accomplish highway driving tasks efficiently and safely. INDEX TERMS Autonomous driving, decision-making, deep reinforcement learning, dueling deep Qnetwork, deep Q-learning, overtaking policy.