Receding-Horizon Reinforcement Learning Approach for Kinodynamic Motion Planning of Autonomous Vehicles

Zhang, Xinglong; Jiang, Yan; Lu, Yang; Xu, Xin

doi:10.1109/tiv.2022.3167271

Cited by 31 publications

(12 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Challenges lie in how to collect massive training samples that are consistent and how to guarantee learning efficiency (e.g., free from overfitting). Reinforcement-learning-based methods obtain knowledge by trial-and-error operations, thus they rely less on the quality and quantity of external training samples [168].…”

Section: B Local Behavior/trajectory Planningmentioning

confidence: 99%

Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys

Chen

Huang

et al. 2023

IEEE Trans. Intell. Veh.

218

View full text Add to dashboard Cite

Interest in autonomous driving (AD) and intelligent vehicles (IVs) is growing at a rapid pace due to the convenience, safety, and economic benefits. Although a number of surveys have reviewed research achievements in this field, they are still limited in specific tasks, lack of systematic summary and research directions in the future. Here we propose a Survey of Surveys (SoS) for total technologies of AD and IVs that reviews the history, summarizes the milestones, and provides the perspectives, ethics, and future research directions. To our knowledge, this article is the first SoS with milestones in AD and IVs, which constitutes our complete research work together with two other technical surveys. We anticipate that this article will bring novel and diverse insights to researchers and abecedarians, and serve as a bridge between past and future.

show abstract

Section: B Local Behavior/trajectory Planningmentioning

confidence: 99%

Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys

Chen

Huang

et al. 2023

IEEE Trans. Intell. Veh.

218

View full text Add to dashboard Cite

show abstract

“…In [12], the unknown disturbances are considered in the kinodynamic motion problem and solved by Pontryagin's Maximum Principle. Zhang et al [13] proposed a receding-horizon RL algorithm with an NN-based uncertain dynamics model for motion planning of intelligent vehicles, while it is generally difficult to collect a sufficient number of samples. In [14], a barrier transformation was incorporated into the RL algorithm for generating optimal control inputs online, but there may exist issues of slow convergence speed and difficulty in selecting appropriate parameters.…”

Section: Related Workmentioning

confidence: 99%

“…RL and ADP have received significant attention in recent years for solving optimal planning and control problems [9], [10]. Previous RL approaches in solving planning problems with uncertain dynamics employ the integration of Pontryagin's Maximum Principle to address model uncertainty and kinodynamic constraints [11], [12], incorporate model learning strategy and barrier function into the cost function to separately handle model uncertainty and collision constraints [13], [14], measure and select the optimal solution among the planning candidates to address the performance degradation caused by model uncertainty [15], [16]. RL approaches necessitate the use of approximators e.g., actor-critic networks to approximate the optimal policy and value function for continuous control tasks, where the feature representation capability greatly impacts learning efficiency and performance [17], [18].…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles

Yang

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Autonomous decision making and motion planning in complex dynamic traffic environments, such as left-turn without traffic signals and multi-lane merging from sideways , are still challenging tasks for intelligent vehicles. It is difficult to generate optimized behavior decisions while considering the motion capabilities and dynamic properties of intelligent vehicles. Aiming at the above problems, this paper proposes a hierarchical reinforcement learning approach for autonomous decision making and motion planning in complex dynamic traffic scenarios. The proposed approach consists of two layers. At the higher layer, a kernel-based least-squares policy iteration algorithm with uneven sampling and pooling strategy (USP-KLSPI) is presented for solving the decision-making problems. The motion capabilities of the ego vehicle and the surrounding vehicles are evaluated with a high-fidelity dynamic model in the decisionmaking layer. By doing so, the consistency between the decisions generated at the higher layer and the operations in the lower planning layer can be well guaranteed. The lower layer addresses the motionplanning problem in the lateral direction using a dual heuristic programming (DHP) algorithm learned in a batch-mode manner, while the velocity profile in the longitudinal direction is inherited from the higher layer. Extensive simulations are conducted in complex traffic conditions including left-turn without traffic signals and multi-lane merging from sideways scenarios. The results demonstrate the effectiveness and efficiency of the proposed approach in realizing optimized decision making and motion planning in complex environments. INDEX TERMS Autonomous driving, hierarchical reinforcement learning, complex dynamic traffics, decision making, motion planning.

show abstract

“…For many physical systems, accurate models play important roles in model-based planning, control, and trajectory forecasting [1][2][3][4]. Therefore, it is meaningful to develop data-driven modelling approaches for systems with unknown dynamics [5,6].…”

Section: Introductionmentioning

confidence: 99%

A deep Koopman operator‐based modelling approach for long‐term prediction of dynamics with pixel‐level measurements

Xiao

Tang

et al. 2023

CAAI Trans on Intel Tech

Self Cite

View full text Add to dashboard Cite

Although previous studies have made some clear leap in learning latent dynamics from high‐dimensional representations, the performances in terms of accuracy and inference time of long‐term model prediction still need to be improved. In this study, a deep convolutional network based on the Koopman operator (CKNet) is proposed to model non‐linear systems with pixel‐level measurements for long‐term prediction. CKNet adopts an autoencoder network architecture, consisting of an encoder to generate latent states and a linear dynamical model (i.e., the Koopman operator) which evolves in the latent state space spanned by the encoder. The decoder is used to recover images from latent states. According to a multi‐step ahead prediction loss function, the system matrices for approximating the Koopman operator are trained synchronously with the autoencoder in a mini‐batch manner. In this manner, gradients can be synchronously transmitted to both the system matrices and the autoencoder to help the encoder self‐adaptively tune the latent state space in the training process, and the resulting model is time‐invariant in the latent space. Therefore, the proposed CKNet has the advantages of less inference time and high accuracy for long‐term prediction. Experiments are performed on OpenAI Gym and Mujoco environments, including two and four non‐linear forced dynamical systems with continuous action spaces. The experimental results show that CKNet has strong long‐term prediction capabilities with sufficient precision.

show abstract

Receding-Horizon Reinforcement Learning Approach for Kinodynamic Motion Planning of Autonomous Vehicles

Cited by 31 publications

References 48 publications

Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys

Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys

Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles

A deep Koopman operator‐based modelling approach for long‐term prediction of dynamics with pixel‐level measurements

Contact Info

Product

Resources

About