A Vehicle Path Planning Algorithm Based on Mixed Policy Gradient Actor‐Critic Model with Random Escape Term and Filter Optimization

Abstract

The transportation system of those countries has a huge traffic flow is bearing great pressure on transportation planning and management. Vehicle path planning is one of the effective ways to alleviate such pressure. Deep reinforcement learning (DRL), as a state-of-the-art solution method in vehicle path planning, can better balance the ability and complexity of the algorithm to reflect the real situation. However, DRL has its own disadvantages of higher search cost and earlier convergence to the local optimum, as vehicle path planning issues are usually in a complex environment, and their action set can be diverse. In this paper, a mixed policy gradient actor-critic (AC) model with random escape term and filter operation is proposed, in which the policy weight is both data driven and model driven. The empirical data-driven method is used to improve the poor asymptotic performance, and the model-driven method is used to ensure the convergence speed of the whole model. At the same time, in order to avoid the model converging local optimum, a random escape term has been added in the policy weight update process to overcome the problem that it is difficult to optimize the non-convex loss function, and the random escape term can help to explore the policy in more directions. In addition, filter optimization has been innovatively introduced in this paper, and the step size of each iteration of the model is selected through the filter optimization algorithm to achieve the better iterative effect. Numerical experiment results have shown that the model proposed in this paper can not only improve the accuracy of the solution without losing the accuracy but also speed up the convergence speed and improve the utilization of data.

Urban Day-to-Day Travel and Its Development in an Information Environment: A Review

“…Moreover, the model has assigned different memory weights to different memory-forgetting durations based on human memory characteristics [67]. Nai et al have proposed a hybrid policy gradient-based actor-critic generative adversarial RL model to describe the route choice strategies and optimization methods for micro travelers [68]. Zhao et al have proposed a universal deep inverse reinforcement learning (IRL) framework for link-based route choice modeling, which combines different features of state, action, and travel context and captures the dynamic properties of micro route choice, achieving competitive interpretability of micro travel decision-making [69].…”

Population Game-Assisted Multi-Agent Reinforcement Learning Method for Dynamic Multi-Vehicle Route Selection

“…Li et al [34] proposed an DRL method based on an attention mechanism, which contains a vehicle selection decoder considering heterogeneous fleet constraints and a node selection decoder considering route construction. Nai et al [35] proposed a mixedstrategy gradient actor-critic model with a stochastic escape term and a filtering operation, using a model-driven approach to ensure the convergence speed of the whole model. Berat et al [36] proposed a synergistic combination of deep reinforcement learning and hierarchical game theory as a modelling framework for driver behaviour prediction in motorway driving scenarios.…”

Compressible Non-Newtonian Fluid Based Road Traffic Flow Equation Solved by Physical-Informed Rational Neural Network

“…Data has shown that China's urbanization process has been continuously accelerating, at the end of the year 2021, the urbanization rate of China's resident population has reached up to 64.72% [1]. In the expanding scale of large and medium-sized cities, limited urban transportation resources and increasing transportation demand have formed a significant supply-demand contradiction [2]. It is very important to achieve effective governance of urban transportation and solve congestion problems as much as possible, and such effective governance relies on a deep understanding of the rules of urban transportation travel.…”