Deep, Consistent Behavioral Decision Making with Planning Features for Autonomous Vehicles

Abstract

Autonomous driving promises to be the main trend in the future intelligent transportation systems due to its potentiality for energy saving, and traffic and safety improvements. However, traditional autonomous vehicles’ behavioral decisions face consistency issues between behavioral decision and trajectory planning and shows a strong dependence on the human experience. In this paper, we present a planning-feature-based deep behavior decision method (PFBD) for autonomous driving in complex, dynamic traffic. We used a deep reinforcement learning (DRL) learning framework with the twin delayed deep deterministic policy gradient algorithm (TD3) to exploit the optimal policy. We took into account the features of topological routes in the decision making of autonomous vehicles, through which consistency between decision making and path planning layers can be guaranteed. Specifically, the features of a route extracted from path planning space are shared as the input states for the behavioral decision. The actor-network learns a near-optimal policy from the feasible and safe candidate emulated routes. Simulation tests on three typical scenarios have been performed to demonstrate the performance of the learning policy, including the comparison with a traditional rule-based expert algorithm and the comparison with the policy considering partial information of a contour. The results show that the proposed approach can achieve better decisions. Real-time test on an HQ3 (HongQi the third ) autonomous vehicle also validated the effectiveness of PFBD.

Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

“…These researches usually use microscopic simulations for the environment, such as Vissim [23], Udacity, [24], SUMO [25], or several self-made models [26]. Though hybrid solutions exist, where the strategic and direct control meets [27], only a few papers deal with defining a path by some geometric approach an RL and then drive through it with a controller [28], [29]. For further information on the topic, a good review of the RL based approaches for vehicle motion planning can be found in [30].…”

Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles

“…Nevertheless, since there is a close coupling relationship between behavioral decision making and motion planning, in complex dynamic scenarios, there may be conflicts between the two layers. In a previous work [34], Qian et al proposed a decision-making algorithm using the planning feature in the training process for performance improvement. However, the planning feature used cannot completely represent the motion capability of the vehicles.…”

Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey

“…Two distinct coordination graph models, identity-based dynamic coordination and positionbased dynamic coordination, are studied in that work. Qian et al [123] described autonomous driving from a different perspective using twin delayed DDPG [155]. They proposed a two-level strategy to fill the gap between decision making and future planning of autonomous vehicle.…”