A Q-Learning-Based Approximate Solving Algorithm for Vehicular Route Game

Abstract

Route game is recognized as an effective method to alleviate Braess’ paradox, which generates a new traffic congestion since numerous vehicles obey the same guidance from the selfish route guidance (such as Google Maps). The conventional route games have symmetry since vehicles’ payoffs depend only on the selected route distribution but not who chose, which leads to the precise Nash equilibrium being able to be solved by constructing a special potential function. However, with the arrival of smart cities, the real-time of route schemes is more of a concerned of engineers than the absolute optimality in real traffic. It is not an easy task to re-construct the new potential functions of the route games due to the dynamic traffic conditions. In this paper, compared with the hard-solvable potential function-based precise method, a matched Q-learning algorithm is designed to generate the approximate Nash equilibrium of the classic route game for real-time traffic. An experimental study shows that the Nash equilibrium coefficients generated by the Q-learning-based approximate solving algorithm all converge to 1.00, and still have the required convergence in the different traffic parameters.

A unified model for the fairness mechanism‐based coordinated vehicle route guidance

“…In this section, similar to our previous works [51], the Q‐learning‐based universal algorithm is adopted to approximate the Nash equilibrium of the route game. A Q‐learning algorithm consists of four elements—reward function, optimization rules, state set, and action set.…”