A Traffic-Aware Federated Imitation Learning Framework for Motion Control at Unsignalized Intersections with Internet of Vehicles

Abstract

The wealth of data and the enhanced computation capabilities of Internet of Vehicles (IoV) enable the optimized motion control of vehicles passing through an intersection without traffic lights. However, more intersections and demands for privacy protection pose new challenges to motion control optimization. Federated Learning (FL) can protect privacy via model interaction in IoV, but traditional FL methods hardly deal with the transportation issue. To address the aforementioned issue, this study proposes a Traffic-Aware Federated Imitation learning framework for Motion Control (TAFI-MC), consisting of Vehicle Interactors (VIs), Edge Trainers (ETs), and a Cloud Aggregator (CA). An Imitation Learning (IL) algorithm is integrated into TAFI-MC to improve motion control. Furthermore, a loss-aware experience selection strategy is explored to reduce communication overhead between ETs and VIs. The experimental results show that the proposed TAFI-MC outperforms imitated rules in the respect of collision avoidance and driving comfort, and the experience selection strategy can reduce communication overheads while ensuring convergence.

T3OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Area

“…Therefore, T 3 OMVP scheme does not have to adopt the policy decoupling strategy. The following experiments show that, in the T 3 network without using policy decoupling can achieve the same benefits as adopting policy decoupling.…”

“…However, when the observations dimension and the number of actions are different, policy decoupling will make it difficult to assign appropriate actions, resulting in performance degradation even worse than original QMIX on 2v4 scenario. Furthermore, T 3 OMVP weakens the influence of observations dimension and number of actions, since the transformer combined with TT − observations endows the pursuing vehicle with more comprehensive information from all other vehicles, which avoids vehicles from falling into the local optimum situation, thereby, the pursuing efficiency can be improved in more scenarios. The purple line and dark blue line in Figure 5 indicate that VDN+UPDeT performs poorly on the OMVP scenario.…”

“…T 3 OMVP utilizes the -greedy strategy when the agent makes a decision. Specifically, the agent randomly chooses an unknown action with probability to explore or exploits with probability 1-by selecting the action with the largest Q-value among the existing actions.…”

“…The Elu can avoid the disappearance of the gradient, reduce the training time, and improve the accuracy in the neural network [19]. Similar to UPDeT, after obtaining the joint observation, T 3 OMVP feeds it into the transformer together with the hidden state. The transformer encoder consists of multiple blocks, each of which contains a multi-head self-attention, a feed-forward neural network, and two resnet structures.…”

“…The Internet of Vehicles (IoVs) is a typical application of Internet of Things (IoT) technologies in the Intelligent Transportation System (ITS) [1][2][3]. With the Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) communications, IoVs combined with Artificial Intelligence (AI) can improve the decision-making of vehicles and the efficiency of ITS [4,5].…”