Enabling Digital Twin in Vehicular Edge Computing: A Multi-Agent Multi-Objective Deep Reinforcement Learning Solution

Abstract

With recent advances in sensing technologies, wireless communications, and computing paradigms, traditional vehicles are evolving to electronic consumer products, driving the research on digital twins in vehicular edge computing (DT-VEC). This paper makes the first attempt to achieve the quality-cost tradeoff in DT-VEC. First, a DT-VEC architecture is presented, where the heterogeneous information can be sensed by vehicles and uploaded to the edge node via vehicle-to-infrastructure (V2I) communications. The DT-VEC are modeled at the edge node, forming a logical view to reflect the physical vehicular environment. Second, we model the DT-VEC by deriving an ISAC (integrated sensing and communication)-assisted sensing model and a reliability-guaranteed uploading model. Third, we define the quality of DT-VEC by considering the timeliness and consistency, and define the cost of DT-VEC by considering the redundancy, sensing cost, and transmission cost. Then, a biobjective problem is formulated to maximize the quality and minimize the cost. Fourth, we propose a multi-agent multiobjective (MAMO) deep reinforcement learning solution implemented distributedly in the vehicles and the edge nodes. Specifically, a dueling critic network is proposed to evaluate the advantage of action over the average of random actions. Finally, we give a comprehensive performance evaluation, demonstrating the superiority of the proposed MAMO.