Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Abstract

Employing machine learning into 6G vehicular networks to support vehicular application services is being widely studied and a hot topic for the latest research works in the literature. This article provides a comprehensive review of research works that integrated reinforcement and deep reinforcement learning algorithms for vehicular networks management with an emphasis on vehicular telecommunications issues. Vehicular networks have become an important research area due to their specific features and applications such as standardization, efficient traffic management, road safety, and infotainment. In such networks, network entities need to make decisions to maximize network performance under uncertainty. To achieve this goal, Reinforcement Learning (RL) can effectively solve decision-making problems. However, the state and action spaces are massive and complex in large-scale wireless networks. Hence, RL may not be able to find the best strategy in a reasonable time. Therefore, Deep Reinforcement Learning (DRL) has been developed to combine RL with Deep Learning (DL) to overcome this issue. In this survey, we first present vehicular networks and give a brief overview of RL and DRL concepts. Then we review RL and especially DRL approaches to address emerging issues in 6G vehicular networks. We finally discuss and highlight some unresolved challenges for further study.

Toward the Age of Intelligent Vehicular Networks for Connected and Autonomous Vehicles in 6G

“…To satisfy KPIs and realize the vision of these futuristic applications, current vehicular networks require new enhancements to many underlying technologies. Inspired by the vision in [2]- [5], [7]- [9], the taxonomy of the enhancements to appear in 6G-enabled vehicular networks is summarized in Figure 4. Most technologies in the figure are still under development with various achievements, e.g., blockchain and massive MIMO.…”

“…By training hundreds of thousands of the state-action examples (e.g., sub-channel and power level selections, selected channels patterns of neighbors, the time used for transmission, etc), the Dueling Deep Q Networks (DDQN)-based learning agent on each vehicle can figure out the optimal resource allocation policy for transmission [7]. The intelligent resource allocation control is extremely meaningful for V2X dense/highthroughput applications (the case 2 of Figure 5).…”

Generation rate control with AoI under traffic hole problem in vehicular networks

“…6G network transmission can reach speeds of 1Tbps, allowing for transmission of multimedia data such as vehicle images for use in machine learning-related applications [13,12]. 6G networks allow for the introduction of the Internet of Things or other real-time application services, such as arti icial intelligence and big data computing applications [14]. With speeds up to 1Tbps and packet delays below 100𝜇/𝑠, 6G networks can meet quality of service guarantee requirements [15].…”

Authentication and Resource Allocation Strategies during Handoff for 5G IoVs Using Deep Learning

“…AI with Machine Learning (ML) and Deep Learning (DL) technology can assist 5G networks in anticipating and managing variable network traffic. Reinforcement Learning (RL), being a type of ML, can effectively solve decision-making problems [18].…”