Deep Reinforcement Learning-Based Beam Tracking for Low-Latency Services in Vehicular Networks

Liu, Yan; Jiang, Zhiyuan; Zhang, Shunqing; Xu, Shugong

doi:10.1109/icc40277.2020.9148759

Cited by 27 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [77], the authors proposed a Deep Deterministic Policy Gradient (DDPG) based beam tracking approach which extracts information and hence achieves the URLLC requirements in typical V2X networks. It is shown that conventional EKF and PF-based [78] approaches performance in non-stationary channels are not satisfactory in terms of average packet latency due to overhead channel training and transmission failures, while a DRL-based approach can reduce the delay to about 6ms.…”

Section: Joint User Association and Beamformingmentioning

confidence: 99%

“…References Role Networking Dynamic spectrum access [56,57,58,59,60] Channel assignment and power allocation for V2V links [61,62] Channel assignment and power allocation for V2V links taking into account vehicle mobility [63,64,65,66] Transmission mode selection, channel assignment and power allocation for V2V links Collision management [68,69,70] Contention Window control, EIED, single-channel operation [71] Contention window control, LIED and EIED, single-channel operation [67] Contention window control, LIED and EIED, multi-channel operation Joint user association and beamforming [74] User association for load balancing with one access point [75,76] User association for load balancing with multiple access points [77] Beam tracking [79] Hand-off management Computing and caching Computation and data offloading [81,82,83,84,87] Find the optimal offloading strategy [80,85,86] Find the optimal offloading strategy and MEC server assignment Caching [88,89,90] Optimize content caching [91,92,93,94,95] Optimize caching, networking, and computing Energy Roadside units scheduling [96,100] Adaptively change Time Division Duplex (TDD) configuration…”

Section: Issuesmentioning

confidence: 99%

See 1 more Smart Citation

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Mekrache

Bradai

Moulay

et al. 2022

Vehicular Communications

View full text Add to dashboard Cite

Employing machine learning into 6G vehicular networks to support vehicular application services is being widely studied and a hot topic for the latest research works in the literature. This article provides a comprehensive review of research works that integrated reinforcement and deep reinforcement learning algorithms for vehicular networks management with an emphasis on vehicular telecommunications issues. Vehicular networks have become an important research area due to their specific features and applications such as standardization, efficient traffic management, road safety, and infotainment. In such networks, network entities need to make decisions to maximize network performance under uncertainty. To achieve this goal, Reinforcement Learning (RL) can effectively solve decision-making problems. However, the state and action spaces are massive and complex in large-scale wireless networks. Hence, RL may not be able to find the best strategy in a reasonable time. Therefore, Deep Reinforcement Learning (DRL) has been developed to combine RL with Deep Learning (DL) to overcome this issue. In this survey, we first present vehicular networks and give a brief overview of RL and DRL concepts. Then we review RL and especially DRL approaches to address emerging issues in 6G vehicular networks. We finally discuss and highlight some unresolved challenges for further study.

show abstract

Section: Joint User Association and Beamformingmentioning

confidence: 99%

Section: Issuesmentioning

confidence: 99%

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Mekrache

Bradai

Moulay

et al. 2022

Vehicular Communications

View full text Add to dashboard Cite

show abstract

“…This nonlinear relationship between UE movement and angular variation makes it hard to accurately track beam directions, especially for high-speed scenarios. To tackle this issue, Liu et al [4] proposed to use deep reinforcement learning (DRL) to fit the nonlinear variations of the LOS angle. Then, the beam tracking for a vehicle movement case was studied, which shows that the conventional extended Kalman filter based tracking is hard to converge, while the DRL based method can achieve high tracking accuracy.…”

Section: A Complex Nonlinear Propertymentioning

confidence: 99%

“…Firstly, mathematical tools generally rely on ideal assumptions, such as additional white Gaussian noise, which may not be consistent with practical scenarios. By contrast, DL could adaptively extract complex nonlinear features from raw data, such that the annoying nonlinearity could be marvelously transformed as reliable properties to facilitate beam management [3], [4]. Secondly, the massive parameters of DL models make it possible to 'remember' complicated high-dimensional features, such as blockage locations and shapes [5], which relieves the burden of manually extracting sophisticated features.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning for Beam-Management: State-of-the-Art, Opportunities and Challenges

Ma¹,

Tian²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Benefiting from huge bandwidth resources, millimeter-wave (mmWave) communications provide one of the most promising technologies for the fifth-generation wireless networks. To compensate for high pathloss of mmWave signals, large antenna arrays are equipped at base stations and user equipment to establish directional beamforming, where beam management is adopted to acquire and track the optimal beam pair with the maximum received power. Naturally, narrow beams are expected to achieve larger beamforming gain, whereas it could impose enormous training overhead and high sensitivity to blockages. Fortunately, the amazing success of deep learning (DL) has stimulated increasing interest in applying it to address those issues. In this article, we first elaborate the motivations of applying DL in beam management. Then, the current state-of-the-arts are reviewed, where their research routes and key features are discussed. Finally, challenges and future opportunities are summarized, highlighting DL design insights and novel beam management mechanisms. We hope this article can stimulate more striking ideas and exciting contributions for DL assisted beam management.

show abstract

“…However, the estimation of mmWave CSI may lead to huge overhead due to large antenna numbers. Furthermore, [6] utilized deep reinforcement learning to track the dominant path based on the selected beam pair. Nevertheless, the measurement of single beam pair is sensitive to noise and multipath interference.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Assisted Beam Prediction Using Out-of-Band Information

Zhao

wang

2020

2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring)

View full text Add to dashboard Cite

Huge overhead of beam training poses a significant challenge to mmWave communications. To address this issue, beam tracking has been widely investigated whereas existing methods are hard to handle serious multipath interference and non-stationary scenarios. Inspired by the spatial similarity between low-frequency and mmWave channels in non-standalone architectures, this paper proposes to utilize prior low-frequency information to predict the optimal mmWave beam, where deep learning is adopted to enhance the prediction accuracy. Specifically, periodically estimated low-frequency channel state information (CSI) is applied to track the movement of user equipment, and timing offset indicator is proposed to indicate the instant of mmWave beam training relative to low-frequency CSI estimation. Meanwhile, long-short term memory networks based dedicated models are designed to implement the prediction. Simulation results show that our proposed scheme can predict the optimal mmWave beam more accurately than the conventional methods while requiring little overhead of mmWave beam training.

show abstract

Deep Reinforcement Learning-Based Beam Tracking for Low-Latency Services in Vehicular Networks

Cited by 27 publications

References 13 publications

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G

Deep Learning for Beam-Management: State-of-the-Art, Opportunities and Challenges

Deep Learning Assisted Beam Prediction Using Out-of-Band Information

Contact Info

Product

Resources

About