Q-Learning-Based Low Complexity Beam Tracking for mmWave Beamforming System

Kim, Seonyong; Kwon, Girim; Park, Hyuncheol

doi:10.1109/ictc49870.2020.9289600

Cited by 6 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This motivated researchers in academia and industry to utilize ML tools to induce intelligence into existing processing blocks of wireless communication networks, e.g., at various network layers [31], [32], for channel estimation [33], [34], [35], for network resource optimization [36], and for wireless power transfer [37]. Motivated by some initial promising studies, ML has also been studied to resolve the BM and tracking problem [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70], [71], [72], [73], [74], [75], [76], …”

Section: B Machine Learning For Beam Managementmentioning

confidence: 99%

“…During the exploitation phase, the agent always selects optimal beams that maximize the received power. Another line of work combines Q-learning with an auxiliary beam pair [121] to further reduce the beam search space [69]. However, a fundamental limitation of Q-learning is that it requires several iterations before convergence, so that all state action pairs are explored, which limits its application to a fast moving UE.…”

Section: ) Q-learning and Deep Q-networkmentioning

confidence: 99%

“…However, FIGURE 8. Comparison of beam measurements overhead of the multi-codebook-based [39], [44], [45], [71] and subspace-reduction-based [40], [41], [43], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [68], [69], [73], [74], [75], [76], [77], [79], [80], [81] surveyed works against exhaustive beam sweeping. The number of UE beams is fixed to N = 16.…”

Section: Evaluation Of Machine-learning-based Millimeter Wave and Ter...mentioning

confidence: 99%

See 2 more Smart Citations

Machine Learning for Millimeter Wave and Terahertz Beam Management: A Survey and Open Challenges

Khan¹,

Gaber²,

Schulz

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Next-generation wireless communication networks will benefit from beamforming gain to utilize higher bandwidths at millimeter wave (mmWave) and terahertz (THz) bands. For high directional gain, a beam management (BM) framework acquires and tracks optimal downlink and uplink beam pairs through exhaustive beam scan. However, for narrower beams at higher carrier frequencies this leads to a huge beam measurement overhead that negatively impacts the beam acquisition and tracking. Moreover, volatility of mmWave and THz channels, user random mobility patterns, and environmental changes further complicate the BM process. Consequently, machine learning (ML) algorithms that can identify and learn complex mobility patterns and track environmental dynamics have been identified as a remedy. In this article, we provide an overview of the existing ML-based mmWave/THz BM and beam tracking techniques. Especially, we highlight key characteristics of an optimal BM and tracking framework. By surveying the recent studies, we identify some open research challenges and provide our recommendations that can serve as a future direction for researchers in this area.INDEX TERMS 6G, beam management (BM), beam tracking, federated learning (FL), machine learning (ML), millimeter wave (mmWave), reinforcement learning (RL), supervised learning (SL), terahertz (THz). I. INTRODUCTION5G New Radio is being deployed across the world, but in order to meet the requirements of extremely demanding applications like holographic video conferencing and extended reality, the industry and academia have already shifted their attention to the sixth generation (6G) of mobile communications systems. In order to support the multiplicity of use cases, 6G will have to provide much higher data rates, e.g., Tbps [1]. To enable high data rate communications, 5G benefits from higher bandwidths in the range of the millimeter wave (mmWave) spectrum, i.e., from 24.25 GHz to 52.6 GHz. It is now anticipated that 6G will stretch theseThe associate editor coordinating the review of this manuscript and approving it for publication was Olutayo O. Oyerinde . limits to terahertz (THz) bands (0.1 THz to 10 THz) [2], [3]. However, communication at higher frequency bands suffers from severe propagation losses. Consequently, directional communication with high beamforming gain is utilized to ensure coverage in mmWave bands. This necessitates the need of a sophisticated beam management (BM) framework at the transmitter (Tx) and the receiver (Rx) to carefully select an ideal beam pair for data transmission. A. PRELIMINARIES OF TRADITIONAL BEAM MANAGEMENTThe mmWave BM framework specified by the 3rd generation partnership project (3GPP) comprises three operations: initial beam establishment including beam sweeping, beam

show abstract

Section: B Machine Learning For Beam Managementmentioning

confidence: 99%

Section: ) Q-learning and Deep Q-networkmentioning

confidence: 99%

Section: Evaluation Of Machine-learning-based Millimeter Wave and Ter...mentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning for Millimeter Wave and Terahertz Beam Management: A Survey and Open Challenges

Khan¹,

Gaber²,

Schulz

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Besides, ABP is a model free tracking algorithm which doesn't rely on the angle variation model. ABP can be combined with Q-learning to further reduce the search space [133].…”

Section: B Side Information Aided Beam Trackingmentioning

confidence: 99%

Beam Training and Tracking in MmWave Communication: A Survey

Wang¹,

Wei²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

Communicating on millimeter wave (mmWave) bands is ushering in a new epoch of mobile communication which provides the availability of 10 Gbps high data rate transmission. However, mmWave links are easily prone to short transmission range communication because of the serious free space path loss and the blockage by obstacles. To overcome these challenges, highly directional beams are exploited to achieve robust links by hybrid beamforming. Accurately aligning the transmitter and receiver beams, i.e. beam training, is vitally important to high data rate transmission. However, it may cause huge overhead which has negative effects on initial access, handover, and tracking. Besides, the mobility patterns of users are complicated and dynamic, which may cause tracking error and large tracking latency. An efficient beam tracking method has a positive effect on sustaining robust links. This article provides an overview of the beam training and tracking technologies on mmWave bands and reveals the insights for future research in the 6th Generation (6G) mobile network. Especially, some open research problems are proposed to realize fast, accurate, and robust beam training and tracking. We hope that this survey provides guidelines for the researchers in the area of mmWave communications.

show abstract

“…Compared to the MAB model, it is more sophisticated to model the beam training as a Markov decision process, where Q-learning methods are typically employed. In [62], the estimation of beam steering angle is divided into two phases. In the first phase, Q-learning based beam tracking finds the optimal beam.…”

Section: Rl-based Approachesmentioning

confidence: 99%

Acquisition of channel state information for mmWave massive MIMO: traditional and machine learning-based approaches

Dong

et al. 2021

Sci. China Inf. Sci.

View full text Add to dashboard Cite

The accuracy of channel state information (CSI) acquisition directly affects the performance of millimeter wave (mmWave) communications. In this article, we provide an overview on CSI acquisition, including beam training and channel estimation for mmWave massive multiple-input multiple-output systems. The beam training can avoid the estimation of a high-dimension channel matrix, while the channel estimation can flexibly exploit advanced signal processing techniques. In addition to introducing the traditional and machine learning-based approaches in this article, we also compare different approaches in terms of spectral efficiency, computational complexity, and overhead.

show abstract

Q-Learning-Based Low Complexity Beam Tracking for mmWave Beamforming System

Cited by 6 publications

References 9 publications

Machine Learning for Millimeter Wave and Terahertz Beam Management: A Survey and Open Challenges

Machine Learning for Millimeter Wave and Terahertz Beam Management: A Survey and Open Challenges

Beam Training and Tracking in MmWave Communication: A Survey

Acquisition of channel state information for mmWave massive MIMO: traditional and machine learning-based approaches

Contact Info

Product

Resources

About