Efficient Beam Alignment in Millimeter Wave Systems Using Contextual Bandits

Abstract

In this paper, we investigate the problem of beam alignment in millimeter wave (mmWave) systems, and design an optimal algorithm to reduce the overhead. Specifically, due to directional communications, the transmitter and receiver beams need to be aligned, which incurs high delay overhead since without a priori knowledge of the transmitter/receiver location, the search space spans the entire angular domain. This is further exacerbated under dynamic conditions (e.g., moving vehicles) where the access to the base station (access point) is highly dynamic with intermittent on-off periods, requiring more frequent beam alignment and signal training. To mitigate this issue, we consider an online stochastic optimization formulation where the goal is to maximize the directivity gain (i.e., received energy) of the beam alignment policy within a time period. We exploit the inherent correlation and unimodality properties of the model, and demonstrate that contextual information improves the performance. To this end, we propose an equivalent structured Multi-Armed Bandit model to optimally exploit the explorationexploitation tradeoff. In contrast to the classical MAB models, the contextual information makes the lower bound on regret (i.e., performance loss compared with an oracle policy) independent of the number of beams. This is a crucial property since the number of all combinations of beam patterns can be large in transceiver antenna arrays, especially in massive MIMO systems. We further provide an asymptotically optimal beam alignment algorithm, and investigate its performance via simulations.

Fast mmwave Beam Alignment via Correlated Bandit Learning

“…where b represents the optimal beam and N π b i (T ) denotes the number of times that b i has been selected up to time slot T . Hence, maximizing the cumulative reward is equivalent to minimizing the expected cumulative regret within T [6], which can be expressed as…”

Pilot contamination mitigation based on antenna subset transmission for mmWave massive MIMO

“…This assumption is related to beam alignment problem, the reader may see Maschietti et al 16 and Hashemi et al 17 for more details about this issue. This assumption is related to beam alignment problem, the reader may see Maschietti et al 16 and Hashemi et al 17 for more details about this issue.…”

Second-Best Beam-Alignment via Bayesian Multi-Armed Bandits

“…A coded beamalignment scheme is proposed in [11] to correct these errors, but with no consideration of feedback to improve beamselection. A multi-armed bandit (MAB) formulation based on upper confidence bound (UCB) is proposed in [9], by selecting the beam based on the empirical SNR distribution. A hierarchical beam-alignment scheme based on posterior matching is proposed in [10]: therein, a first-best policy is formulated, which selects the most likely beam pair based on the posterior distribution on the AoA-AoD pair.…”

“…We derive lower and upper bounds to the value function, based on which we propose a heuristic policy which selects the beam pair with the second-best preference. We show numerically that this policy strikes a favorable trade-off between exploration and exploitation: instead of greedily choosing the beam corresponding to the most likely AoA-AoD pair (first-best [10]), it chooses the second most likely one, leading to better exploration; at the same time, it avoids wasting precious resources to scan unlikely beam pairs, leading to better exploitation than other MAB techniques, such as linear Thompson sampling (LTS) [12] and UCB [9]. The proposed second-best scheme is shown to outperform first-best [10], LTS-based [12] and UCBbased [9] schemes by up to 7%, 10% and 30% in alignment probability, respectively.…”