Two-Stage Multiarmed Bandit for Reconfigurable Intelligent Surface Aided Millimeter Wave Communications

Abstract

A reconfigurable intelligent surface (RIS) is a promising technology that can extend short-range millimeter wave (mmWave) communications coverage. However, phase shifts (PSs) of both mmWave transmitter (TX) and RIS antenna elements need to be optimally adjusted to effectively cover a mmWave user. This paper proposes codebook-based phase shifters for mmWave TX and RIS to overcome the difficulty of estimating their mmWave channel state information (CSI). Moreover, to adjust the PSs of both, an online learning approach in the form of a multiarmed bandit (MAB) game is suggested, where a nested two-stage stochastic MAB strategy is proposed. In the proposed strategy, the PS vector of the mmWave TX is adjusted in the first MAB stage. Based on it, the PS vector of the RIS is calibrated in the second stage and vice versa over the time horizon. Hence, we leverage and implement two standard MAB algorithms, namely Thompson sampling (TS) and upper confidence bound (UCB). Simulation results confirm the superior performance of the proposed nested two-stage MAB strategy; in particular, the nested two-stage TS nearly matches the optimal performance.

Cost-Effective MAB Approaches for Reconfigurable Intelligent Surface Aided Millimeter Wave Relaying

“…The incident signal from the antenna feeder can be modulated with low performance RF chains by controlling the RIS reflection coefficients. In RIS relay applications, the transferred power from the BS to the RIS surface is reflected to the mobile user through intelligent adjustment of the PSs [8] [20].…”

“…Few related research studies examine RIS deployment's impact on mmWave networks. A two-stage MAB-based approach for mmWave RIS systems was proposed in [8] to obtain the best connection that achieves maximum spectral efficiency using RIS. However, no relay selection was considered like in this paper.…”

“…To exchange the information between the mmWave BS and the RIS relay, a dedicated communication link between them is utilized. Following RIS-aided mmWave channel models given in [8] and [30], the received signal at the UE can be expressed as:…”

“…Also, 𝐚 𝑞 (𝜃 𝑙 (𝐴𝑜𝐷) , 𝜙 𝑙 (𝐴𝑜𝐷) ) is the response vector of the l-th path at the RIS relay 𝑞, where 𝜃 𝑙 (𝐴𝑜𝐷) and 𝜙 𝑙 (𝐴𝑜𝐷) are the azimuth and elevation AoD. Generally, 𝐚 𝑞 (𝜃, 𝜙 ) is expressed as [8]:…”

“…Moreover, RIS-aided mmWave relaying enormously reduces the high complexity of the mmWave conventional relaying due to eliminating the massive RF chains. Towards that, some searchers investigated the application of RIS to empower mmWave communication, especially in a high blockage environment [8][9][10][11]. Various challenges, including the joint estimation of the cascaded RISaided mmWave channel as well as the joint optimization of antenna PSs of both mmWave BS and RIS relay, were extensively investigated in literature [12][13][14].…”