Deep Contextual Bandits for Orchestrating Multi-User MISO Systems with Multiple RISs

Abstract

The emergent technology of Reconfigurable Intelligent Surfaces (RISs) has the potential to transform wireless environments into controllable systems, through programmable propagation of information-bearing signals. Techniques stemming from the field of Deep Reinforcement Learning (DRL) have recently gained popularity in maximizing the sum-rate performance in multi-user communication systems empowered by RISs. Such approaches are commonly based on Markov Decision Processes (MDPs). In this paper, we instead investigate the sum-rate design problem under the scope of the Multi-Armed Bandits (MAB) setting, which is a relaxation of the MDP framework. Nevertheless, in many cases, the MAB formulation is more appropriate to the channel and system models under the assumptions typically made in the RIS literature. To this end, we propose a simpler DRL approach for orchestrating multiple metasurfaces in RIS-empowered multi-user Multiple-Input Single-Output (MISO) systems, which we numerically show to perform equally well with a state-of-the-art MDP-based approach, while being less demanding computationally.

Deep Contextual Bandit and Reinforcement Learning for IRS-Assisted MU-MIMO Systems

“…In this subsection, we analyze two different strategies to solve similar optimization problems to understand the reasons that sustain our proposals. We focus on [36] and [37] which formulate the joint optimization of the IRS and precoder matrices as an RL and CB problem, respectively. Table I shows some differences and similarities between these two approaches.…”

“…On the other hand, [37] considers a CB-based approach to the joint optimization of the precoders and the IRS matrices in the downlink of a multi-IRS MU-MISO system. As in [36], the instantaneous sum-rate value is used as the reward but the state vectors are only composed of the channel coefficients for the current channel realization which is in accordance with the CB formulation.…”

“…The major limitation of the solution proposed in [37] is that they only consider discrete-valued actions, i.e., IRS scattering elements with only two possible phase values and a codebook set for the precoder selection. If a low-dimension set of feasible actions is considered, the system performance is very constrained and the benefits of the IRS-assisted and MIMO communication setups cannot be fully leveraged.…”

“…complex-valued circularly symmetric Gaussian random variables. On the other hand, considering that the IRS is typically installed to ensure line-of-sight (LoS) to the BS, a Rician fading channel model is adopted to describe H IB as in [37], [40], i.e.,…”

“…CB-based algorithms are mostly oriented to solve discrete action formulations such as those presented in [33]- [35], [37]. To account for continuous actions, we propose an innovative CB framework termed as DCB-DDPG since it is inspired on the DRL-based DDPG approach in [38].…”