Multi-Agent Reinforcement Learning-Based Buffer-Aided Relay Selection in IRS-Assisted Secure Cooperative Networks

Abstract

This paper proposes a multi-agent deep reinforcement learning-based buffer-aided relay selection scheme for an intelligent reflecting surface (IRS)-assisted secure cooperative network in the presence of an eavesdropper. We consider a practical phase model where both phase shift and reflection amplitude are discrete variables to vary the reflection coefficients of the IRS. Furthermore, we introduce the buffer-aided relay to enhance the secrecy performance, but the use of the buffer leads to the cost of delay. Thus, we aim to maximize either the average secrecy rate with a delay constraint or the throughput with both delay and secrecy constraints, by jointly optimizing the buffer-aided relay selection and the IRS reflection coefficients. To obtain the solution of these two optimization problems, we divide each of the problems into two sub-tasks and then develop a distributed multi-agent reinforcement learning scheme for the two cooperative sub-tasks, each relay node represents an agent in the distributed learning. We apply the distributed reinforcement learning scheme to optimize the IRS reflection coefficients, and then utilize an agent on the source to learn the optimal relay selection based on the optimal IRS reflection coefficients in each iteration. Simulation results show that the proposed learning-based scheme uses an iterative approach to learn from the environment for approximating an optimal solution via the exploration of multiple agents, which outperforms the benchmark schemes.

Reconfigurable Intelligent Surface Assisted OFDM Relaying: Subcarrier Matching with Balanced SNR

“…As shown in Fig. 1, Case I refers to the scenario where the RIS-destination link is ignored in the first time slot, which has been assumed in most existing works (see, e.g., [25]- [32]). The channel between source and relay at subcarrier p = 1, 2 .…”

“…To further improve the communication performance in relaying/RIS networks, the recent works in [25]- [32] have been investigating the intergation of RIS and relay in wireless networks, rather than unilaterally considering one of them. To be specific, a RIS-assisted relaying system was studied in [25], where tight upper bounds on the ergodic capacity were obtained under different channel environments.…”

“…In [31], a deep reinforcement learning algorithm was proposed to design the relay selection scheme in RIS-assisted cooperative networks, which was shown to achieve significant performance gains as compared to random relay selection and random passive beamforming designs. Furthermore, a multi-agent deep reinforcement learning-based algorithm was proposed for devising the buffer-aided relay selection scheme for integrated relay and RIS secure networks in the presence of an eavesdropper was introduced in [32].…”

Intelligent Reflective Surface Deployment in 6G: A Comprehensive Survey

“…Motivated by this, Huang et al, [71] proposed a multi-agent deep reinforcement learning-based buffer-aided relay selection scheme for an IRS-assisted secure cooperative network in the presence of an eavesdropper. They considered a practical scenario where both phase shift and reflection amplitude of the IRSs are optimized to improve the wireless network's performance.…”

“…Simulation results showed that the proposed learning-based scheme used an iterative approach to learn from the environment to approximate an optimal solution by exploring multiple agents, which outperforms benchmark schemes. However, this paper [71] only considered a single untrusted relay as an eavesdropper in the proposed network. This can be further extended to consider multiple eavesdropper scenarios.…”