Distributed Learning Over Markovian Fading Channels for Stable Spectrum Access

Abstract

We consider the problem of multi-user spectrum access in wireless networks. The bandwidth is divided into K orthogonal channels, and M users aim to access the spectrum. Each user chooses a single channel for transmission at each time slot. The state of each channel is modeled by a restless unknown Markovian process. Previous studies have analyzed a special case of this setting, in which each channel yields the same expected rate for all users. By contrast, we consider a more general and practical model, where each channel yields a different expected rate for each user. This model adds a significant challenge of how to efficiently learn a channel allocation in a distributed manner to yield a global system-wide objective. We adopt the stable matching rate as the system objective, which is known to yield strong performance in multichannel wireless networks, and develop a novel Distributed Stable Strategy Learning (DSSL) algorithm to achieve the objective. We prove theoretically that DSSL converges to the stable matching allocation, and the regret, defined as the loss in total rate with respect to the stable matching solution, has a logarithmic order with time. Simulation results demonstrate the strong performance of the DSSL algorithm. We show that DSSL has significant advantages over the state-of-the-art methods in terms of implementation complexity, convergence speed, and network churn conditions.INDEX TERMS Wireless networks, spectrum access, online learning.

Markov chain modelling of ordered Rayleigh fading channels in non‐orthogonal multiple access wireless networks

“…One-dimensional Markov chain [3] � First-order FSMC [3,7] � Mathematical derivation [13][14][15] Single channel in OMA [3] � Higher-order FSMC [4] � Measurement [9,10] � Hidden Markov model [9,10] � Machine learning [11] � ⋯ Multidimensional Markov chain [3,24] Parallel Markov chains [8,25,26] � Measurement [6,27] Multiple channels in OMA [5] � Machine learning [12,28] Order statistics of multidimensional Markov chains [29] --Multiple channels in NOMA…”

“…Various parameter estimation schemes based on real-world measurement data are proposed to estimate the hidden Markov model parameters. In addition to the conventional mathematical methods [9,10], machine learning has recently been utilised in modelling hidden Markov channels [11,12].…”

“…In refs. [12,28], the hidden multidimensional Markov chain, modelled by machine learning, was used in the performance evaluation of a wireless network. However, the order statistics of multidimensional Markov chains differs from the multidimensional Markov chains itself.…”

Multi-Agent Deep Reinforcement Learning for Interference-Aware Channel Allocation in Non-Terrestrial Networks

“…DCA enhances the efficiency and flexibility of wireless networks, ensuring efficient channel utilization and mitigating the impact of interference, leading to improved overall performance and quality of service (QoS) . Recent studies on this topic, which involves learning the environment and channel allocation algorithms, have employed frameworks such as multi-armed bandits (MAB) [1]- [9], stable matching [10], [11], game theoretic optimization and congestion control [1], [8], [12]- [21], and, more recently, deep reinforcement learning (DRL) for multiuser scenarios [22]- [36]. The latter was initially explored in our earlier work (Naparstek and Cohen [22], [23]) within a multi-agent framework, following single-agent DRL research in [37], paving the way for a significant amount of subsequent research in the wireless communications community.…”

Efficient dynamic channel assignment through laser chaos: a multiuser parallel processing learning algorithm

“…Bistritz and Leshem 8 proposed The game of thrones (GoT) algorithm, which does not require user communication, and proved that it achieves an expected sum of regrets of near- . Gafni and Cohen 9 introduced a fully distributed algorithm—distributed stable strategy learning (DSSL)—to solve a multiuser channel assignment problem and achieved a stable state through a large number of exchanges between users with the upper boundary of the regret given in the theoretical proof.…”