Upper Confidence Bound Based Decision Making Strategies and Dynamic Spectrum Access

Abstract

Abstract-In this paper, we consider the problem of exploiting spectrum resources for a secondary user (SU) of a wireless communication network. We suggest that Upper Confidence Bound (UCB) algorithms could be useful to design decision making strategies for SUs to exploit intelligently the spectrum resources based on their past observations. The algorithms use an index that provides an optimistic estimation of the availability of the resources to the SU. The suggestion is supported by some experimental results carried out on a specific dynamic spectrum access (DSA) framework.

Aggregation of multi-armed bandits learning algorithms for opportunistic spectrum access

“…We realized experiments on simple MAB problems already used in the community of bandit algorithms for OSA [6], and the simulations results showed that Aggregator works as expected, being able to identify on the fly the best algorithm to trust for a specific problem. Experiments on problems mixing different families of distributions were also presented, with similar conclusions in favor of Aggregator.…”

“…The SU has to select the best expected channel each time to maximize its throughput: if successful communications are seen as rewards, the SU has to maximize its cumulative rewards, as in the Multi-Armed Bandit (MAB) problem [3], [4]. MAB learning algorithms have been shown to be useful for the OSA setting [5], [6], and UCB algorithms and other variants (e.g., kl-UCBor Bayes-UCB, [7], [8], [9], [10]) have been successfully applied to both numerical and physically simulated CR problems [11]. The performance of such learning algorithm A can be measured by different criteria.…”

Improvement of the LPWAN AMI backhaul’s latency thanks to reinforcement learning algorithms

“…In this section, we will show that the channel selection can be viewed as a multi-armed bandit (MAB) problem [23], which can be solved thanks to simple reinforcement learning algorithms. This modelling has already been used in dynamic spectrum access (DSA) [26,27]. In such a scenario, spectrum sensing is used as a feedback for channel selection.…”

“…In this paper, with no loss of generality, we analyse the performance of two algorithms, the upper confidence bound (UCB) algorithm [26] which is frequentist and the Thompson sampling (TS) algorithms [25] which is Bayesian. The main advantages of these two algorithms are their low computational complexity and their low memory requirements, which allow them to be implemented in any end-device and in particular in aggregators.…”

“…These algorithms have already been proposed for dynamic spectrum access (DSA) [26,27] in a cognitive radio (CR) [28] context. In [29,30], the authors propose to use MAB learning algorithms in a time-slotted IoT network and in [31] these algorithms have been proposed for Wifi networks.…”