Upper Confidence Bound Algorithm for Opportunistic Spectrum Access with Sensing Errors

Abstract

In this paper we consider the problem of exploiting spectrum resources within the Opportunistic Spectrum Access context. We mainly focus on the case where one secondary user (SU) probes a pool of possibly available channels dedicated to a primary network. The SU is assumed to have imperfect sensing abilities. We, first, model the problem as a Multi-Armed Bandit problem with sensing errors. Then, we suggest to analyze the performances of the well known Upper Confidence Bound algorithm 1 within this framework, and show that we still can obtain an order optimal channel selection behavior. Finally we compare these results to those obtained in the case of perfect sensing. Simulation results are provided to support the suggested approach.

Ten years of research in spectrum sensing and sharing in cognitive radio

“…A minimum variancebased and a maximum capacity-based spectrum decision schemes have been developed for real-time applications and best-effort applications, respectively. Moreover, learning mechanisms can be equipped to assist the decision making procedure [110][111][112]. In [110], the use of reinforcement learning algorithm for spectrum assignment in WCDMA systems has been evaluated to show the satisfactory results under different load conditions.…”

Decision making for cognitive radio equipment: analysis of the first 10 years of exploration

“…With several network models based on game theory [13], Markov chains or multi-armed Bandit (MAB) (and machine learning in general) [44][45][46][47][48][49][50], to name a few, and relying on the concept of CR, the community tackled several challenges encountered when dealing with OSA such as (non exhaustive): dynamic power allocation, optimal band selection (with or without prior knowledge on the occupancy pattern of the spectrum bands by PUs), as well as cooperation among the different SUs [12] centralized or decentralized, with or without observation errors.…”

“…In Section 5.2 an OSA scenario based on a MAB model, described in article [48], is summarized and illustrates the impact of observation errors on decision making for CR. In the following section, however, we introduce prior knowledge as a classification criteria among the main learning and decision making tools suggested in CR articles.…”

“…We usually refer to this trade-off as the exploration-exploitation dilemma. If the problem is assumed modeled as a MAB framework an interesting way to tackle the problem is to use the class of so-called upper confidence bound algorithms n (UCB) [17,47,48,50,76]. The main advantage of UCB methods for CR is to offer a balance between exploration and exploitation phases without interrupting the communication process, i.e., while providing a certain service to the user [17].…”

“…A complete model is provided in [48]. Only one band can be sensed and tried at each iteration in order to keep the system's complexity reasonable.…”