Channel sensing order for cognitive radio networks using reinforcement learning

Abstract

Abstract-This work investigates the problem of channel sensing order used by a cognitive multichannel network, where each user is able to perform primary user detection on only one channel at a time. The sensing order indicates the sequence of channels sensed by the secondary users when searching for an available channel. When using an optimal sensing order, the secondary user can find faster a free channel with high quality. Brute-force algorithms may be used to find the optimal sensing order. However, this approach requires great computational effort. Even in scenarios where the secondary user knows the probability of each channel being available, the sensing order where the most available channels are sensed first is not ideal when using adaptive modulation. Therefore, we propose an approach using reinforcement learning to search dynamically for the optimal sensing order. Through simulations, we evaluated our proposal and compared its performance with other mechanisms, and the results obtained are close to the optimal value provided by the brute-force and superior to the other mechanisms in most of the scenarios.

Robust Multiuser Sequential Channel Sensing and Access in Dynamic Cognitive Radio Networks: Potential Games and Stochastic Learning

“…According to (12) and (25), it is noted that Φ2(O n , O −n ) is exactly the negative value of the expected aggregate interference level of all the potential users. Now, suppose that an arbitrary player n unilaterally changes its channel sensing order from O n to O * n while all other active players keep their sensing orders unchanged, then the change in player n's utility function is given by:…”

“…In particular, an adaptive persistent sensing order selection strategy was proposed in [10], a dynamic programming based order selection strategy in [11], and a reinforcement learning based order selection algorithm in [12]. The main difference in our work is that the active user set in each slot is randomly changing, and all the above mentioned algorithms do not converge in the presence of changing active user set.…”

Energy-Efficient, Learning-Inspired Channel Decision and Access Technique for Cognitive Radio-based Wireless Sensor Networks

“…Different authors have focused on different research areas other than energy efficiency. For instance, in [5], the authors used reinforcement learning (RL) to search dynamically the optimal sensing order. However, their method is not adaptable to CRWSN.…”

“…Computational complexities involved in this method make deployment in low power, battery-driven and less complex CRWSN impossible. The work in [6] proposed optimal sensing and access mechanisms, but not specifically adaptable to sensor networks for the same reason as observed with [5]. In [7], sensing time and PU activities were used to maximize the SU throughput and to keep the probability of collision below certain threshold.…”

Channel Sensing Order for Distributed Cognitive Radio Networks Using No-Regret Learning

“…OSA requires reconfigurable networks devices, called cognitive radios (CR), can adapt their behavior in response to environment stimuli [3]. For this, these cognitive devices or secondary users (SUs) need to determine by spectrum sensing when primary users are active in order to avoid causing them a harmful interference [4]. Therefore, OSA is important to utilize the licensed frequency spectrum more efficiently through opportunistic access to unused spectrum bands.…”