Reinforcement Learning-Based Routing Protocol to Minimize Channel Switching and Interference for Cognitive Radio Networks

Abstract

In the existing network-layered architectural stack of Cognitive Radio Ad Hoc Network (CRAHN), channel selection is performed at the Medium Access Control (MAC) layer. However, routing is done on the network layer. Due to this limitation, the Secondary/Unlicensed Users (SUs) need to access the channel information from the MAC layer whenever the channel switching event occurred during the data transmission. This issue delayed the channel selection process during the immediate routing decision for the channel switching event to continue the transmission. In this paper, a protocol is proposed to implement the channel selection decisions at the network layer during the routing process. The decision is based on past and expected future routing decisions of Primary Users (PUs). A learning agent operating in the cross-layer mode of the network-layered architectural stack is implemented in the spectrum mobility manager to pass the channel information to the network layer. This information is originated at the MAC layer. The channel selection is performed on the basis of reinforcement learning algorithms such as No-External Regret Learning, Q-Learning, and Learning Automata. This leads to minimizing the channel switching events and user interferences in the Reinforcement Learning- (RL-) based routing protocol. Simulations are conducted using Cognitive Radio Cognitive Network simulator based on Network Simulator (NS-2). The simulation results showed that the proposed routing protocol performed better than all the other comparative routing protocols in terms of number of channel switching events, average data rate, packet collision, packet loss, and end-to-end delay. The proposed routing protocol implies the improved Quality of Service (QoS) of the delay sensitive and real-time networks such as Cellular and Tele Vision (TV) networks.

Untitled

“…In [18], impact of variable packet size on routing performance in cognitive radio networks is evaluated. MCSUI [19] performs joint channel selection and routing which minimizes channel switching and user interferences in cognitive radio networks. In [20], particle swarm optimization is used to optimize the channel allocation when multiple secondary users listen to the channel at the same time, and the best cooperative channel selection is performed based on Q-reinforcement learning.…”

A policy configured resource management scheme for AHNS using link reliability K‐means clustering algorithm and Weibull distribution‐based blue monkey optimization

“…8 In CR networks, for channel allocation schemes, various graph theoretic-based optimization techniques are deployed. 9 The goal of all those methods is to provide spectrum access to SUs without interfering with Pus's regular operations. However, neither the mobility of SU devices nor the issue of network topology changes due to neighboring CR network operations were addressed.…”

“…For designing every sort of mathematical model of SA, enormous techniques like estimation technique, pricing theory, evolutionary computation, game theory, fuzzy logic, and theory of social science are implemented 8 . In CR networks, for channel allocation schemes, various graph theoretic‐based optimization techniques are deployed 9 . The goal of all those methods is to provide spectrum access to SUs without interfering with Pus's regular operations.…”