Harvest-or-Transmit Policy for Cognitive Radio Networks: A Learning Theoretic Approach

Abstract

We consider an underlay cognitive radio network where the secondary user (SU) harvests energy from the environment. We consider a slotted-mode of operation where each slot of SU is used for either energy harvesting or data transmission. Considering block fading with memory, we model the energy arrival and fading processes as a stationary Markov process of first order. We propose a harvest-or-transmit policy for the SU along with optimal transmit powers that maximize its expected throughput under three different settings. First, we consider a learning-theoretic approach where we do not assume any apriori knowledge about the underlying Markov processes. In this case, we obtain an online policy using Q-learning. Then, we assume that the full statistical knowledge of the governing Markov process is known apriori. Under this assumption, we obtain an optimal online policy using infinite horizon stochastic dynamic programming. Finally, we obtain an optimal offline policy using the generalized Benders decomposition algorithm. The offline policy assumes that for a given time deadline, the energy arrivals and channel states are known in advance at all the transmitters. Finally, we compare all policies and study the effects of various system parameters on the system performance.

Optimal Sensing Policy for Energy Harvesting Cognitive Radio Systems

“…Primary problem in CR-EH is regarded to how Secondary Users (SUs) efficiently use harvested energy over time to maximize data throughput while keeping Primary Users (PUs) protected from interference. In this case, CR-EH primary problem is addressed through different solutions according to overlay [2], [7]- [13], underlay [14] or hybrid overlay-underlay dynamic spectrum access [15]. Through spectrum sensing (SS) operations, according to spectrum overlay model, SUs get opportunistic access to idle spectrum licensed by PUs to maximize data throughput.…”

Machine Learning-Based Energy-Spectrum Two-Dimensional Cognition in Energy Harvesting CRNs

“…Aiming to conquer the double challenges in improving spectrum efficiency and reducing energy consumption, the energy harvesting CRN (EH-CRN) driven by renewable energy has received increasing attention from both academia and industry [3]- [8]. EH-CRNs are usually defined as CRNs wherein SUs perform energy harvesting powered by electromagnetic radiation, light, thermal gradients, or fluid flow from environment.…”

“…Moreover, there are some emerging works focusing on temporal-spatial sensing in EH-CRNs. Specifically, some reinforcement learning (RL) based energy harvesting resource allocation schemes are proposed to enhance the capacity of the EH-CRNs [7], [8]. However, the works in [7] and [8] do not make use of the potential correlation between PUs and SUs in the energy dimension.…”

“…Specifically, some reinforcement learning (RL) based energy harvesting resource allocation schemes are proposed to enhance the capacity of the EH-CRNs [7], [8]. However, the works in [7] and [8] do not make use of the potential correlation between PUs and SUs in the energy dimension. Note that such correlation is an inherent but possible hidden behavior in the process of energy harvesting, which requires an in-depth mining to grasp the essential mechanism.…”

“…1) We design a new energy-spectrum 2D cognition scheme, which contains not only the spectrum sensing module but also the state prediction module and the transmission energy decision module. Different from the existing cognition schemes that only contain sensing module [7], [8], [10], [11], our proposed 2D cognition scheme is more comprehensive and practical. In particular, the prediction module and the decision module are complementary to spectrum sensing for predicting the licensed spectrum status by exploiting the observations of SUs and estimations of PUs, thus making a smart and proactive decision on the transmission energy of SUs for enhanced performance.…”