Learning Frameworks for Dynamic Joint RF Energy Harvesting and Channel Access

Abstract

The fifth generation mobile networks (5G) envision to interconnect the massive number of devices with a wide range of characteristics and demands for 2020 and beyond. With more radio-frequency (RF) bands to support multiple frequency transmission, separate antennas, and RF chips may be required for widely separated frequency bands. The development of cognitive radio systems, in which a wireless transceiver automatically adapts its communication parameters to network and user demands, is therefore crucial to the successful roll-out of the 5G. This paper introduces an innovative CR system that performs the spectrum sensing for the joint energy harvesting and channel access. A prototype test bed is employed to detect and convert RF signals from different bands to a DC voltage for powering a sensor board that communicates over another channel to an access point. The system's objective is to use the harvested energy for concurrent data transmission. We model the selection of channels to maximize this objective as a multi-armed bandit (MAB) problem. Depending on the type of the MAB problem, three spectrum sensing strategies are developed for joint energy harvesting and channel access. The first one, applicable for stochastic MAB, conceptualizes some formulations of the exploration/exploitation balancing technique of the available frequency bands. The second strategy is an opportunistic sensing framework for the Markovian MAB when the state of the underlying Markov process is partially observed. The third strategy is based on the Gittins index allocation framework for the fully observed Markovian MAB. The simulation results show that lower regrets can be obtained with more information on the underlying Markov process of the MAB. INDEX TERMS Markov decision process, multi-armed bandit, RF energy harvesting, Gittins index, POMDP.

Reinforcement Learning based Per-antenna Discrete Power Control for Massive MIMO Systems

“…The CSI in a cellular network varies if the user is walking, running or in a vehicle. In literature [19], [20], the time varying channel is modeled using a finite state Markov chain, where the ergodic channel in each time slot takes value in one of the Markov states. Let H = H (1) , .…”