Notice of Violation of IEEE Publication Principles: QoE-Driven Resource Allocation for SUs with Heterogeneous Traffic using Deep Reinforcement Learning

AI Meets CRNs: A Prospective Review on the Application of Deep Architectures in Spectrum Management

“…As a result, the dynamic and uncertainty of the network status, the coexistence and coupling among different wireless devices with disparate requirements makes the manageable range of RL to suffer from the curse of dimensionality. Thus, deep architectures such as DL open a new era for the extension and development of RL into DRL in order to address the curse of dimensionality [94].…”

“…Therefore, the utility obtained after channel access is represented using the reward R = {r 1 , r 2 , • • • , r K,K }, which represent the available rates offered by the BS. At this point, the problem becomes one of finding a policy that will maximize the received discounted reward V i with a discount factor γ t , using the value function as follows [94]:…”

“…Thus, in order to solve (25) using this technique, assuming that a DNN is employed as an efficient nonlinear approximator that will be used to estimate the action-value function Q i (s, a; θ i ) ≈ Q * i (s, a), using the advantage of experience replay, the sampling procedure is illustrated in FIGURE 13 below [94]: FIGURE 13: An illustration of a memory object using a sumtree to represent the sampled data.…”

“…Here, the learning process was improved by incorporating transfer learning to the learning procedure and simulation results showed a substantial reduction of the iterations towards convergence compared to a DQN without transfer learning. A DQN framework that used a QoE-driven CR scheme where SUs offered heterogeneous traffic to the network was proposed in [94]. Here, docitive learning schemes were used to improve SU perception of the QoE through transfer learning.…”

“…The DQN framework demonstrated that it can combine deep exploration with DNNs for exponentially faster learning and convergence. The work in [94] was improved using an SU-experience-oriented resource allocation scheme for efficient real-time CR processes in [107]. The objective of the authors was to improve CR adaptation and intelligent RA using deep neuroevolution (DNE).…”