Dynamic resource allocations based on Q-learning for D2D communication in cellular networks

Luo, Yong; Shi, Zhiping; Zhou, Xin; Liu, Qiaoyan; Yi, Qicong

doi:10.1109/iccwamtip.2014.7073432

Cited by 54 publications

(38 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Resource allocation in D2D communication is such an application. Here, we describe at first some classical approaches [7][8][9][10][11][12][13][14][15][16] followed by existing RL-based resource allocation algorithms [17,18].…”

Section: Related Workmentioning

confidence: 99%

“…With regards to machine learning for resource allocation in D2D communication, there are only few works, e.g., [17,18]. Luo et al [17] and Nie et al [18] exploit machine learning algorithms for D2D resource allocation.…”

Section: Related Workmentioning

confidence: 99%

“…Luo et al [17] and Nie et al [18] exploit machine learning algorithms for D2D resource allocation. Luo et al [17] propose Distributed reinforcement learning (DIRL), Q-learning algorithm for resource allocation which improves the overall system performance in comparison to the random allocator. However, the model of Reinforcement learning (RL) is not well structured.…”

Section: Related Workmentioning

confidence: 99%

“…We set ρ = 1 for learning rate calculation in Equation (14). The discount factor, γ 1 = 0.9 is considered based on the work [17] for fair comparison with our work.…”

Section: Parameter Valuementioning

confidence: 99%

“…We compare our method with the distributed reinforcement learning proposed in [17] and a base-line random allocation of resources. Figure 3a shows that the proposed cooperative reinforcement learning outperforms both the random allocation and the distributed reinforcement learning regarding average system throughput calculated by Equation (6) considering 12 D2D user pairs and other parameter values as in Table 2.…”

Section: Parameter Valuementioning

confidence: 99%

See 4 more Smart Citations

Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication

et al. 2017

View full text Add to dashboard Cite

Device-to-device (D2D) communication is an essential feature for the future cellular networks as it increases spectrum efficiency by reusing resources between cellular and D2D users. However, the performance of the overall system can degrade if there is no proper control over interferences produced by the D2D users. Efficient resource allocation among D2D User equipments (UE) in a cellular network is desirable since it helps to provide a suitable interference management system. In this paper, we propose a cooperative reinforcement learning algorithm for adaptive resource allocation, which contributes to improving system throughput. In order to avoid selfish devices, which try to increase the throughput independently, we consider cooperation between devices as promising approach to significantly improve the overall system throughput. We impose cooperation by sharing the value function/learned policies between devices and incorporating a neighboring factor. We incorporate the set of states with the appropriate number of system-defined variables, which increases the observation space and consequently improves the accuracy of the learning algorithm. Finally, we compare our work with existing distributed reinforcement learning and random allocation of resources. Simulation results show that the proposed resource allocation algorithm outperforms both existing methods while varying the number of D2D users and transmission power in terms of overall system throughput, as well as D2D throughput by proper Resource block (RB)-power level combination with fairness measure and improving the Quality of service (QoS) by efficient controlling of the interference level.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…We set ρ = 1 for learning rate calculation in Equation (14). The discount factor, γ 1 = 0.9 is considered based on the work [17] for fair comparison with our work.…”

Section: Parameter Valuementioning

confidence: 99%

Section: Parameter Valuementioning

confidence: 99%

See 3 more Smart Citations

Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication

et al. 2017

View full text Add to dashboard Cite

show abstract

Learning‐based resource allocation in D2D communications with QoS and fairness considerations

Rashed

Shahbazian

Ghorashi

2017

Trans Emerging Tel Tech

View full text Add to dashboard Cite

In device-to-device (D2D) communications, D2D users establish a direct link by utilizing the cellular users' spectrum to increase the network spectral efficiency. However, due to the higher priority of cellular users, interference imposed by D2D users to cellular ones should be controlled by channel and power allocation algorithms. Due to the unknown distribution of dynamic channel parameters, learning-based resource allocation algorithms work more efficient than classic optimization methods. In this paper, the problem of the joint channel and power allocation for D2D users in realistic scenarios is formulated as an interactive learning problem, where the channel state information of selected channels is unknown to the decision center and learned during the allocation process. In order to achieve the maximum reward function by choosing an action (channel and power level) for each D2D pair, a recency-based Q-learning method is introduced to find the best channel-power for each D2D pair. The proposed method is shown to achieve logarithmic regret function asymptotically, which makes it an order optimal policy, and it converges to the stable equilibrium solution. The simulation results confirm that the proposed method achieves better responses in terms of network sum rate and fairness criterion in comparison with conventional learning methods and random allocation.

show abstract

A survey of applied machine learning techniques for optical orthogonal frequency division multiplexing based networks

Mrabet

Dayoub

Belghith

2021

Trans Emerging Tel Tech

View full text Add to dashboard Cite

In this survey, we analyze the newest machine learning (ML) techniques for optical orthogonal frequency division multiplexing (O-OFDM)-based optical communications. ML has been proposed to mitigate channel and transceiver imperfections. For instance, ML can improve the signal quality under low modulation extinction ratio or can tackle both determinist and stochastic-induced nonlinearities such as parametric noise amplification in long-haul transmission. The proposed ML algorithms for O-OFDM can in particularly tackle inter-subcarrier nonlinear effects such as four-wave mixing and cross-phase modulation. In essence, these ML techniques could be beneficial for any multi-carrier approach (e.g. filter bank modulation). Supervised and unsupervised ML techniques are analyzed in terms of both O-OFDM transmission performance and computational complexity for potential real-time implementation. We indicate the strict conditions under which a ML algorithm should perform classification, regression or clustering. The survey also discusses open research issues and future directions towards the ML implementation.

show abstract

Dynamic resource allocations based on Q-learning for D2D communication in cellular networks

Cited by 54 publications

References 7 publications

Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication

Throughput-Aware Cooperative Reinforcement Learning for Adaptive Resource Allocation in Device-to-Device Communication

Learning‐based resource allocation in D2D communications with QoS and fairness considerations

A survey of applied machine learning techniques for optical orthogonal frequency division multiplexing based networks

Contact Info

Product

Resources

About