An adaptive Q-learning approach to power control for D2D communications

Toumi, Salwa; Hamdi, Monia; Zaied, Mourad

doi:10.1109/aset.2018.8379860

Cited by 12 publications

(12 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where f i (e) is defined in ( 12) and e i represents the service order of user i. The problem (13) aims to find the optimal trajectory so that the UAV can complete most of the users' requests within their endurance time after receiving all the user requests. Since finding the optimal trajectory needs to evaluate all possible permutations of service order e, which takes up a substantial amount of service time, it's essential to introduce a learning algorithm to shorten the calculation time for the trajectory.…”

Section: Problem Formulationmentioning

confidence: 99%

“…To solve the maximization problem in (13), we introduce a reinforcement learning framework based on double Q-learning. Compared to the existing reinforcement learning algorithms [12]- [14] such as Q-learning that may result in sub-optimal trajectory and leads to the number of satisfied users not maximized, the proposed double Q-learning algorithm enables the UAV to find the optimal flying trajectory to serve the users so as to maximize the number of satisfied users.…”

Section: Double Q-learning Framework For Maximizing the Number Of Sat...mentioning

confidence: 99%

“…The work in [12] applied deep Q-network in a mobile communication system to reduce the exploration time for achieving an optimal communication policy. The authors in [13] proposed a Qlearning based algorithm to coordinate power allocation and control interference levels, so as to maximize the sum data rate of device-to-device (D2D) users while guaranteeing QoS for cellular users. In [14], the authors proposed an expected Q-learning algorithm to solve the spectrum allocation problem in LTE networks that operate in unlicensed spectrum (LTE-U) with downlink-uplink decoupling and improve the total rate.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimized Trajectory Design in UAV Based Cellular Networks: A Double Q-Learning Approach

Liu

Chen

Yin

2018

2018 IEEE International Conference on Communication Systems (ICCS)

View full text Add to dashboard Cite

In this paper, the problem of trajectory design of unmanned aerial vehicles (UAVs) for maximizing the number of satisfied users is studied in a UAV based cellular network where the UAV works as a flying base station that serves users, and the user indicates its satisfaction in terms of completion of its data request within an allowable maximum waiting time. The trajectory design is formulated as an optimization problem whose goal is to maximize the number of satisfied users. To solve this problem, a machine learning framework based on double Qlearning algorithm is proposed. The algorithm enables the UAV to find the optimal trajectory that maximizes the number of satisfied users. Compared to the traditional learning algorithms, such as Q-learning that selects and evaluates the action using the same Q-table, the proposed algorithm can decouple the selection from the evaluation, therefore avoid overestimation which leads to sub-optimal policies. Simulation results show that the proposed algorithm can achieve up to 19.4% and 14.1% gains in terms of the number of satisfied users compared to random algorithm and Q-learning algorithm.

show abstract

Section: Problem Formulationmentioning

confidence: 99%

Section: Double Q-learning Framework For Maximizing the Number Of Sat...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimized Trajectory Design in UAV Based Cellular Networks: A Double Q-Learning Approach

Liu

Chen

Yin

2018

2018 IEEE International Conference on Communication Systems (ICCS)

View full text Add to dashboard Cite

show abstract

“…The reinforcement learning (RL) based resource allocation schemes have been applied to device-to-device (D2D) communication widely. In [26], a Q-learning based power control algorithm was proposed which decorrelated the actions selected by users and expand the solution space, and it had higher quality of service (QoS) than the schemes based on correlated Q-learning. In [27], two RL based power control methods were proposed, i.e., centralized method and distributed method.…”

Section: Introductionmentioning

confidence: 99%

Intelligent Resource Allocation for Train-to-Train Communication: A Multi-Agent Deep Reinforcement Learning Approach

2020

View full text Add to dashboard Cite

The application of train-to-train (T2T) communication in urban rail transit is expected to simplify system structure, reduce maintenance costs, and improve operational efficiency. In particular, trainto-wayside (T2W) communication coexist with T2T communication in the train control system based on T2T communication. To make full use of limited spectrum resources, frequency reuse is adopted as an efficient technique, but it brings the co-channel interference unfortunately, which affects the quality of service (QoS) for T2T and T2W users. In this paper, we propose a multi-agent deep reinforcement learning (MADRL) based autonomous channel selection and transmission power selection algorithm for T2T communication to reduce the co-channel interference. Specifically, each agent interacts with the environment and selects actions to implement a distributed resource allocation mechanism independently, adopting asynchronous updates to avoid different agents choosing the same sub-band. Simulation results show the superiority of our proposed algorithm: compared with the existing resource allocation schemes for T2T communication, the system throughput and the successful transmission probability of T2T links are greatly improved.INDEX TERMS Train-to-train (T2T) communication, resource allocation, multi-agent deep reinforcement learning (MADRL), urban rail transit.

show abstract

“…Solutions are exploring on low-complexity methods to small-cell base-station design appropriate for future 5G indoor deployments. 21 Machine learning is one of the tools that could provide best set of solutions to learn the influential scenarios and certain parameters of the communication networks. The reinforcement leaning 11 in machine learning holds much power over the D2D communication network due to its self-healing nature using many corrective actions.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning algorithm for 5G indoor device‐to‐device communications

Sreedevi

Rao

2019

Trans Emerging Tel Tech

View full text Add to dashboard Cite

Fifth generation (5G), the next generation telecommunications will be striking the markets in near future. Device‐to‐device (D2D) communication would be a part of 5G to serve communication needs for billions of connected devices to support high data rate ultrareliable low latency communications. Indoor 5G will be relying on distributed small cell solutions and D2D along with machine‐to‐machine connections. Machine learning is one of the most promising tools for providing the best set of solutions to learn the influential scenarios and certain parameters of the communication networks. This research proposes reinforcement‐learning‐based latency controlled D2D connectivity (RL‐LCDC) algorithm and its Q‐learning approach in an indoor D2D communication network for strong 5G connectivity with minimum latency. The proposed approach, RL‐LCDC efficiently discovers the neighbors, decides the D2D link, and adaptively controls the communication range for maximum network connectivity. The results show that RL‐LCDC optimizes the connectivity with lower end‐to‐end delay and better energy efficiency with efficient convergence time when compared with other conventional schemes.

show abstract

An adaptive Q-learning approach to power control for D2D communications

Cited by 12 publications

References 6 publications

Optimized Trajectory Design in UAV Based Cellular Networks: A Double Q-Learning Approach

Optimized Trajectory Design in UAV Based Cellular Networks: A Double Q-Learning Approach

Intelligent Resource Allocation for Train-to-Train Communication: A Multi-Agent Deep Reinforcement Learning Approach

Reinforcement learning algorithm for 5G indoor device‐to‐device communications

Contact Info

Product

Resources

About