Distributed Deep Reinforcement Learning using TensorFlow

Rao, Prakash; Kumar, Bhumesh; Cadabam, Siddharth; Praveena, T. Lakshmi

doi:10.1109/ctceec.2017.8455196

Cited by 9 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A3C, short for Asynchronous Advanced Actor-Critic, is an improvement on the traditional Actor-Critic algorithm. A3C algorithm converges faster, contains more stable policies [40,41]. As its name shows, the A3C algorithm consists of two parts, the Actor-Critic algorithm and asynchronous algorithm.…”

Section: Tcn Enabled A3c Modelmentioning

confidence: 99%

Resource Allocation on Blockchain Enabled Mobile Edge Computing System

et al. 2022

View full text Add to dashboard Cite

Currently, the concept of Mobile Edge Computing (MEC) has been applied as a solution against the plethora of demands for high-quality computing services. It comprises several essential processes, such as resource allocation, data transmission, and task processing. Furthermore, researchers applied blockchain technology, aiming to enhance the robustness of the MEC system. At present, resource allocation in the MEC system is a very hot field, but there are still some problems in the resource allocation process under the traditional MEC architecture, such as privacy disclosure and so on. Moreover, the resource allocation problem in a blockchain-enabled MEC system will be more complicated, while the mining process may have an impact on resource allocation policy. To address this issue, this paper investigates the resource allocation problem with blockchain-based MEC system architecture. A brand new consensus mechanism: proof of learning (PoL), is applied to the system, which does not waste the computing resources of edge computing servers. Based on this, we modeled the system mathematically, focusing on server processing latency, mining latency, rewards under the new consensus, and total cost. The asynchronous advantage Actor-Critic (A3C) algorithm is used to optimize resource allocation policy. To better capture the long-time trend of the system, the temporal convolutional network (TCN) is implemented to represent the policy function and state-value function in the reinforcement learning model. The results show that the A3C algorithm based on TCN not only converges faster but also is more stable.

show abstract

Section: Tcn Enabled A3c Modelmentioning

confidence: 99%

Resource Allocation on Blockchain Enabled Mobile Edge Computing System

et al. 2022

View full text Add to dashboard Cite

show abstract

“…DQN algorithm is also a typical model-free DQL [35], so the computational resource allocation problem can be formulated as MDP as well, the Agent, State, Action, and Reward are described as follows.…”

Section: Dqn Algorithm-based Computation Resource Allocation Schemementioning

confidence: 99%

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Jiang

Gao

et al. 2021

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

This paper investigates a computing offloading policy and the allocation of computational resource for multiple user equipments (UEs) in device-to-device (D2D)-aided fog radio access networks (F-RANs). Concerning the dynamically changing wireless environment where the channel state information (CSI) is difficult to predict and know exactly, we formulate the problem of task offloading and resource optimization as a mixed-integer nonlinear programming problem to maximize the total utility of all UEs. Concerning the non-convex property of the formulated problem, we decouple the original problem into two phases to solve. Firstly, a centralized deep reinforcement learning (DRL) algorithm called dueling deep Q-network (DDQN) is utilized to obtain the most suitable offloading mode for each UE. Particularly, to reduce the complexity of the proposed offloading scheme-based DDQN algorithm, a pre-processing procedure is adopted. Then, a distributed deep Q-network (DQN) algorithm based on the training result of the DDQN algorithm is further proposed to allocate the appropriate computational resource for each UE. Combining these two phases, the optimal offloading policy and resource allocation for each UE are finally achieved. Simulation results demonstrate the performance gains of the proposed scheme compared with other existing baseline schemes.

show abstract

“…DQN algorithm is also a typical model-free DQL [31], so the computational resource allocation problem can be formulated as MDP as well, the Agent, State, Action, and Reward are described as follows.…”

Section: ) Ddqn Algorithmmentioning

confidence: 99%

A Reinforcement Learning based Computing Offloading and Resource Allocation Scheme in F-RAN

Jiang

Gao

2021

Preprint

View full text Add to dashboard Cite

This paper investigates a computing offloading policy and the allocation of computational resource for multiple user equipments (UEs) in Device-to-Device (D2D) aided fog radio access networks (F-RANs). Concerning the dynamically changing wireless environment where the channel state information (CSI) is difficult to predict and know exactly, we formulate the problem of task offloading and resource optimization as a mixed-integer nonlinear programming problem to maximize the total utility of all UEs. Concerning the non-convex property of the formulated problem, we decouple the original problem into two phases to solve. Firstly, a centralized deep reinforcement learning (DRL) algorithm called Dueling Deep Q-Network (DDQN) is utilized to obtain the most suitable offloading mode for each UE. Particularly, to reduce the complexity of the proposed offloading scheme based DDQN algorithm, a pre-processing procedure is adopted. Then a distributed Deep Q-Network (DQN)algorithm based on the training result of the DDQN algorithm is further proposed to allocate the appropriate computational resource for each UE. Combining these two phases, the optimal offloading policy and resource allocation for each UE are finally achieved. Simulation results demonstrate the performance gains of the proposed scheme compared with other existing baseline schemes.

show abstract

Distributed Deep Reinforcement Learning using TensorFlow

Cited by 9 publications

References 5 publications

Resource Allocation on Blockchain Enabled Mobile Edge Computing System

Resource Allocation on Blockchain Enabled Mobile Edge Computing System

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

A Reinforcement Learning based Computing Offloading and Resource Allocation Scheme in F-RAN

Contact Info

Product

Resources

About