Dynamic Task Offloading in Multi-Agent Mobile Edge Computing Networks

Heydari, Javad; Ganapathy, V.; Shah, Mohak

doi:10.1109/globecom38437.2019.9013115

Cited by 28 publications

(15 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, many researchers are currently engaged in applying artificial neural networks-based supervised learning and reinforcement learning theories to address some complex task offloading issues, such as the studies in [14], [28], [29]. For example, the authors in [14] present the compu-tation offloading problem in a space-air-ground integrated network as a Markov decision process and then propose a deep reinforcement learning approach for a UAV-user to learn an optimal policy for gaining a minimum long-term comprehensive cost (i.e., including delay, energy and server usage costs).…”

Section: Related Workmentioning

confidence: 99%

“…To minimize the long-term delay of a task, [29] designs two reinforcement learning methods, including a Qlearning method and a deep reinforcement learning method, to obtain the optimal policies for computation offloading and resource allocation in a vehicle edge-computing network. In [28], the authors also apply a deep reinforcement learning approach to make task offloading decisions with the goal of minimizing the task drop rate and the execution delay. From the methodological perspective, the reinforcement learningbased solutions can provide the advantages over traditional optimization techniques (e.g., convex optimization) in selforganization, evolvability and adaptability.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Task Offloading Optimization With Queueing Dynamics in Multiagent Mobile-Edge Computing Networks

Zhou

Sheng

Duan

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

Task offloading decision-making plays a key role in enabling mobile-edge computing (MEC) technologies in Internetof-Things (IoT). However, it meets the significant challenges arising from the stochastic dynamics of task queueing in the application layer and coupled wireless interference in the physical layer in a distributed multi-agent network without any centralized communication and computing coordination. In this paper, we investigate the distributed task offloading optimization problem with consideration of the upper-layer queueing dynamics and the lower-layer coupled wireless interference. We first propose a new optimization model that aims at maximizing the expected offloading rate of multiple agents by optimizing their offloading thresholds. Then, we transform the problem into a game-theoretic formulation, which further leads to the design of a distributed best-response (DBR) iterative optimization framework. The existence of Nash equilibrium strategies in the game-theoretic model has been analyzed. For the individual optimization of each agent's threshold policy, we further propose a programming scheme by transforming a constrained threshold optimization into an unconstrained Lagrangian optimization (ULO). The individual ULO is integrated into the DBR framework to enable agents to cooperate and converge to a global optimum in a distributed manner. Finally, simulation results are provided to validate the proposed method and demonstrate its significant advantage over other existing distributed methods. The numerical results also show that the proposed method can achieve comparable performance to a centralized optimization method.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Distributed Task Offloading Optimization With Queueing Dynamics in Multiagent Mobile-Edge Computing Networks

Zhou

Sheng

Duan

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…An interesting alternative is to use multi-agent-DRL (MA-DRL) [15] for supporting intelligent task offloading in MEC networks [16]. The work in [17] proposed a non-cooperative MA-DRL scheme where EDs could build their offloading policy independently. Another study in [18] also suggested an MA-DRL approach for joint data offloading and resource allocation in multiple independent edge clouds.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, this breaks the Markov properties required by the Q-learning algorithm and thus, DQN may not be capable of learning the cooperative offloading policies of EDs. Moreover, the non-cooperative multi-agent DRL solutions [17]- [19] may not be able to learn the cooperative policy; and thus resource usage over the edge network is not efficient which limits the overall offloading performance, e.g., offloading utility. • In addition, in most existing blockchain-based MEC schemes [22]- [26], the design and optimization of task offloading and blockchain mining are done separately, which would result in sub-optimal performance.…”

Section: Motivations and Our Key Contributionsmentioning

confidence: 99%

“…In the cooperative TOBM problem in our blockchain-based MEC system, conventional single-agent [13], [14], [32] or independent multi-agent [17]- [19] solutions are unable to obtain the cooperative policies of EDs due to the nonstationary and partially observable environment. Indeed, when policies of other agents change (i.e., due to computation mode preference), the ED (agent) n observation O n can be changed (nonstationary), which makes the obtained reward r n (t) different from the accumulated reward from its actual state-action pair.…”

Section: Proposed Ma-drl Algorithm For Cooperative Tobmmentioning

confidence: 99%

See 1 more Smart Citation

Cooperative Task Offloading and Block Mining in Blockchain-based Edge Computing with Multi-agent Deep Reinforcement Learning

Nguyen¹,

Ding²,

Pathirana³

et al. 2021

Preprint

View full text Add to dashboard Cite

The convergence of mobile edge computing (MEC) and blockchain is transforming the current computing services in mobile networks, by offering task offloading solutions with security enhancement empowered by blockchain mining. Nevertheless, these important enabling technologies have been studied separately in most existing works. This article proposes a novel cooperative task offloading and block mining (TOBM) scheme for a blockchain-based MEC system where each edge device not only handles data tasks but also deals with block mining for improving the system utility. To address the latency issues caused by the blockchain operation in MEC, we develop a new Proof-of-Reputation consensus mechanism based on a lightweight block verification strategy. A multi-objective function is then formulated to maximize the system utility of the blockchain-based MEC system, by jointly optimizing offloading decision, channel selection, transmit power allocation, and computational resource allocation. We propose a novel distributed deep reinforcement learning-based approach by using a multi-agent deep deterministic policy gradient algorithm. We then develop a game-theoretic solution to model the offloading and mining competition among edge devices as a potential game, and prove the existence of a pure Nash equilibrium. Simulation results demonstrate the significant system utility improvements of our proposed scheme over baseline approaches.

show abstract

Deep Reinforcement Learning Models and Techniques

2023

Deep Reinforcement Learning for Wireless Communications and Networking

View full text Add to dashboard Cite

Dynamic Task Offloading in Multi-Agent Mobile Edge Computing Networks

Cited by 28 publications

References 16 publications

Distributed Task Offloading Optimization With Queueing Dynamics in Multiagent Mobile-Edge Computing Networks

Distributed Task Offloading Optimization With Queueing Dynamics in Multiagent Mobile-Edge Computing Networks

Cooperative Task Offloading and Block Mining in Blockchain-based Edge Computing with Multi-agent Deep Reinforcement Learning

Deep Reinforcement Learning Models and Techniques

Contact Info

Product

Resources

About