Distributed Task Migration Optimization in MEC by Extending Multi-Agent Deep Reinforcement Learning Approach

Liu, Chubo; Tang, Fan; Hu, Yikun; Li, Kenli; Tang, Zhuo

doi:10.1109/tpds.2020.3046737

Cited by 88 publications

(32 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Offloading can be used to balance the load among cores or processors in a multiprocessor system. This is often regarded as task migration that aims at moving the task execution from one core/processor using a given performance metric: power consumption, thermal energy, and dark silicon [25]. Communication-driven task migration attempts to migrate tasks to adjacent cores.…”

Section: Related Workmentioning

confidence: 99%

Requirements for Energy-Harvesting-Driven Edge Devices Using Task-Offloading Approaches

et al. 2022

View full text Add to dashboard Cite

Energy limitations remain a key concern in the development of Internet of Medical Things (IoMT) devices since most of them have limited energy sources, mainly from batteries. Therefore, providing a sustainable and autonomous power supply is essential as it allows continuous energy sensing, flexible positioning, less human intervention, and easy maintenance. In the last few years, extensive investigations have been conducted to develop energy-autonomous systems for the IoMT by implementing energy-harvesting (EH) technologies as a feasible and economically practical alternative to batteries. To this end, various EH-solutions have been developed for wearables to enhance power extraction efficiency, such as integrating resonant energy extraction circuits such as SSHI, S-SSHI, and P-SSHI connected to common energy-storage units to maintain a stable output for charge loads. These circuits enable an increase in the harvested power by 174% compared to the SEH circuit. Although IoMT devices are becoming increasingly powerful and more affordable, some tasks, such as machine-learning algorithms, still require intensive computational resources, leading to higher energy consumption. Offloading computing-intensive tasks from resource-limited user devices to resource-rich fog or cloud layers can effectively address these issues and manage energy consumption. Reinforcement learning, in particular, employs the Q-algorithm, which is an efficient technique for hardware implementation, as well as offloading tasks from wearables to edge devices. For example, the lowest reported power consumption using FPGA technology is 37 mW. Furthermore, the communication cost from wearables to fog devices should not offset the energy savings gained from task migration. This paper provides a comprehensive review of joint energy-harvesting technologies and computation-offloading strategies for the IoMT. Moreover, power supply strategies for wearables, energy-storage techniques, and hardware implementation of the task migration were provided.

show abstract

Section: Related Workmentioning

confidence: 99%

Requirements for Energy-Harvesting-Driven Edge Devices Using Task-Offloading Approaches

et al. 2022

View full text Add to dashboard Cite

show abstract

“…More details are given as follows: S1. After mapping the focused objective function and constraints in the MEC-based networks into the states, actions, and reward in the MDP, the RL algorithms are directly employed to the original optimization problem, such as in [69]- [71]. Among them, many advanced RL algorithms are applied to balance the convergence speed and learning accuracy, such as DDPG and A3C.…”

Section: B Reinforcement Learning-empowered Mecmentioning

confidence: 99%

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Wei,

Guo,

et al. 2022

Preprint

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond. However, its uncertainty, referred to as dynamic and randomness, from the mobile device, wireless channel, and edge network sides, results in high-dimensional, nonconvex, nonlinear, and NP-hard optimization problems. Thanks to the evolved reinforcement learning (RL), upon iteratively interacting with the dynamic and random environment, its trained agent can intelligently obtain the optimal policy in MEC. Furthermore, its evolved versions, such as deep RL (DRL), can achieve higher convergence speed efficiency and learning accuracy based on the parametric approximation for the large-scale state-action space. This paper provides a comprehensive research review on RL-enabled MEC and offers insight for development in this area. More importantly, associated with free mobility, dynamic channels, and distributed services, the MEC challenges that can be solved by different kinds of RL algorithms are identified, followed by how they can be solved by RL solutions in diverse mobile applications. Finally, the open challenges are discussed to provide helpful guidance for future research in RL training and learning MEC. Index Terms-Mobile edge computing (MEC), network uncertainty, reinforcement learning (RL). emote Areas irplane taverse vers Diverse Scenarios MEC Deployment Future Applications Smart City Smart Home Hazard/Remote Areas Smart Agriculture Digital Twin Holographic Communication AR/VR/XR Metaverse Metaverse Metavers Autonomous Driving Small Data Center Switch BS Traffic Light Satellite Ship Airplane S ll S it h BS T ffi Li ht S t llit Shi Ai l 5G Networks and Beyond 5G Networks and Beyond Smart Factory performance. Its typical algorithms are K-means clustering, principal component analysis, and independent component analysis, which can be used in small cell clustering, heterogeneous network clustering, smart grid user classification, etc. However, in unreliable radio network environments, the classification accuracy decreases, which readily causes slow and inaccurate actions in MEC systems. • Different from the static solutions provided by supervised learning and unsupervised learning, RL gives a constantly evolving intelligent framework [21]-[27]. In RL [29], an agent is enabled to make proper decisions based on frequent interactions with the stochastic and dynamic environment. Based on the Markov models, the RL operates in a feedback mechanism (a closed loop) without the knowledge of input data, where based on the previous and current states, the agent can execute its actions to maximize the reward function. Additionally, the above behaviors, including the states, actions, and rewards, accumulate to generate experiences. The classic RL algorithm is Q-learning, which suffers from the curse of dimensionality caused by the large dimensions of the state-action spaces. Accordingly, upon leveraging the low-dimensional representation for the high-dimensional state-ac...

show abstract

“…In order to verify the average completion time, we also implemented Extensive Service Migration(ESM) [19], Always Migration(AM) [20], Counterfactual Multi-Agent (COMA) [21], Never Migration (NM) as a baseline. AWDDPG adopts the experience replay mechanism to reduce the correlation between samples, and designs an adaptive weighted sampling method to increase the sampling efficiency, which greatly increases the convergence speed and stability.…”

Section: Performance Evaluationmentioning

confidence: 99%

Distributed Task Migration Optimization in MEC by Deep Reinforcement Learning Strategy

Cui

Zhang

et al. 2021

2021 IEEE 46th Conference on Local Computer Networks (LCN)

View full text Add to dashboard Cite

─Mobile management is a challenging technology in Mobile Edge Computing (MEC). When the device is moving, computation tasks need to be dynamically migrated between multiple edge servers to maintain service continuity. This paper proposes a migration optimization of distributed task in MEC by deep reinforcement learning solution to optimize delay. In Multi-agent Deep Reinforcement Learning (MADRL), we construct an adaptive weight deep deterministic policy gradient (AWDDPG) algorithm to optimize the migration cost and service delay, and adopt centralized training and distributed execution to solve the high-dimensional problem. Experiments show that our algorithm greatly reduces the service delay compared with the related algorithms.

show abstract

Distributed Task Migration Optimization in MEC by Extending Multi-Agent Deep Reinforcement Learning Approach

Cited by 88 publications

References 28 publications

Requirements for Energy-Harvesting-Driven Edge Devices Using Task-Offloading Approaches

Requirements for Energy-Harvesting-Driven Edge Devices Using Task-Offloading Approaches

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Distributed Task Migration Optimization in MEC by Deep Reinforcement Learning Strategy

Contact Info

Product

Resources

About