Distributed and Collective Deep Reinforcement Learning for Computation Offloading: A Practical Perspective

Qiu, Xiaoyu; Zhang, Weikun; Chen, Wuhui; Zheng, Zibin

doi:10.1109/tpds.2020.3042599

Cited by 87 publications

(39 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve the convergence of the DQN algorithm in an edge computing environment, Xiong et al [30] proposed a DQN-based algorithm combined with multiple replay memories to minimize the execution time of one IoT application. Qiu et al [31] studied the distributed DRL in an edge computing environment with a single edge server to minimize the energy cost of running IoT applications, consisted of independent tasks. To obtain this goal, they combined deep neuro-evolution and policy gradient to improve the convergence results.…”

Section: Edge Computingmentioning

confidence: 99%

“…Moreover, the weighted cost model can be changed to execution time or energy consumption model by assigning w 1 = 1, w 2 = 0 or w 1 = 0, w 2 = 1, respectively. Since the application placement problem in heterogeneous environments is an NP-hard problem [31], the problem's complexity grows exponentially as the number of heterogeneous servers and/or the number of tasks within an IoT application increases. Thus, the optimal policy of the application placement problem cannot be obtained in polynomial time by iterative approaches.…”

Section: Weighted Cost Modelmentioning

confidence: 99%

“…11 if the task can be executed (done = 1). Moreover, we define a constant penalty value, which is usually a very large negative number [31]. Furthermore, the penalty value can be dynamically set based on the goal of the optimization problem and environmental variables.…”

Section: Deep Reinforcement Learning Modelmentioning

confidence: 99%

“…To improve the selection probability of better actions by the actor, the parameters of the actor network are updated using the feedback of the TD-error, while the network parameters of the critic network are updated to achieve better value estimation. While the actor-critic frameworks work very well in long-term performance optimizations, their learning speeds are slow and they incur huge exploration costs, especially in problems with high dimensional-state space [31]. The distributed learning techniques in which diverse trajectories are generated in parallel can greatly improve the exploration costs and learning speed of actor-critic frameworks.…”

Section: Distributed Drl-based Frameworkmentioning

confidence: 99%

“…Hence, we vectorized the fog computing environment, generated using OpenAI Gym, so that distributed brokers can interact with their fog computing environments and make application placement decisions in parallel. Unlike prior work [16], [31], [32], [33], we consider a heterogeneous fog computing environment consisting of IoT devices, resource-constrained FSs, and resource-rich CSs. In fog computing environment, we used the following server setup, unless it is stated in the experiments: two Raspberrypi 3B (Broadcom BCM2837 4 cores @1.2GHz, 1GB RAM) 1 , one Raspberrypi 4B (ARM Cortex-A72 4 cores @1.5GHz, 4GB RAM) 2 , and one Jetson Nano (ARM Cortex-A57 4 cores @1.43GHz, 4GB RAM, 128-core Maxwell GPU) 3 as heterogeneous FSs.…”

Section: Simulation Setupmentioning

confidence: 99%

See 4 more Smart Citations

A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments

Goudarzi¹,

Palaniswami²,

Buyya³

2021

Preprint

View full text Add to dashboard Cite

Fog/Edge computing is a novel computing paradigm supporting resource-constrained Internet of Things (IoT) devices by placement of their tasks on edge and/or cloud servers. Recently, several Deep Reinforcement Learning (DRL)-based placement techniques have been proposed in fog/edge computing environments, which are only suitable for centralized setups. The training of well-performed DRL agents requires manifold training data while obtaining training data is costly. Hence, these centralized DRL-based techniques lack generalizability and quick adaptability, thus failing to efficiently tackle application placement problems. Moreover, many IoT applications are modeled as Directed Acyclic Graphs (DAGs) with diverse topologies. Satisfying dependencies of DAG-based IoT applications incur additional constraints and increase the complexity of placement problem. To overcome these challenges, we propose an actor-critic-based distributed application placement technique, working based on the IMPortance weighted Actor-Learner Architectures (IMPALA). IMPALA is known for efficient distributed experience trajectory generation that significantly reduces exploration costs of agents. Besides, it uses an adaptive off-policy correction method for faster convergence to optimal solutions. Our technique uses recurrent layers to capture temporal behaviors of input data and a replay buffer to improve the sample efficiency. The performance results, obtained from simulation and testbed experiments, demonstrate that our technique significantly improves execution cost of IoT applications up to 30% compared to its counterparts.

show abstract

Section: Edge Computingmentioning

confidence: 99%

Section: Weighted Cost Modelmentioning

confidence: 99%

Section: Deep Reinforcement Learning Modelmentioning

confidence: 99%

Section: Distributed Drl-based Frameworkmentioning

confidence: 99%

Section: Simulation Setupmentioning

confidence: 99%

See 3 more Smart Citations

A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments

Goudarzi¹,

Palaniswami²,

Buyya³

2021

Preprint

View full text Add to dashboard Cite

show abstract

iQuantum: A toolkit for modeling and simulation of quantum computing environments

Nguyen,

Usman,

Buyya

2024

Softw Pract Exp

View full text Add to dashboard Cite

SummaryQuantum computing resources are predominantly accessible through cloud services, with a potential future shift to edge networks. This paradigm and the increasing global interest in quantum computing have amplified the need for efficient, adaptable resource management strategies and service models for quantum systems. However, many limitations in the quantum resources' quantity, quality, availability, and cost pose significant challenges for conducting research in practical environments. To address these challenges, we proposed iQuantum, a holistic and lightweight discrete‐event simulation toolkit uniquely tailored to model hybrid quantum computing environments. We also present a detailed system model for prototyping and problem formulation in quantum resource management. Through rigorous empirical validation and evaluations using large‐scale quantum workload datasets, we demonstrate the flexibility and applicability of our toolkit in various use cases. iQuantum provides a versatile environment for designing and evaluating quantum resource management policies such as quantum task scheduling, backend selection, hybrid task offloading, and orchestration in the quantum cloud‐edge continuum. Our work endeavors to create substantial contributions to quantum computing modeling and simulation, empowering the creation of future resource management strategies and quantum computing's broader applications.

show abstract

Intelligent Allocation Technologies for All-Scenario KDN Resources

Liao,

He,

Wang

et al. 2024

Wireless Networks

View full text Add to dashboard Cite

Distributed and Collective Deep Reinforcement Learning for Computation Offloading: A Practical Perspective

Cited by 87 publications

References 28 publications

A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments

A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments

iQuantum: A toolkit for modeling and simulation of quantum computing environments

Intelligent Allocation Technologies for All-Scenario KDN Resources

Contact Info

Product

Resources

About