D3PG: Dirichlet DDPG for Task Partitioning and Offloading With Constrained Hybrid Action Space in Mobile-Edge Computing

Ale, Laha; King, Scott A.; Zhang, Ning; Sattar, Abdul; Skandaraniyam, Janahan

doi:10.1109/jiot.2022.3166110

Cited by 33 publications

(9 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within the realm of DRL, DDPG stands out for its effectiveness in handling continuous action spaces, attributed to its utilization of an actor-critic architecture that facilitates stable learning [10], [47]. In our problem (21), the action space comprises both continuous and discrete variables, culminating in a hybrid action space [48]. To navigate this complexity, one could manipulate the continuous action space to select discrete actions [48], [49], e.g., by generating a 10-dimensional action vector with values ranging between 0 and 1, and subsequently choosing the action corresponding to the position of the maximum value.…”

Section: Gdm-based Ddpg With Llms Interaction For Joint Resource Allo...mentioning

confidence: 99%

“…In our problem (21), the action space comprises both continuous and discrete variables, culminating in a hybrid action space [48]. To navigate this complexity, one could manipulate the continuous action space to select discrete actions [48], [49], e.g., by generating a 10-dimensional action vector with values ranging between 0 and 1, and subsequently choosing the action corresponding to the position of the maximum value. This methodology is plausible due to the continuous relaxation of discrete action spaces, allowing gradient-based optimization methods, like DDPG, to operate.…”

Section: Gdm-based Ddpg With Llms Interaction For Joint Resource Allo...mentioning

confidence: 99%

See 1 more Smart Citation

Exploring Collaborative Distributed Diffusion-Based AI-Generated Content (AIGC) in Wireless Networks

Du¹,

Ruichen²,

Niyato³

et al. 2024

IEEE Network

View full text Add to dashboard Cite

Distributed Artificial Intelligence-Generated Content (AIGC) has attracted increasing attention. However, it faces two significant challenges: how to maximize the subjective Quality of Experience (QoE) and how to enhance the energy efficiency, which are particularly pronounced in widely adopted Generative Diffusion Model (GDM)-based AIGC services for image generation. In this paper, we propose a novel user-centric Interactive AI (IAI) approach for service management, with a distributed GDM-based AIGC framework, prioritizing efficient and collaborative GDM deployment. Specifically, we restructure the GDM's inference process, i.e., the denoising chain, to enable users' semantically similar prompts to share a portion of diffusion steps. Furthermore, to maximize the users' subjective QoE, we propose a IAI approach, i.e., Reinforcement Learning With Large Language Models Interaction (RLLI), which utilizes Large Language Model (LLM)-empowered generative agents to replicate users interaction, providing real-time and subjective QoE feedback that reflects a spectrum of user personalities. Lastly, we present the GDM-based Deep Deterministic Policy Gradient (G-DDPG) algorithm, adapted to the proposed RLLI framework, for effective communication and computing resource allocation while considering user subjective personalities and dynamic wireless environments in decision-making. Simulation results show that G-DDPG can increase the sum QoE by 15%, compared with the conventional DDPG algorithm.

show abstract

Section: Gdm-based Ddpg With Llms Interaction For Joint Resource Allo...mentioning

confidence: 99%

Section: Gdm-based Ddpg With Llms Interaction For Joint Resource Allo...mentioning

confidence: 99%

Exploring Collaborative Distributed Diffusion-Based AI-Generated Content (AIGC) in Wireless Networks

Du¹,

Ruichen²,

Niyato³

et al. 2024

IEEE Network

View full text Add to dashboard Cite

show abstract

“…Reference 20 proposes a low‐complexity algorithm based on queuing theory in MEC networks, which minimizes the energy consumption and delay of mobile terminals by optimizing spectrum resources and edge server computing resources. In reference 21, an equal power allocation scheme for uplink is proposed, based on which the energy consumption of equipment is optimized.…”

Section: Related Workmentioning

confidence: 99%

A mechanism for network resource allocation and task offloading in mobile edge computing and network engineering

Shu,

Zhang

2024

Computational Intelligence

View full text Add to dashboard Cite

At present, most of the resource allocation methods in mobile edge computing allocate computing resources according to the time order in which task requests are calculated and unloaded, without considering the priority of tasks in practical applications. According to the computing requirements in such cases, a priority task‐oriented resource allocation method is proposed. According to the average processing time of the task execution, the corresponding priority for task is given. The tasks with different priorities are weighted to allocate computing resources, which not only ensures that the high‐priority tasks obtain sufficient computing resources, but also reduces the total time and energy consumption to complete the calculation of all tasks, thus improving the quality of service. The experimental results show that the proposed method can achieve better performance.

show abstract

“…However, the immediate reward was also affected by other actions; thus, the reward could not be maximized. L. Ale et al [37] proposed the Dirichlet deep deterministic policy (D3PG) to minimize the service times of tasks. They improved the performance by adding noise to the action, based on the Ornstein-Uhlenbeck process in the DDPG structure.…”

Section: Rl-based Computation Offloadingmentioning

confidence: 99%

“…TADPG [34] Single-agent B. Gu [35] Multi-agent DC-DRL [7] Multi-agent L. Ale et al [37] ding53 Single-agent A. Oadder et al [38] Single-agent Proposed Multi-agent…”

Section: Scheme Temporal State Experience Priority Agentmentioning

confidence: 99%

Distributed DRL-Based Computation Offloading Scheme for Improving QoE in Edge Computing Environments

Park

Chung

2023

Sensors

View full text Add to dashboard Cite

Various edge collaboration schemes that rely on reinforcement learning (RL) have been proposed to improve the quality of experience (QoE). Deep RL (DRL) maximizes cumulative rewards through large-scale exploration and exploitation. However, the existing DRL schemes do not consider the temporal states using a fully connected layer. Moreover, they learn the offloading policy regardless of the importance of experience. They also do not learn enough because of their limited experiences in distributed environments. To solve these problems, we proposed a distributed DRL-based computation offloading scheme for improving the QoE in edge computing environments. The proposed scheme selects the offloading target by modeling the task service time and load balance. We implemented three methods to improve the learning performance. Firstly, the DRL scheme used the least absolute shrinkage and selection operator (LASSO) regression and attention layer to consider the temporal states. Secondly, we learned the optimal policy based on the importance of experience using the TD error and loss of the critic network. Finally, we adaptively shared the experience between agents, based on the strategy gradient, to solve the data sparsity problem. The simulation results showed that the proposed scheme achieved lower variation and higher rewards than the existing schemes.

show abstract

D3PG: Dirichlet DDPG for Task Partitioning and Offloading With Constrained Hybrid Action Space in Mobile-Edge Computing

Cited by 33 publications

References 41 publications

Exploring Collaborative Distributed Diffusion-Based AI-Generated Content (AIGC) in Wireless Networks

Exploring Collaborative Distributed Diffusion-Based AI-Generated Content (AIGC) in Wireless Networks

A mechanism for network resource allocation and task offloading in mobile edge computing and network engineering

Distributed DRL-Based Computation Offloading Scheme for Improving QoE in Edge Computing Environments

Contact Info

Product

Resources

About