Multi-agent deep reinforcement learning algorithm with self-adaption division strategy for VNF-SC deployment in SDN/NFV-Enabled Networks

Xuan, Hejun; Zhou, Yi; Zhao, Xuelin; Liu, Zhenghui

doi:10.1016/j.asoc.2023.110189

Cited by 5 publications

(5 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…8.B illustrates the number of supported requests. In this figure, the results of DDQL-CCRA are compared with those of WF-CCRA, FSA [13], BSA [13], CEP [10], A-DDPG [11], and MDRL-SaDS [14]. FSA is a heuristic algorithm that randomly assigns resources to requests in descending order of their required computing capacity.…”

Section: Numerical Resultsmentioning

confidence: 99%

“…A-DDPG, where requests are assigned to nodes with a lower E2E delay, is another costly method. Increasing the number of requests , BSA [13], CEP [10], A-DDPG [11], and MDRL-SaDS [14] vs. network size. In this scenario, the first four nodes are added to the first tier, followed by the second four nodes to the second tier, and then the last four nodes to the third tier.…”

Section: Numerical Resultsmentioning

confidence: 99%

“…Iwamoto et al [12] investigated the problem of scheduling VNF migrations in order to optimize the QoS degradation of all traffic flows and proposed a stochastic method on the basis of the load degree of VNF instances. Link [13] max profit ✓ ✓ ✓ ✓ ✓ [10] min cost ✓ ✓ ✓ ✓ [11] max profit -cost ✓ ✓ ✓ ✓ [14] min energy ✓ ✓ ✓ ✓ ✓ [15] min cost ✓ ✓ ✓ ✓ [12] max fairness ✓ ✓ ✓ ✓ [16] max profit -cost ✓ ✓ ✓ ✓ [7] min latency ✓ ✓ ✓ ✓ [8] min latency ✓ ✓ ✓ [9] min cost ✓ ✓ ✓ [17] max rate ✓ ✓ ✓ ✓ [18] min cost ✓ ✓ ✓ ✓ ✓ [19] max rate ✓ ✓ ✓ ✓ ✓ [20] min latency + cost ✓ ✓ ✓ ✓ [21] min latency…”

Section: Introductionmentioning

confidence: 99%

“…They took into account traffic routing constraints and proposed heuristic approaches to address the problem. By considering energy consumption as the most significant cost associated with networking and computing resources, Xuan et al [14] addressed the same problem by proposing an algorithm based on a multi-agent DRL and a self-adaptation division strategy. Nguyen et al [13] investigated the problem of VNF placement, where requests are weighted according to their priority and the goal is to maximize the total weight of services accepted for deployment on the infrastructure.…”

Section: Introductionmentioning

confidence: 99%

“…Fig.8. A) Mean cost of each supported request, B) number of supported requests, and C) mean E2E delay of each supported request for DDQL-CCRA, WF-CCRA, FSA & BSA[13], CEP[10], A-DDPG[11], and MDRL-SaDS[14] vs. request burstiness. The delay requirement of requests is 10 ms, and the number of network nodes is 9.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Double Deep Q-Learning-Based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

Shokrnezhad

Taleb

Dazzi

2024

IEEE Trans. on Mobile Comput.

View full text Add to dashboard Cite

Nowadays, as the need for capacity continues to grow, entirely novel services are emerging. A solid cloudnetwork integrated infrastructure is necessary to supply these services in a real-time responsive, and scalable way. Due to their diverse characteristics and limited capacity, communication and computing resources must be collaboratively managed to unleash their full potential. Although several innovative methods have been proposed to orchestrate the resources, most ignored network resources or relaxed the network as a simple graph, focusing only on cloud resources. This paper fills the gap by studying the joint problem of communication and computing resource allocation, dubbed CCRA, including function placement and assignment, traffic prioritization, and path selection considering capacity constraints and quality requirements, to minimize total cost. We formulate the problem as a non-linear programming model and propose two approaches, dubbed B&B-CCRA and WF-CCRA, based on the Branch & Bound and Water-Filling algorithms to solve it when the system is fully known. Then, for partially known systems, a Double Deep Q-Learning (DDQL) architecture is designed. Numerical simulations show that B&B-CCRA optimally solves the problem, whereas WF-CCRA delivers near-optimal solutions in a substantially shorter time. Furthermore, it is demonstrated that DDQL-CCRA obtains near-optimal solutions in the absence of request-specific information.

show abstract