An Approach to Apply Reinforcement Learning for a VNF Scaling Problem

Self Cite

Section: Discussionmentioning

confidence: 99%

A Queueing Model for Threshold-Based Scaling of UPF Instances in 5G Core

Rotter¹,

2021

Self Cite

“…It makes a balance between the service delay and the network cost. The authors of [21] vertically scaled the VNF instance using the Actor-Critic algorithm. To improve resource utilization and service reliability, the algorithm adjusts the number of VNF instances by observing the overheads in the network.…”

Section: Related Workmentioning

confidence: 99%

A Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers

Tang

2020

Service function chaining (SFC) focuses mainly on deploying various network functions in geographically distributed data centers and providing interconnect routing among them. Traditional (convex optimization-based) SFC algorithms exhibit some disadvantages on the scalability and accuracy. Recently, researches have shown the effectiveness of deep reinforcement learning (DRL) in the field of SFC. However, current DRL-based algorithms possess an extremely large action space, which leads to slow convergence and poor scalability. Some researchers relieve this issue by reformulating the SFC problem, which usually results in low utilization and high cost. To address this issue, we develop a hybrid DRL-based framework which decouples the VNF deployment and flow routing into different modules. In the proposed framework, a DRL agent is only responsible for learning the policy of VNF deployment. We customize the structure of the agent base on deep deterministic policy gradient (DDPG) and adopt several techniques to improve the learning efficiency, such as adaptive parameter noise, wolpertinger policy, and prioritized experience replay. The flow routing is conducted in a game-based module (GBM). We design a decentralized routing algorithm for the GBM to address the scalability. The end-to-end latency of flows is minimized while the resource capacity and location constraints are satisfied. During the learning process of the proposed framework, the DRL agent improves its deployment policy with the reward from the GBM (the value of reward depends on flow routing). Thus, the VNF deployment and flow routing are still jointly optimized. Compared to existing DRL-learning algorithms, the proposed hybrid DRL framework can achieve a lower cost since 1) the action space is significantly reduced due to flow routing decoupling; 2) the flow routing procedure is more efficient (the GBM adopts model-based information, e.g., the gradient). Through trace-driven simulations, we show the efficiency of our algorithm compared to existing DRL-based algorithms. INDEX TERMS service function chaining; deep reinforcement learning; game theory; cloud computing

“…Network Intelligence (NI) considers the embedding of Artificial Intelligence (AI) in future networks to fasten service delivery and operations, leverage Quality of Experience (QoE), and guarantee service availability, better agility, resiliency, faster customization, and security [6]. NI is envisioned to manage, pilot, and operate the forthcoming networking built upon SDN, NFV, and cloud [7].…”

Section: Introductionmentioning

confidence: 99%

DQN Approach for Adaptive Self-Healing of VNFs in Cloud-Native Network

Arulappan,

Mahanti,

Passi

et al. 2024

The network transformation from physical network function to Virtual Network Function (VNF) requires a fundamental design change in the way applications and services are tested and assured in a hybrid virtual network. Once the VNFs are onboarded in a cloud network infrastructure, operators need to automatically test VNFs in real-time at the time of instantiation. This paper explicitly analyses the problem of adaptive self-healing of a Virtual Machine (VM) allocated by the VNF with the Deep Reinforcement Learning (DRL) approach. The DRL-based big data collection and analytics engine performs aggregation to probe data and analyze it for troubleshooting and performance management. This engine helps to determine corrective actions (self-healing), such as scaling or migrating VNFs. Hence, we proposed a Deep Queue Learning (DQL) based Deep Queue Networks (DQN) mechanism for self-healing VNFs in the virtualized infrastructure manager. Virtual network probes of closed-loop orchestration perform the automation of the VNF and provide analytics for real-time, policy-driven orchestration in an open networking automation platform through the stochastic gradient descent method for VNF service assurance and network reliability. The proposed DQN/DDQN mechanism optimizes the price and lowers the cost by 18% for resource usage without disrupting the Quality of Service (QoS) provided by the VNF. The outcome of adaptive self-healing of the VNFs enhances the computational performance by 27% compared to other state-of-the-art algorithms.