Virtual Network Function Embedding under Nodal Outage Using Deep Q-Learning

Abstract

With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by virtualizing network functions and placing them on shared commodity servers. However, one of the critical issues in NFV is the resource allocation for the highly complex services; moreover, this problem is classified as an NP-Hard problem. To solve this problem, our work investigates the potential of Deep Reinforcement Learning (DRL) as a swift yet accurate approach (as compared to integer linear programming) for deploying Virtualized Network Functions (VNFs) under several Quality-of-Service (QoS) constraints such as latency, memory, CPU, and failure recovery requirements. More importantly, the failure recovery requirements are focused on the node-outage problem where outage can be either due to a disaster or unavailability of network topology information (e.g., due to proprietary and ownership issues). In DRL, we adopt a Deep Q-Learning (DQL) based algorithm where the primary network estimates the action-value function Q, as well as the predicted Q, highly causing divergence in Q-value’s updates. This divergence increases for the larger-scale action and state-space causing inconsistency in learning, resulting in an inaccurate output. Thus, to overcome this divergence, our work has adopted a well-known approach, i.e., introducing Target Neural Networks and Experience Replay algorithms in DQL. The constructed model is simulated for two real network topologies—Netrail Topology and BtEurope Topology—with various capacities of the nodes (e.g., CPU core, VNFs per Core), links (e.g., bandwidth and latency), several VNF Forwarding Graph (VNF-FG) complexities, and different degrees of the nodal outage from 0% to 50%. We can conclude from our work that, with the increase in network density or nodal capacity or VNF-FG’s complexity, the model took extremely high computation time to execute the desirable results. Moreover, with the rise in complexity of the VNF-FG, the resources decline much faster. In terms of the nodal outage, our model provided almost 70–90% Service Acceptance Rate (SAR) even with a 50% nodal outage for certain combinations of scenarios.

Dynamic Decomposition of Service Function Chain Using a Deep Reinforcement Learning Approach

“…Furthermore, in the literature, most authors employed the shortest-distance technique to discover the path between the substrate nodes to deploy the VNFs and mVNFs, resulting in a highly uneven load on certain links. Thus, to maintain equilibrium across the topology, our studies [8] and [9] opt for a practical link-selection technique and consider a more realistic relationship between the CPU and RAM (nodal resource), which is embraced in this work also. According to [8] (our initial study), over here, the services are deployed using QL models to account for various nodal failures and communication delays between the two VNFs, such as 30 ms, 50 ms, and 100 ms and network density.…”

“…From this experiment, we observed as the network density and complexity increased, the model was affected by the curse of dimensionality, which caused inadequate performance. In contrast, the [9] is formulated from the drawbacks of the [8] model. In [9], the Deep Q Learning (DQL) model performs the deployment, and the model's performance is examined for a) various nodal capacity, b) VNF complexity, c) nodal outage and d) network density.…”

“…In contrast, the [9] is formulated from the drawbacks of the [8] model. In [9], the Deep Q Learning (DQL) model performs the deployment, and the model's performance is examined for a) various nodal capacity, b) VNF complexity, c) nodal outage and d) network density. During this analysis, we came across the overestimation problem caused by the DQL model.…”

“…Solving this would improve service acceptability. Thus, this work adopts the advanced DRL model, DDQL, which overcomes the drawbacks given by the DQL model in [9].…”

“…To support this evolution towards 6G, the NFV architecture, which was initiated in 2012 [3], is also evolving towards a microservices-based architecture [4], [5], [6], [7]. We assume that readers have basic knowledge of NFV; however, for more details on the NFV framework, interested readers can refer to [8], [9], and [10]. Microservices are a standard software design paradigm that decomposes monolithic systems into several smaller (micro) individual and independent fragments.…”