Virtual Network Function Embedding under Nodal Outage using Reinforcement Learning

Abstract

With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by disintegrating the dependency on the hardware devices via virtualizing the network functions and placing them on shared data centres. However, one of the main challenges of the NFV paradigm is the resource allocation problem which is known as NFV-Resource Allocation (NFV-RA). NFV-RA is a method of deploying software-based network functions on the substrate nodes, subject to the constraints imposed by the underlying infrastructure and the agreed Service Level Agreement (SLA). This work investigates the potential of Reinforcement Learning (RL) as a fast yet accurate means (as compared to integer linear programming) for deploying the softwarized network functions onto substrate networks under several Quality of Service (QoS) constraints. In addition to the regular resource constraints and latency constraints, we introduced the concept of a complete outage of certain nodes in the network. This outage can be either due to a disaster or unavailability of network topology information due to proprietary and ownership issues. We have analyzed the network performance on different network topologies, different capacities of the nodes and the links, and different degrees of the nodal outage. The computational time escalated with the increase in the network density to achieve the optimal solutions; this is because Q-Learning is an iterative process which results in a slow exploration. Our results also show that for certain topologies and a certain combination of resources, we can achieve between 70-90% service acceptance rate even with a 40% nodal outage.

Dynamic Decomposition of Service Function Chain Using a Deep Reinforcement Learning Approach

“…Furthermore, in the literature, most authors employed the shortest-distance technique to discover the path between the substrate nodes to deploy the VNFs and mVNFs, resulting in a highly uneven load on certain links. Thus, to maintain equilibrium across the topology, our studies [8] and [9] opt for a practical link-selection technique and consider a more realistic relationship between the CPU and RAM (nodal resource), which is embraced in this work also. According to [8] (our initial study), over here, the services are deployed using QL models to account for various nodal failures and communication delays between the two VNFs, such as 30 ms, 50 ms, and 100 ms and network density.…”

“…Thus, to maintain equilibrium across the topology, our studies [8] and [9] opt for a practical link-selection technique and consider a more realistic relationship between the CPU and RAM (nodal resource), which is embraced in this work also. According to [8] (our initial study), over here, the services are deployed using QL models to account for various nodal failures and communication delays between the two VNFs, such as 30 ms, 50 ms, and 100 ms and network density. From this experiment, we observed as the network density and complexity increased, the model was affected by the curse of dimensionality, which caused inadequate performance.…”

“…From this experiment, we observed as the network density and complexity increased, the model was affected by the curse of dimensionality, which caused inadequate performance. In contrast, the [9] is formulated from the drawbacks of the [8] model. In [9], the Deep Q Learning (DQL) model performs the deployment, and the model's performance is examined for a) various nodal capacity, b) VNF complexity, c) nodal outage and d) network density.…”

“…To support this evolution towards 6G, the NFV architecture, which was initiated in 2012 [3], is also evolving towards a microservices-based architecture [4], [5], [6], [7]. We assume that readers have basic knowledge of NFV; however, for more details on the NFV framework, interested readers can refer to [8], [9], and [10]. Microservices are a standard software design paradigm that decomposes monolithic systems into several smaller (micro) individual and independent fragments.…”

Virtual Network Function Embedding under Nodal Outage Using Deep Q-Learning

“…A summary of the related works is presented in Table 1, where M, RT, Rel, NodOut, CT, BW, and PD represents Methodology, Resources Type, Relationship between the CPU and RAM, Nodal Outage, Computation Time, Bandwidth and Propagation Delay respectively. In our previous work [17], we have considered the QL model for solving the VNF-FGE problem. QL is a robust off-policy model-free algorithm but practices an iterative method to achieve the solution, causing the degradation in the performance with the increase in the network complexity and density.…”