Extendable NFV-Integrated Control Method Using Reinforcement Learning

Abstract

Network functions virtualization (NFV) enables telecommunications service providers to realize various network services by flexibly combining multiple virtual network functions (VNFs). To provide such services, an NFV control method should optimally allocate such VNFs into physical networks and servers by taking account of the combination(s) of objective functions and constraints for each metric defined for each VNF type, e.g., VNF placements and routes between the VNFs. The NFV control method should also be extendable for adding new metrics or changing the combination of metrics. One approach for NFV control to optimize allocations is to construct an algorithm that simultaneously solves the combined optimization problem. However, this approach is not extendable because the problem needs to be reformulated every time a new metric is added or a combination of metrics is changed. Another approach involves using an extendable network-control architecture that coordinates multiple control algorithms specified for individual metrics. However, to the best of our knowledge, no method has been developed that can optimize allocations through this kind of coordination. In this paper, we propose an extendable NFV-integrated control method by coordinating multiple control algorithms. We also propose an efficient coordination algorithm based on reinforcement learning. Finally, we evaluate the effectiveness of the proposed method through simulations.

Cooperative Multi-Agent Deep Reinforcement Learning for Dynamic Virtual Network Allocation With Traffic Fluctuations

“…Note that, though the VN model in this paper is assumed to consist of a single VM, it can be extended to more complex VN models consisting of a graph with multiple VMs and virtual link(s) by using an extendable NFV-integrated control architecture [22]. They [22] define 12 types of VN models and describe how to extend the formulation of the RL-based VN allocation algorithm when changing the VN model from one model to the other models.…”

“…Note that, though the VN model in this paper is assumed to consist of a single VM, it can be extended to more complex VN models consisting of a graph with multiple VMs and virtual link(s) by using an extendable NFV-integrated control architecture [22]. They [22] define 12 types of VN models and describe how to extend the formulation of the RL-based VN allocation algorithm when changing the VN model from one model to the other models. For example, they describe how to extend the VN model consisting of one VM to the VN model consisting of the chain of multiple VMs assumed in the use case of service function chaining (SFC) (see [22] for details).…”

“…They [22] define 12 types of VN models and describe how to extend the formulation of the RL-based VN allocation algorithm when changing the VN model from one model to the other models. For example, they describe how to extend the VN model consisting of one VM to the VN model consisting of the chain of multiple VMs assumed in the use case of service function chaining (SFC) (see [22] for details). Though this formulation can be extended to more complex VN models in principle, its performance has not been evaluated yet, and its evaluation is one of the future works.…”

“…In this formulation, the action sets that are changed by candidate VN allocation are determined only from the VM allocation, and the routes between users and VMs are uniquely determined in the environment update. We update the environment on the basis of the extendable NFV-integrated control architecture [22] that coordinates multiple control algorithms specified for individual metrics. We use the route-optimization algorithm as the specified optimization algorithm.…”

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

“…1) VNF Placement and Scheduling: In the literature, the authors propose several solutions for VNF placement and the SFC mapping. In [20], the authors employ an efficient coordination algorithm to allocate VNFs into physical networks and route between the VNFs based on reinforcement learning (RL). Although their proposed model adaptively changes the VNF placement based on the network condition, they do not focus on the QoS parameters as well as AoI and the VNF placement cost.…”