Reinforcement Learning for Service Function Chain Reconfiguration in NFV-SDN Metro-Core Optical Networks

Troia, Sebastian; Alvizu, Rodolfo; Maier, Guido

doi:10.1109/access.2019.2953498

Cited by 39 publications

(17 citation statements)

References 34 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The action performed by the Fig. 1.3 Schema of the proposed RL system [18] agent will produce another state of the environment and a reward at a time t + 1 and so on. The environment, namely network environment, consists of a multi-layer model formulation based on Mixed ILP (MILP) that, given a set of SFC requests, finds the optimal VNF placement and Routing and Wavelength Assignment (RWA).…”

Section: Reinforcement Learning For Adaptive Network Resource Allocationmentioning

confidence: 99%

See 1 more Smart Citation

Machine-Learning Defined Networking: Towards Intelligent Networks

Troia

2021

Special Topics in Information Technology

Self Cite

View full text Add to dashboard Cite

With the advent of 5G technology and an ever-increasing traffic demand, today Communication Service Providers (CSPs) experience a progressive congestion of their networks. The operational complexity, the use of manual configuration, the static nature of current technologies together with fast-changing traffic profiles lead to: inefficient network utilization, over-provisioning of resources and very high Capital Expenditures (CapEx) and Operational Expenses (OpEx). This situation is forcing the CSPs to change their underlying network technologies, and have started to look at new technological solutions that increase the level of programmability, control, and flexibility of configuration, while reducing the overall costs related to network operations. Software Define Networking (SDN), Network Function Virtualization (NFV) and Machine Learning (ML) are accepted as effective solutions to reduce CapEx and OpEx and to boost network innovation. This chapter summarizes the content of my Ph.D. thesis, by presenting new ML-based approaches in order to efficiently optimize resources in 5G metro-core SDN/NFV networks. The main goal is to provide the modern CSP with intelligent and dynamic network optimization tools in order to address the requirements of increasing traffic demand and 5G technology.

show abstract

Section: Reinforcement Learning For Adaptive Network Resource Allocationmentioning

confidence: 99%

“…The methodology and the detailed results can be found in Chap. 6 of the Ph.D. thesis and in the following paper: [18].…”

Section: Reinforcement Learning For Adaptive Network Resource Allocationmentioning

confidence: 99%

Machine-Learning Defined Networking: Towards Intelligent Networks

Troia

2021

Special Topics in Information Technology

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sang Il Kim et al considered the consumption of CPU and memory resources, then utilized reinforcement learning to solve SFC optimization problem dynamically [ 17 ], yet the algorithm was not satisfied with delay sensitive SFC. Sebastian Troia et al investigated the application of reinforcement learning for performing dynamic SFC resources allocation in NFV-SDN enabled metro-core optical networks [ 18 ], they decided how to reconfigure the SFCs according to state of the network and historical traffic traces. However, reinforcement learning exists the limitation that it cannot satisfy the networks which has large-scale states.…”

Section: Related Workmentioning

confidence: 99%

“…However, traditional backup scheme is too simple to satisfy complex SFC requests. On the other hand, more advanced and powerful reinforcement learning models have significantly achieved performance gains in SFC deployment problem recently [ 16 , 17 , 18 ]. However, Q-learning, a classical algorithm of reinforcement learning, needs to maintain a quite large Q-table because of large state and action set, so that the computing power of the algorithm will be affected with wasting of CPU and memory.…”

Section: Introductionmentioning

confidence: 99%

Reliable Service Function Chain Deployment Method Based on Deep Reinforcement Learning

Wang

Zhao

2021

Sensors

View full text Add to dashboard Cite

Network function virtualization (NFV) is a key technology to decouple hardware device and software function. Several virtual network functions (VNFs) combine into a function sequence in a certain order, that is defined as service function chain (SFC). A significant challenge is guaranteeing reliability. First, deployment server is selected to place VNF, then, backup server is determined to place the VNF as a backup which is running when deployment server is failed. Moreover, how to determine the accurate locations dynamically with machine learning is challenging. This paper focuses on resource requirements of SFC to measure its priority meanwhile calculates node priority by current resource capacity and node degree, then, a novel priority-awareness deep reinforcement learning (PA-DRL) algorithm is proposed to implement reliable SFC dynamically. PA-DRL determines the backup scheme of each VNF, then, the model jointly utilizes delay, load balancing of network as feedback factors to optimize the quality of service. In the experimental results, resource efficient utilization, survival rate, and load balancing of PA-DRL were improved by 36.7%, 35.1%, and 78.9% on average compared with benchmark algorithm respectively, average delay was reduced by 14.9%. Therefore, PA-DRL can effectively improve reliability and optimization targets compared with other benchmark methods.

show abstract

“…The policy determines the action to be taken when the perceived states of the environment are given. The authors of [4]- [6] have proven the efficiency of DRL in the field of SFC.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers

Tang

2020

IEEE Access

View full text Add to dashboard Cite

Service function chaining (SFC) focuses mainly on deploying various network functions in geographically distributed data centers and providing interconnect routing among them. Traditional (convex optimization-based) SFC algorithms exhibit some disadvantages on the scalability and accuracy. Recently, researches have shown the effectiveness of deep reinforcement learning (DRL) in the field of SFC. However, current DRL-based algorithms possess an extremely large action space, which leads to slow convergence and poor scalability. Some researchers relieve this issue by reformulating the SFC problem, which usually results in low utilization and high cost. To address this issue, we develop a hybrid DRL-based framework which decouples the VNF deployment and flow routing into different modules. In the proposed framework, a DRL agent is only responsible for learning the policy of VNF deployment. We customize the structure of the agent base on deep deterministic policy gradient (DDPG) and adopt several techniques to improve the learning efficiency, such as adaptive parameter noise, wolpertinger policy, and prioritized experience replay. The flow routing is conducted in a game-based module (GBM). We design a decentralized routing algorithm for the GBM to address the scalability. The end-to-end latency of flows is minimized while the resource capacity and location constraints are satisfied. During the learning process of the proposed framework, the DRL agent improves its deployment policy with the reward from the GBM (the value of reward depends on flow routing). Thus, the VNF deployment and flow routing are still jointly optimized. Compared to existing DRL-learning algorithms, the proposed hybrid DRL framework can achieve a lower cost since 1) the action space is significantly reduced due to flow routing decoupling; 2) the flow routing procedure is more efficient (the GBM adopts model-based information, e.g., the gradient). Through trace-driven simulations, we show the efficiency of our algorithm compared to existing DRL-based algorithms. INDEX TERMS service function chaining; deep reinforcement learning; game theory; cloud computing

show abstract

Reinforcement Learning for Service Function Chain Reconfiguration in NFV-SDN Metro-Core Optical Networks

Abstract: The work leading to these results has been supported by the European Community under grant agreement no. 761727 Metro-Haul Project.

Cited by 39 publications

References 34 publications

Machine-Learning Defined Networking: Towards Intelligent Networks

Machine-Learning Defined Networking: Towards Intelligent Networks

Reliable Service Function Chain Deployment Method Based on Deep Reinforcement Learning

A Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers

Contact Info

Product

Resources

About