A novel reinforcement learning algorithm for virtual network embedding

Hu, Yao; Xu, Chen; Li, Maozhen; Zhang, Peiying; Wang, Luyao

doi:10.1016/j.neucom.2018.01.025

Cited by 127 publications

(60 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FFD approach adopts First-Fit algorithm for allocating VNFs to the substrate network and Dijkstra algorithm (the shortest path in terms of the number of hops) to define the has been used as one of the baselines to assess the performance in [57]- [59]. With given VNF allocation, Dijkstra algorithm could be adopted to determine the path to connect VNFs [60]. In DDPG-HFA and E 2 D 2 PG, the locations of VNFs and the substrate paths of VLs are determined by the neural networks and HFA.…”

Section: B Impact Of Hfamentioning

confidence: 99%

A Deep Reinforcement Learning Approach for VNF Forwarding Graph Embedding

Quang

Hadjadj‐Aoul

Outtagarts

2019

IEEE Trans. Netw. Serv. Manage.

121

View full text Add to dashboard Cite

Network Function Virtualization (NFV) and service orchestration simplify the deployment and management of network and telecommunication services. The deployment of these services requires, typically, the allocation of Virtual Network Function -Forwarding Graph (VNF-FG), which implies not only the fulfillment of the service's requirements in terms of Quality of Service (QoS), but also considering the constraints of the underlying infrastructure. This topic has been well-studied in existing literature, however, its complexity and uncertainty of available information unveil challenges for researchers and engineers. In this paper, we explore the potential of reinforcement learning techniques for the placement of VNF-FGs. However, it turns out that even the most well-known learning technique is ineffective in the context of a large-scale action space. In this respect, we propose approaches to find out feasible solutions while improving significantly the exploration of the action space. The simulation results clearly show the effectiveness of the proposed learning approach for this category of problems. Moreover, thanks to the deep learning process, the performance of the proposed approach is improved over time.

show abstract

Section: B Impact Of Hfamentioning

confidence: 99%

A Deep Reinforcement Learning Approach for VNF Forwarding Graph Embedding

Quang

Hadjadj‐Aoul

Outtagarts

2019

IEEE Trans. Netw. Serv. Manage.

121

View full text Add to dashboard Cite

show abstract

“…In addition, the proposed algorithms are evaluated through comparisons with a Qlearning-based approach. A policy network based on reinforcement learning is designed and implemented in [8]. Authors use policy gradient to achieve optimization automatically by training the policy network with the historical data based on virtual network requests.…”

Section: B Application Of Artificial Intelligence Technology In Vnementioning

confidence: 99%

VNE-SDN Algorithms for Different Physical Network Environments

2020

View full text Add to dashboard Cite

Network virtualization (NV) is widely considered as one of the key technologies for future networks, which allows multiple virtual networks (VNs) to run on the same substrate network simultaneously. Software defined networking (SDN) can be used as a platform to realize NV. Therefore, the NV-based SDN has attracted intensive attention from both academic and industry in recent years. In this article, we focus on virtual network embedding (VNE) problem, which is one of the most important technologies of NV. Specifically, resource situation of underlying network (RSUN) is not considered in previous VNE algorithms, which seriously affects the efficiency of VNE. To solve this problem, we develop two heuristic algorithms to select the appropriate VNE according to RSUN. Our proposed algorithms adopt a novel node-ranking method, which fully considers the node topology and resource attributes, to sort physical nodes. In addition, two sorting approaches for virtual nodes and the global matching embedding (GME) approach are proposed. Our numerical analyses validate our proposed schemes and algorithms in terms of the average revenue to cost ratio, the average revenue, and the average VN request acceptance ratio. INDEX TERMS Network virtualization, virtual network embedding, SDN, global resource matching, node-ranking approach.

show abstract

“…e Monte Carlo tree search algorithm is used to design an action strategy (node mapping) for the proposed MDP. Yao et al [41] used reinforcement learning to study virtual network mapping problems.…”

Section: Heuristic Solutions Vs Exact Solutions Vs Metaheuristicmentioning

confidence: 99%

EPVNE: An Efficient Parallelizable Virtual Network Embedding Algorithm

Zhang

2019

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Virtual network embedding (VNE) problem is a key issue in network virtualization technology, and much attention has been paid to the virtual network embedding. However, very little research work focuses on parallelized virtual network embedding problems which assumes that the substrate infrastructure supports parallel computing and allows one virtual node to be mapped to multiple substrate nodes. Based on the work of Liang and Zhang, we extend the well-known VNE to parallelizable virtual network embedding (PVNE) in this paper. Furthermore, to the best of our knowledge, we give the first formulation of the PVNE problem. A new heuristic algorithm named efficient parallelizable virtual network embedding (EPVNE) is proposed to reduce the cost of embedding the VN request and increase the VN request acceptance ratio. EPVNE is a two-stage mapping algorithm, which first performs node mapping and then performs link mapping. In the node mapping phase, we present a simple and efficient virtual node and physical node sorting formula and perform the virtual node mapping in order. When mapping virtual nodes, we map virtual nodes to physical nodes that just meet the CPU requirements. Substrate nodes with more CPU resources will be retained for subsequent virtual network mapping requests. In the link mapping phase, Dijkstra’s algorithm is used to find a substrate path for each virtual link. Finally, simulations are carried out and simulation results show that our algorithm performs better than the existing heuristic algorithms.

show abstract

A novel reinforcement learning algorithm for virtual network embedding

Cited by 127 publications

References 16 publications

A Deep Reinforcement Learning Approach for VNF Forwarding Graph Embedding

A Deep Reinforcement Learning Approach for VNF Forwarding Graph Embedding

VNE-SDN Algorithms for Different Physical Network Environments

EPVNE: An Efficient Parallelizable Virtual Network Embedding Algorithm

Contact Info

Product

Resources

About