Virtual Network Embedding via Monte Carlo Tree Search

Abstract

Network virtualization helps overcome shortcomings of the current Internet architecture. The virtualized network architecture enables coexistence of multiple virtual networks (VNs) on an existing physical infrastructure. VN embedding (VNE) problem, which deals with the embedding of VN components onto a physical network, is known to be -hard. In this paper, we propose two VNE algorithms: MaVEn-M and MaVEn-S. MaVEn-M employs the multicommodity flow algorithm for virtual link mapping while MaVEn-S uses the shortest-path algorithm. They formalize the virtual node mapping problem by using the Markov decision process (MDP) framework and devise action policies (node mappings) for the proposed MDP using the Monte Carlo tree search algorithm. Service providers may adjust the execution time of the MaVEn algorithms based on the traffic load of VN requests. The objective of the algorithms is to maximize the profit of infrastructure providers. We develop a discrete event VNE simulator to implement and evaluate performance of MaVEn-M, MaVEn-S, and several recently proposed VNE algorithms. We introduce profitability as a new performance metric that captures both acceptance and revenue to cost ratios. Simulation results show that the proposed algorithms find more profitable solutions than the existing algorithms. Given additional computation time, they further improve embedding solutions.

A Deep Reinforcement Learning Approach for VNF Forwarding Graph Embedding

“…These techniques, although effective, can present problems of convergence and therefore slowness. More recently, techniques based on deep learning have been used [7]. These techniques show some effectiveness but are simply ineffective when the space of actions is very wide.…”

“…However, very few studies have tackled the whole issue. In [7], the authors consider a Markov Decision Process (MDP) to model a decision making strategy consisting in placing the VNFs and, then, connecting them using the shortest path algorithm. As finding the optimal policy using such a strategy is intractable given the state space, the authors proposed a Monte Carlo Tree Search (MCTS) for solving the MDP.…”

Self‐adaptive online virtual network migration in network virtualization environments

“…We can distinguish between static and dynamic VNE. More recent works 16,17 have introduced the use of RL techniques while addressing the VNE problem. 11 Some works, like the ones reported by Chowdhury et al 12 and Shanbhag et al, 13 solve the VNE problem in one-shot, assigning virtual nodes and links simultaneously, using linear programming or mixed integer programming techniques.…”

“…Static VNE solutions try to solve the twofold problem of node mapping and link mapping considering that all VNRs are known in advance 3 or that each new arrival of a VNR is a new embedding problem to be solved with the available resources at the time of the arrival. In the work of Haeri and Trajković, 16 a Markov decision process and the Monte Carlo tree search technique are used to solve the node and link mapping problem. In other works, 14,15 node and link mappings are solved independently, usually by first ranking the nodes to select the candidate ones and then finding the shortest paths among the available links.…”

Service disruption index (SDI): A novel user perspective reliability indicator for modern virtual service networks

“…In the literature, most of the advances in network virtualization concentrate on virtual resource allocation, which is known as the virtual network embedding (VNE) problem. [11][12][13][14][15][16][17][18] Moreover, VNE algorithms deal with the allocation of virtual topology onto substrate infrastructure with regard to different objectives such as cost efficiency, 15 energy efficiency, 16 QoS, 17,18 etc.…”