Monkey Business: Reinforcement learning meets neighborhood search for Virtual Network Embedding

Elkael, Maxime; Ait, Massinissa; Araldo, Andrea; Castel-Taleb, Hind; Jouaber, Badii

doi:10.1016/j.comnet.2022.109204

Cited by 5 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another set of works, to which the present article belongs, focuses on flow-based models that take paths into account. To our knowledge, a large part of these works solves the so-called Virtual Network Embedding (VNE) problem [3], [15]. In VNE, a virtual graph has to be placed on the physical network.…”

Section: Related Workmentioning

confidence: 99%

“…The most common objective is to minimize resource consumption (measured in terms of CPU and bandwidth) [15], [17]. Various techniques have been studied such as metaheuristics [18], [19], Reinforcement Learning [3], [4], [20] or integer linear programming [15], [21].…”

Section: Related Workmentioning

confidence: 99%

“…This assumption, common in recent works [2], [3], [15], is reasonable since we consider that the PNO reserves to each flow the amount of bandwidth demanded by the respective MVNO on the entire route. We assume MVNOs request a sufficient amount of bandwidth, so that large queuing times on such links are unlikely to happen.…”

Section: System Model and Assumptionsmentioning

confidence: 99%

See 2 more Smart Citations

Joint Placement, Routing and Dimensioning at the Network Edge for Energy Minimization

Elkael,

Araldo,

D'Oro

et al. 2023

GLOBECOM 2023 - 2023 IEEE Global Communications Conference

Self Cite

View full text Add to dashboard Cite

Thanks to resource virtualization, Physical Network Operators (PNOs) can share their 5G network to multiple Mobile Virtual Network Operators (MVNOs) which can leverage the shared physical infrastructure to deploy their services up to the edge. This allows much more flexibility with respect to the previous generation of cellular networks: MVNO software components can be placed at different locations, can be allocated a certain amount of virtual resources (e.g., bandwidth, CPU cycles), and be reachable via different paths. To the best of our knowledge, strategies to minimize energy consumption while satisfying Service Level Agreements (SLAs) between the PNO and the MVNOs are still largely missing, particularly if it is required to take the nonlinearity of delays into account. To fill this gap, we formulate the problem of joint placement of software components, routing of user requests and resource dimensioning. SLAs are represented in terms of latency and reliability constraints. Via Column Generation, we obtain exact solutions in real-sized networks. Our numerical results show that we can save up to 50% energy in networks with up to 30 nodes compared to the state-of-the-art algorithms, which are focused on placement or resource minimization.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Joint Placement, Routing and Dimensioning at the Network Edge for Energy Minimization

Elkael,

Araldo,

D'Oro

et al. 2023

GLOBECOM 2023 - 2023 IEEE Global Communications Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…An RL agent learns to find the optimal VNE solution through interaction with the network environment, and it obtains a policy model that maximizes a predefined reward function. Several RL algorithms, including simple Q-learning, deep Q-learning (DQN), advantage actor-critic (A2C), asynchronous advantage actor-critic (A3C), proximal policy optimization (PPO), and deep deterministic policy gradient (DDPG) have been utilized to improve the performance of VNE algorithms [23]- [32]. Most existing methods solve the VNE problem by first embedding all nodes without considering link embedding information.…”

Section: Introductionmentioning

confidence: 99%

Virtual Network Embedding Based on Hierarchical Cooperative Multiagent Reinforcement Learning

Lim,

Ullah,

Kim

et al. 2024

IEEE Internet Things J.

View full text Add to dashboard Cite

Virtual network embedding (VNE) is a promising technique enabling 5G networks to satisfy the given requirements of each service via network virtualization. For better performance of the embedding algorithm, it is necessary to automatically detect the network status and provide an optimal embedding decision. However, existing VNE algorithms disregard the longterm effect by focusing on selecting only one virtual network request from the waiting queue, without considering all waiting virtual network requests concurrently. In this study, we propose a hierarchical cooperative multi-agent reinforcement learning algorithm to optimize the VNE problem by maximizing average revenue, minimizing average cost, and also improving the request acceptance ratio. The proposed algorithm applies two reinforcement learning algorithms: 1) two-level hierarchical reinforcement learning to efficiently solve the problem by dividing it into subproblems, and 2) multi-agent-based cooperative reinforcement learning to improve algorithm performance through the cooperation of multiple agents. In order to evaluate and analyze the proposed scheme from the long-term perspective, four performance parameters are evaluated: revenue, cost, revenue-to-cost ratio, and acceptance ratio. The simulation results demonstrate that the proposed VNE algorithm based on hierarchical and multiagent reinforcement learning outperforms the existing RL-based approaches.

show abstract

“…combines nested search, memorizing the best sequence of moves found at each level, and the online learning of a playout policy using this sequence. NRPA has world records in Morpion Solitaire and crossword puzzles and has also been applied to many other combinatorial problems such as the Traveling Salesman Problem with Time Windows [16,21], 3D Packing with Object Orientation [23], the physical traveling salesman problem [24], the Multiple Sequence Alignment problem [25], Logistics [22,13], Graph Coloring [14], Vehicle Routing Problems [22,12], Network Traffic Engineering [19], Virtual Network Embedding [26] or the Snake in the Box [20].…”

Section: Introductionmentioning

confidence: 99%

Generalized Nested Rollout Policy Adaptation

Cazenave¹

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

Generalized Nested Rollout Policy Adaptation (GNRPA) is a Monte Carlo search algorithm for optimizing a sequence of choices. We propose to improve on GNRPA by avoiding too deterministic policies that find again and again the same sequence of choices. We do so by limiting the number of repetitions of the best sequence found at a given level. Experiments show that it improves the algorithm for three different combinatorial problems: Inverse RNA Folding, the Traveling Salesman Problem with Time Windows and the Weak Schur problem.

show abstract

Monkey Business: Reinforcement learning meets neighborhood search for Virtual Network Embedding

Cited by 5 publications

References 20 publications

Joint Placement, Routing and Dimensioning at the Network Edge for Energy Minimization

Joint Placement, Routing and Dimensioning at the Network Edge for Energy Minimization

Virtual Network Embedding Based on Hierarchical Cooperative Multiagent Reinforcement Learning

Generalized Nested Rollout Policy Adaptation

Contact Info

Product

Resources

About