SCHEMA: Service Chain Elastic Management with Distributed Reinforcement Learning

Abstract

As the demand for Network Function Virtualization accelerates, service providers are expected to advance the way they manage and orchestrate their network services to offer lower latency services to their future users. Modern services require complex data flows between Virtual Network Functions, placed in separate network domains, risking an increase in latency that compromises the offered latency constraints. This shift requires high levels of automation to deal with the scale and load of future networks. In this paper, we formulate the Service Function Chaining (SFC) placement problem and then we tackle it by introducing SCHEMA, a Distributed Reinforcement Learning (RL) algorithm that performs complex SFC orchestration for low latency services. We combine multiple RL agents with a Bidding Mechanism to enable scalability on multi-domain networks. Finally, we use a simulation model to evaluate SCHEMA, and we demonstrate its ability to obtain a 60.54% reduction of average service latency when compared to a centralised RL solution.

Towards Massive Distribution of Intelligence for 6G Network Management using Double Deep Q-Networks

“…Dalgkitsis et al [31] introduce SCHEMA, a distributed RL framework, that addresses the SFC placement problem for low latency URLLC services. Being one of the few works to address scalability of distributed learning, the authors in this study assign a local agent to each domain level network graph.…”

Survivable SFC deployment method based on federated learning in multi-domain networks

“…Dalgkitsis et al modeled the SFC deployment problem and solved the multi-domain SFC deployment process for low latency requirements by using distributed reinforcement learning. Compared with the centralized reinforcement learning scheme, it has a better delay optimization effect [14]. Toumi et al used the advantages of deep reinforcement learning to solve the problem of SFC deployment in a multi-domain network environment, and proposed a deep deterministic strategy gradient method, which ensures an approximately optimal deployment cost and transmission delay on the basis of a low service rejection rate [15].…”

“…However, there are still some problems to be considered in the current method, such as insufficient consideration of dynamics, reliability and isolation. In view of these shortcomings, some studies have solved the large-scale SFC deployment problem in dynamic scenarios by using relevant machine learning methods [14][15]. In addition, some studies have proposed effective SFC deployment algorithms for the isolation requirements between network domains [16][17].…”

AI-driven predictive and scalable management and orchestration of network slices

“…In the process of further advancing the sophistication of algorithms for SFC placement, the authors in [22] proposed DRL to perform the placement of virtual network function forwarding graphs considering the constraints of the underlying infrastructure. The authors of [23] used DRL as in [22] for building an orchestration solution consisting of multiple DRL agents with the objectives of minimizing latency across SFCs and minimizing energy consumption. This approach also has the capability of orchestrating VNFs across multiple domains in a network.…”