Request delay and survivability optimization for software defined‐wide area networking (SD‐WAN) using multi‐agent deep reinforcement learning

Abstract

Data exchange between headquarters and local branches represents a major challenge issue for business success. For this issue, traditional solutions applied to wide area networks (WAN) remain unrealistic and require a good knowledge of the systems. Recently, software‐defined wide area networking (SD‐WAN) plays a pivotal role and constitutes, in general, a reliable solution for wide area networking. Compared to the classical WAN, SD‐WAN decouples the control plane from gateway devices. Moreover, SD‐WAN are based on centralized WAN management with dynamic reconfiguration, allowing it to control the entire service requirements. Nevertheless, traffic management is the most usual metric in SD‐WAN. It gives a clear indication on propagation latency, request delay and survivability; that is, network connectivity. In this article, to guarantee an efficient quality of service and deal with the increased delay problem, we investigate the problem of joint optimization of average request delay and survivability in the proposed SD‐WAN model. The optimization approaches are formulated so as to minimize the average request delay and maximize the network connectivity. Subsequently, we adopt a multi‐agent deep Q‐Network algorithm to solve it, where the reward function is reformulated based on the optimization objective. Simulation results show that our strategy improves the system's performance significantly in terms of request delay and survivability, compared to traditional baseline algorithms.

Evaluating Scalability, Resiliency, and Load Balancing in Software-Defined Networking

“…In [2], the authors explore machine learning techniques applied in SDN. The authors optimize delay and connectivity for SD-WAN environments using a multi-agent deep reinforcement learning algorithm.…”

Energy-Aware MPTCP Scheduling in Heterogeneous Wireless Networks Using Multi-Agent Deep Reinforcement Learning Techniques

“…Similarly, [22], implemented an RL-based strategy to optimize data scheduling in MPTCP, but the proposal primarily focuses on improving throughput, leaving out the critical factors such as energy consumption and latency. Also, the work in the studies in [24,25], proposed a Deep Reinforcement Learning (DRL) based technique predominantly concentrating on single-agent scenarios. The ReLes scheme [26], was the first to apply Deep Reinforcement Learning (DRL) to solve the scheduling problem in multi-path TCP (MPTCP).…”

Flow Table Saturation Attack against Dynamic Timeout Mechanisms in SDN

“…These features also promote novel security solutions, e.g., stateful firewalls [3], dynamic access control [4], and suspicious traffic redirection [5]. The dynamic reconfiguration and centralized management capabilities of SDN have also been applied to various scenarios, such as IoT [6], cloud, and WAN [7], in recent years.…”