Load Balancing Optimization in Software-Defined Wide Area Networking (SD-WAN) using Deep Reinforcement Learning

Abstract

Software-Defined Wide Area Network (SD-WAN) holds tremendous potential to provide multi-cloud multi-network interconnection and prevent channel congestion. However, traffic among Customer Premises Edge (CPE) and controllers continuously increases, requiring pre-emptive load balancing in the control plane. In this paper problem in SD-WANs when the controller presents a limited processing capacity. Specifically, the data plane may include one or more CPE deployed at a site where service traffic is forwarded. To address this narrow, we propose a new approach based on a Deep Reinforcement Learning (DRL) strategy to optimize the balancing process under a latency constraint. As far as we can tell, we have not observed any pertinent research published in this context. The obtained simulation results revealed that our proposed approach decreases the load balancing and outperforms other baseline methods.

A Comprehensive Survey of In-Band Control in SDN: Challenges and Opportunities

“…They present a new routing algorithm that they implemented in a real SDN network with better results than other routing algorithms, such as OSPF and Least Loaded (LL). Ouamri et al [111] provides another example of using ML on a WAN SDN for load balancing optimisation, which could also be used for in-band backup links. ML should also be used to improve network resilience by providing some alternatives or backup paths for the in-band links.…”

Software defined wireless sensor load balancing routing for internet of things applications: Review of approaches

“…To address this challenge, authors in Ref. [ 109 ] presented DRL in Software Defined -Wide Area Network (SD-WAN). Their solution can potentially balance the load on network resources with minimal delay and better network sustainability.…”

“… [ 108 ] Application-specific Routing Reinforcement Learning Software Defined Wireless Sensor Networks Improved network operational lifetime and response time May not perform well in a large-scale dynamic network environment [ 110 ] Quality of Service (QoS) enabled load balance Energy-aware routing Markov Decision Process (MDP) and Q-learning Software energy internet Improved load variation and average waiting time It may have lower robustness to the burst traffic in large-scale dynamic networks. [ 109 ] Load balancing due to congestion Deep reinforcement learning (DRL) SD-WAN Optimize delay and improved network life time Focused on balancing the load on the controller only without incorporating the data forwarding element …”

Request delay and survivability optimization for software defined‐wide area networking (SD‐WAN) using multi‐agent deep reinforcement learning

“…For most problems, DQN is able to optimize strategies with high dimensional input data by forming a network, which can deal with the dimensional problem caused by the state and action spaces. 42,43 However, the state-action is explored by a series of soft technologies such as 𝜀-greedy. In DQN, there are two types of approximators: a linear function or a non-linear function.…”

“…Deep Q‐Network (DQN) is an RL algorithm that combine Q‐learning with a conventional neural network to generate a target Q‐value. For most problems, DQN is able to optimize strategies with high dimensional input data by forming a network, which can deal with the dimensional problem caused by the state and action spaces 42,43 . However, the state‐action is explored by a series of soft technologies such as $$$ \varepsilon $$$‐greedy.…”