Constrained Deep Reinforcement Learning for Smart Load Balancing

Houidi, Omar; Zeghlache, Djamal; Perrier, Victor; Quang, Pham Tran Anh; Huin, Nicolas; Leguay, Jérémie; Medagliani, Paolo

doi:10.1109/ccnc49033.2022.9700657

Cited by 11 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their approach selected the optimal bitrate based on user preferences, network throughput, and buffer occupancy. Similarly, Omar Houidi et al [25] adjusted the actor-critic architecture for DRL to consider Quality of Service (QoS) and balance network traffic, thereby maximizing QoE.…”

Section: Deep Reinforcement Learning-based Approachesmentioning

confidence: 99%

Deep Reinforcement Learning-Based Approach for Video Streaming DASH

Souane,

Bourenane,

Douga

2023

Preprint

View full text Add to dashboard Cite

Dynamic adaptive video streaming over HTTP (DASH) plays a crucial role in video transmission across networks. Traditional adaptive bitrate (ABR) algorithms adjust the quality of video segments based on network conditions and buffer occupancy. However, these algorithms rely on fixed rules within a complex environment, making it challenging to achieve optimal decisions considering the overall context. In this paper, we propose a novel Deep Reinforcement Learning-based approach for streaming DASH, focusing on maintaining consistent perceived video quality throughout the streaming session to enhance user experience. Our approach optimizes the Quality of Experience (QoE) by dynamically controlling the quality distance factor between consecutive video segments. We evaluate this approach through a simulation model that encompasses diverse wireless network environments and various video sequences. Additionally, we compare our proposed approach with state-of-the-art methods. The experimental results demonstrate significant improvements in QoE, ensuring users enjoy stable, high-quality video streaming sessions.

show abstract

Section: Deep Reinforcement Learning-based Approachesmentioning

confidence: 99%

Deep Reinforcement Learning-Based Approach for Video Streaming DASH

Souane,

Bourenane,

Douga

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Rosello [12] proposed a DQN agent with the purpose of selecting the optimal paths for MPTCP, while Liao et al [13] used an actor-critic framework to the same end. Finally, Houdi et al [14] proposed a multi-agent actor-critic framework to perform path selection and optimize quality of experience.…”

Section: Related Workmentioning

confidence: 99%

Graph Convolutional Reinforcement Learning for Collaborative Queuing Agents

Fawaz¹,

Lesca²,

Quang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we explore the use of multi-agent deep learning as well as learning to cooperate principles to meet stringent service level agreements, in terms of throughput and end-to-end delay, for a set of classified network flows. We consider agents built on top of a weighted fair queuing algorithm that continuously set weights for three flow groups: gold, silver, and bronze. We rely on a novel graph-convolution based, multi-agent reinforcement learning approach known as DGN. As benchmarks, we propose centralized and distributed deep Q-network approaches and evaluate their performances in different network, traffic, and routing scenarios, highlighting the effectiveness of our proposals and the importance of agent cooperation. We show that our DGN-based approach meets stringent throughput and delay requirements across all scenarios.

show abstract

“…This paper adapts QMIX [21]which achieves similar performance as in [27]-in an asynchronous mechanism to make highly frequent load balancing decisions, and to learn from partial observations and solve the network load balancing problem as a cooperative game. RL-based algorithm has also been applied on other load balancing problems [3,14,16,19]. However, the network load balancing problem studied in this paper is different from link load balancing problems studied in [14,16], where link utilisation is to be maximised and load balancers have observations on link utilisations.…”

Section: Related Workmentioning

confidence: 99%

“…RL-based algorithm has also been applied on other load balancing problems [3,14,16,19]. However, the network load balancing problem studied in this paper is different from link load balancing problems studied in [14,16], where link utilisation is to be maximised and load balancers have observations on link utilisations. As discussed in [32], in network load balancing problems-more precisely, Layer-4 server load balancing problem is studied in this paper-load balancers have no direct observation on server utilisations.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

Yao

Zihan

Clausen

2022

Proceedings of the 31st ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

This paper presents the network load balancing problem, a challenging real-world task for multi-agent reinforcement learning (MARL) methods. Conventional heuristic solutions like Weighted-Cost Multi-Path (WCMP) and Local Shortest Queue (LSQ) are less flexible to the changing workload distributions and arrival rates, with a poor balance among multiple load balancers. The cooperative network load balancing task is formulated as a Dec-POMDP problem, which naturally induces the MARL methods. To bridge the reality gap for applying learning-based methods, all models are directly trained and evaluated on a real-world system from moderateto large-scale setups. Experimental evaluations show that the independent and "selfish" load balancing strategies are not necessarily the globally optimal ones, while the proposed MARL solution has a superior performance over different realistic settings. Additionally, the potential difficulties of the application and deployment of MARL methods for network load balancing are analysed, which helps draw the attention of the learning and network communities to such challenges.

show abstract

Constrained Deep Reinforcement Learning for Smart Load Balancing

Cited by 11 publications

References 12 publications

Deep Reinforcement Learning-Based Approach for Video Streaming DASH

Deep Reinforcement Learning-Based Approach for Video Streaming DASH

Graph Convolutional Reinforcement Learning for Collaborative Queuing Agents

Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

Contact Info

Product

Resources

About