Performance Analysis of Active Queue Management Algorithm Based on Reinforcement Learning

Abstract

In the information society, data explosion has led to more congestion in the core network, dampening the network performance. Random early detection (RED) is currently the standard algorithm for active queue management (AQM) recommended by the Internet Engineering Task Force (IETF). However, RED is particularly sensitive to both service load and algorithm parameters. The algorithm cannot fully utilize the bandwidth at a low service load, and might suffer a long delay at a high service load. This paper designs the reinforcement learning AQM (RLAQM), a simple and practical variant of RED, which controls the average queue length to the predictable value under various network loads, such that the queue size is no longer sensitive to the level of congestion. Q-learning was adopted to adjust the maximum discarding probability, and derive the optimal control strategy. Simulation results indicate that RLAQM can effectively overcome the deficiency of RED and achieve better congestion control; RLAQM improves the network stability and performance in complex environment; it is very easy to migrate from RED to RLAQM on Internet routers: the only operation is to adjust the discarding probability.

Queue Management Algorithm for Satellite Networks Based on Traffic Prediction

“…The random-drop and the drop-front algorithms are derived subsequently to effectively avoid the deadlock problem, but the full queue and global synchronisation problems remain unaddressed. Later, proposed the use of active queue management algorithms [8], which are represented by the random early detection (RED) algorithm, the random exponential marking (REM) algorithm based on optimisation theory, and the ARED algorithm. The RED algorithm proposed by Floyd et al is the earliest version of the active queue management algorithm; this algorithm greatly improves the operating quality of the network but is highly sensitive to parameter settings [9].…”

“…The smoothing coefficient and parameters of each time point in the dynamic triple exponential smoothing model change dynamically, requiring iterative calculation in each period. The smoothing value of the data is calculated first, and then the optimal value * in Algorithm 1 is designated the dynamic smoothing coefficient α m,n , which is substituted into Equations ( 7), (8), and ( 9) to obtain the predicted value using Equation (6). The process of traffic prediction performed with the optimised triple exponential smoothing model is shown in Table 2, where m is the predicted traffic data and n is the time point of the data set prediction.…”