Peekaboo: Learning-Based Multipath Scheduling for Dynamic Heterogeneous Environments

Abstract

Multipath transport protocols utilize multiple network paths (e.g., WiFi and cellular) to achieve improved performance and reliability, compared with their single-path counterparts. The scheduler of a multipath transport protocol determines how to distribute the data packets onto different paths. However, state-of-the-art multipath schedulers face the challenge when dealing with heterogeneous paths with dynamic path characteristics (i.e., packet loss, fluctuation of delay). In this paper, we propose Peekaboo, a novel learning-based multipath scheduler that is aware of the dynamic characteristics of the heterogeneous paths. Peekaboo is able to learn scheduling decisions to adopt over time based on the current path characteristics and dynamicity levels -from both deterministic and stochastic perspectives. We implement Peekaboo in Multipath QUIC (MPQUIC) and compare it with state-of-the-art multipath schedulers for a wide range of dynamic heterogeneous environments, upon both emulated and real networks. Our results show that Peekaboo outperforms the other schedulers by up to 31.2% in emulated networks and up to 36.3% in real network scenarios.

A Survey of Multipath TCP Scheduling Schemes: Open Challenges and Potential Enablers

“…Figure 2: Multipath scenario should consider the path heterogeneity [49] subflows to control the data transfer rate at each subflow [27]. A congestion control should satisfy three rules.…”

“…The main concern is the complexity of the optimization function which may increase the optimization time in complex scheduling scenarios. Wu et al [49] proposed Peekaboo, a novel learning-based multipath QUIC (MPQUIC) scheduler in that keeps monitoring the impact caused by the current dynamicity level of each path and selects the most suitable scheduling strategy accordingly. According to the reward function, paths with highest throughput are selected.…”

“…Objective Lim et al [24] Heuristics Earliest Completion First Pokhrel et al [31] Heuristics throughput-based Chung et al [10] ML predictive QoS Beig et al [7] Heuristics throughput-based Pokhrel and Mandjes [37] Q-Learning QoS Chiariotti et al [9] Heuristics QoS Zhang et al [52] RL QoS Wu et al [49] RL highest throughput Pokhrel and Williamson [42] game theory throughput and responsiveness Huang et al [18] A distributed (DRL) loss and delay Pokhrel and Garg [35] DRL QoS Abbasloo et al [1] DRL delays Pokhrel and Choi [30] Federated learning (BFL) privacy-based Table 3.1 provides general comparison between some of heuristics and learning-based MPTCP scheduling techniques.…”

Guest Editorial Special Issue on Advances in Artificial Intelligence and Machine Learning for Networking

“…Using measurements from a private testbed and traces from several commercial networks, the authors demonstrate that the cache size can be small while still allowing for accurate estimation. Finally, in "Peekaboo: Learning-based Multipath Scheduling for Dynamic Heterogeneous Environments," Wu et al present an adaptive multipath scheduling algorithm named Peekaboo [7]. This algorithm learns and adopts scheduling decisions based on the changing characteristics of the heterogeneous paths and is effective specifically in wireless networks.…”

“…Regarding specific machine-learning methods that are introduced to address networking problems, many papers propose reinforcement learning techniques [2], [3], [7], [10]- [13], all of which, except [11], use model-free reinforcement learning. More specifically, [10], [12], and [13] apply a value-based approach, often referred to as Q-learning, while [2] and [3] follow a policy-based Approach; [7] applies a multiarmed bandit model as the basis for reinforcement learning, while all other above-referenced papers rely on Markov Decision Processes (MDPs).…”