Internet Congestion Control via Deep Reinforcement Learning

Jay, Nathan; Rotman, Noga H.; Godfrey, P. Brighten; Schapira, Michael; Tamar, Aviv

doi:10.48550/arxiv.1810.03259

Cited by 5 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These solutions, called Deep RL, have achieved remarkable results matching humans at playing Atari games [37], and beating the Go world champion [46]. These results have encouraged researchers to apply Deep RL to networking and systems problems, from routing, to congestion control, to video streaming, and to job scheduling [4,6,16,34,35,54,62,64,65]. Building a decision tree can be easily cast as an RL problem: the environment's state is the current decision tree, an action is either cutting a node or partitioning a set of rules, and the reward is either the classification time, memory footprint, or a combination of the two.…”

Section: How To Learn?mentioning

confidence: 99%

“…In particular, our approach learns to optimize packet classification for a given set of rules and objective, can easily incorporate pre-engineered heuristics to leverage their domain knowledge, and does so with little human involvement. The recent successes of deep learning in solving notoriously hard problems, such as image recognition [23] and language translation [51], have inspired many practitioners and researchers to apply deep learning, in particular, and machine learning, in general, to systems and networking problems [4,6,16,34,35,54,62,64,65]. While in some of these cases there are legitimate concerns about whether machine learning is the right solution for the problem at hand, we believe that deep learning is a good fit for our problem.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural packet classification

Liang

Zhu

Jin

et al. 2019

Proceedings of the ACM Special Interest Group on Data Communication

106

View full text Add to dashboard Cite

Packet classification is a fundamental problem in computer networking. This problem exposes a hard tradeoff between the computation and state complexity, which makes it particularly challenging. To navigate this tradeoff, existing solutions rely on complex hand-tuned heuristics, which are brittle and hard to optimize.In this paper, we propose a deep reinforcement learning (RL) approach to solve the packet classification problem. There are several characteristics that make this problem a good fit for Deep RL. First, many existing solutions iteratively build a decision tree by splitting nodes in the tree. Second, the effects of these actions (e.g., splitting nodes) can only be evaluated once the entire tree is built. These two characteristics are naturally captured by the ability of RL to take actions that have sparse and delayed rewards. Third, it is computationally efficient to generate data traces and evaluate decision trees, which alleviate the notoriously high sample complexity problem of Deep RL algorithms. Our solution, NeuroCuts, uses succinct representations to encode state and action space, and efficiently explore candidate decision trees to optimize for a global objective. It produces compact decision trees optimized for a specific set of rules and a given performance metric, such as classification time, memory footprint, or a combination of the two. Evaluation on ClassBench shows that NeuroCuts outperforms existing hand-crafted algorithms in classification time by 18% at the median, and reduces both classification time and memory footprint by up to 3×.

show abstract

Section: How To Learn?mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Neural packet classification

Liang

Zhu

Jin

et al. 2019

Proceedings of the ACM Special Interest Group on Data Communication

106

View full text Add to dashboard Cite

show abstract

“…ML techniques for networking. Recent works suggest using ML techniques for solving networking problems such as TCP congestion control [3,11,40], resource management [23], quality of experience in video streaming [24,38], routing [35], and decision tree optimization for packet classification [20]. NuevoMatch is different in that it uses an ML technique for building space-efficient representations of the rules that fit in the CPU cache.…”

Section: Gpus For Classification Accelerating Classification Onmentioning

confidence: 99%

A Computational Approach to Packet Classification

Rashelbach,

Rottenstreich,

Silberstein

2020

Preprint

View full text Add to dashboard Cite

Multi-field packet classification is a crucial component in modern software-defined data center networks. To achieve high throughput and low latency, state-of-the-art algorithms strive to fit the rule lookup data structures into ondie caches; however, they do not scale well with the number of rules.We present a novel approach, NuevoMatch, which improves the memory scaling of existing methods. A new data structure, Range Query Recursive Model Index (RQ-RMI), is the key component that enables NuevoMatch to replace most of the accesses to main memory with model inference computations. We describe an efficient training algorithm which guarantees the correctness of the RQ-RMI-based classification. The use of RQ-RMI allows the packet rules to be compressed into model weights that fit into the hardware cache and takes advantage of the growing support for fast neural network processing in modern CPUs, such as wide vector processing engines, achieving a rate of tens of nanoseconds per lookup.Our evaluation using 500K multi-field rules from the standard ClassBench benchmark shows a geomean compression factor of 4.9×, 8×, and 82×, and an average performance improvement of 2.7×, 4.4× and 2.6× in latency and 1.3×, 2.2×, and 1.2× in throughput compared to CutSplit, NeuroCuts, and TupleMerge, all state-of-the-art algorithms 1 .

show abstract

“…Authors in [2] explore designing a TCP-style congestion control algorithm using Q-learning. The recent eort [31] proposes a DRL-based adaptive framework for congestion control. In this model, state is bounded histories of network statistics (e.g., sending rate, latency), action is periodically tuning the sending rates, and the reward is a linear function that rewards throughput while penalizing loss and latency.…”

Section: Related Workmentioning

confidence: 99%

Towards A Machine Learning-Based Framework For Automated Design of Networking Protocols

Pasandi

2019

2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)

View full text Add to dashboard Cite

Networking protocols are designed through long-time and hard-work human eorts. Machine Learning (ML)-based solutions have been developed for communication protocol design to avoid manual eorts to tune individual protocol parameters. While other proposed ML-based methods mainly focus on tuning individual protocol parameters (e.g., adjusting contention window), our main contribution is to propose a novel Deep Reinforcement Learning (DRL)-based framework to systematically design and evaluate networking protocols. We decouple a protocol into a set of parametric modules, each representing a main protocol functionality that is used as DRL input to better understand the generated protocols design optimization and analyze them in a systematic fashion. As a case study, we introduce and evaluate Deep-MAC a framework in which a MAC protocol is decoupled into a set of blocks across popular avors of 802.11 WLANs (e.g., 802.11 b/a/g/n/ac). We are interested to see what blocks are selected by DeepMAC across dierent networking scenarios and whether DeepMAC is able to adapt to network dynamics.

show abstract

Internet Congestion Control via Deep Reinforcement Learning

Cited by 5 publications

References 0 publications

Neural packet classification

Neural packet classification

A Computational Approach to Packet Classification

Towards A Machine Learning-Based Framework For Automated Design of Networking Protocols

Contact Info

Product

Resources

About