ADAM & RAL: Adaptive Memory Learning and Reinforcement Active Learning for Network Monitoring

Abstract

Network-traffic data commonly arrives in the form of fast data streams; online network-monitoring systems continuously analyze these kinds of streams, sequentially collecting measurements over time. Continuous and dynamic learning is an effective learning strategy when operating in these fast and dynamic environments, where concept drifts constantly occur. In this paper, we propose different approaches for stream-based machine learning, able to analyze network-traffic streams on the fly, using supervised learning techniques. We address two major challenges associated to stream-based machine learning and online network monitoring: (i) how to dynamically learn from and adapt to non-stationary data and patterns changing over time, and (ii) how to deal with the limited availability of ground truth or labeled data to continuously tune a supervised learning model. We introduce ADAM & RAL, two stream-based machine-learning approaches to tackle these challenges. ADAM implements multiple stream-based machine-learning models and relies on an adaptive memory strategy to dynamically adapt the size of the system's learning memory to the most recent data distribution, triggering new learning steps when concept drifts are detected. RAL implements a stream-based active-learning strategy to reduce the amount of labeled data needed for streambased learning, dynamically deciding on the most informative samples to integrate into the continuous learning scheme. Using a reinforcement learning loop, RAL improves prediction performance by additionally learning from the goodness of its previous sample-selection decisions. We focus on a particularly challenging problem in network monitoring: continuously tuning detection models able to recognize network attacks over time. By continuously learning from and detecting concept drifts within real network measurements, we show that ADAM & RAL can continuously achieve high detection accuracy and limit the amount of training data needed to detect attacks over dynamic network data streams.

Adaptive and Reinforcement Learning Approaches for Online Network Monitoring and Analysis

“…This paper builds on and extends our recent work on adaptive learning for network monitoring [4], in multiple directions. In particular: (i) it brings a more comprehensive overview on the state of the art in adaptive learning; (ii) it provides an extended evaluation of ADAM for different types of attacks, as well as a comparative analysis against other adaptation strategies, besides evaluating non-adaptive learning approaches; (iii) it develops a theoretical analysis on the expected performance of RAL, in particular with respect to the implemented reinforcement-learning policy; (iv) it further evaluates RAL in other datasets, to show the general advantages of the proposal; (v) last but not least, it presents (and evaluates) the integration of both ADAM and RAL into a single, reinforcement-based, adaptive active-learning system, adding explicit concept-drift detection into RAL.…”

Active Learning for Network Traffic Classification: A Technical Study

“…AL is a promising technique to alleviate the challenge of streaming-based learning scenarios [66], [67]. AL algorithms designed for streaming scenarios can control the labeling process and gradually perform this process over time [68]. Using this strategy, it is expected that the labeling process will be in balance and the algorithms will detect the changes.…”

“…Wassermann et al in [86] examine two central issues in stream-based ML and online network monitoring, i.e., 1) how to learn in dynamic environments in the presence of concept drifts, and 2) how to learn with a small number of labeled data and, regularly improve a supervised model through new samples. To deal with these issues, the authors propose two stream-based ML algorithms, namely ADAM and Reinforcement AL (RAL).…”

Active Learning for Network Traffic Classification: A Technical Study

“…AL is a promising technique to alleviate the challenge of streaming-based learning scenarios [67], [68]. AL algorithms designed for streaming scenarios can control the labeling process and gradually perform this process over time [69]. Using this strategy, it is expected that the labeling process will be in balance and the algorithms will detect the changes.…”

“…Wassermann et al in [87] examine two central issues in stream-based ML and online network monitoring, i.e. 1) how to learn in dynamic environments in the presence of concept drifts, and 2) how to learn with a small number of labeled data and, regularly improve a supervised model though new samples.…”