Adaptive and Reinforcement Learning Approaches for Online Network Monitoring and Analysis

Abstract

Network-monitoring data commonly arrives in the form of fast and changing data streams. Continuous and dynamic learning is an effective learning strategy when dealing with such data, where concept drifts constantly occur. We propose different stream-based, adaptive learning approaches to analyze networktraffic streams on the fly. We address two major challenges associated to stream-based machine learning and online network monitoring: (i) how to dynamically learn from and adapt to non-stationary data changing over time, and (ii) how to deal with the limited availability of labeled data to continuously tune a supervised-learning model. We introduce ADAM & RAL, two stream-based machine-learning techniques to tackle these challenges. ADAM relies on adaptive memory strategies to dynamically tune stream-based learning models to changes in the input data distribution. RAL combines reinforcement learning with stream-based active-learning to reduce the amount of labeled data needed for continual learning, dynamically deciding on the most informative samples to learn from. We apply ADAM & RAL to the real-time detection of network attacks in Internet network traffic, and show that it is possible to continuously achieve high detection accuracy even under the occurrence of concept drifts, limiting the amount of labeled data needed for learning.

Reinforcement Learning for Intrusion Detection: More Model Longness and Fewer Updates

“…In contrast to traditional ML techniques, reinforcement learning (RL) aims to find an optimal learning policy strategy during the model-building phase [18]. RL-based techniques are performed through an entity called the agent.…”

“…Traditional ML relies upon an input and output pair, where an ML model receives a given event feature vector as input and outputs its estimated class value. RL-based approaches are significantly different from traditional ML-based ones [18]. RL-based techniques are not given the event class value [3].…”

IBAC: An Intelligent Dynamic Bandwidth Channel Access Avoiding Outside Warning Range Problem

“…Then, on the basis of the estimated throughput, we apply the Thompson sampling (TS) [33] based Bayesian approach to find the best DBCA channel bonding level intelligently. TS has several advantages -(a) it is a reinforcement learning algorithm which is suitable for problems demanding online decisions [34]- [36], (b) it generates the probability distribution of the success rate of applying an action based on it knowledge of exploiting the action in the past, and (c) in order to maximize the cumulative rewards obtained by applying actions, the application of exploration and exploitation leads to the learning of a new environment and utilizing the best past knowledge as well. In TS, the actions are chosen sequentially by a manner such that a balance is maintained between decisions exploring new information to improve future performance and decisions exploiting the past knowledge to maximize present performance.…”

An Incremental Learning Method Based on Dynamic Ensemble RVM for Intrusion Detection

“…Cui is with the School of Computer Science, University of Technology Sydney, Sydney, Australia use individual classifier with incremental learning capability to update its structure when new data arrives. Unfortunately, concept drift is prevalent in dynamic streaming environment such as intrusion detection systems (IDS), in where the model performance of incremental learning may degrade [4], [5], [6]. Specifically, when an unknown intrusion occurs in the network, the current data distribution will change dynamically.…”